Electronic device and method for visal disortion compensation
By using processors to detect angles and user gaze, the electronic device corrects visual distortions in flexible displays, maintaining a consistent image across varying states.
Patent Information
- Application Number
- KR1020220115765
- Authority / Receiving Office
- KR · KR
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2022-08-03
- Filing Date
- 2022-09-14
- Publication Date
- 2026-11-11
- Estimated Expiration
- 2042-09-14
AI Technical Summary
Flexible displays in electronic devices experience visual distortion due to changes in their state and the user's line of sight, leading to an uneven perception of the screen.
The electronic device employs a processor to detect the angle between housing surfaces using sensors, calculates user gaze information, and adjusts display coordinates to correct visual distortion by generating an undistorted image based on folding and gaze information.
The solution effectively maintains a consistent image view across different device states by correcting visual distortions, ensuring a uniform display regardless of the device's configuration.
Smart Images

Figure 112022096417230-PAT00040_ABST
Abstract
Description
Technology Field
[0001] The following descriptions relate to an electronic device and method for correcting visual distortion caused by a change in the state of a display in a flexible display. Background Technology
[0003] An electronic device including a flexible display may have a changing state. Depending on the change in the state of the flexible display and the user's line of sight, the screen displayed on the flexible display may be visually distorted. A method for correcting visual distortion of the screen is described below. means of solving the problem
[0005] According to one embodiment, an electronic device may include a first housing comprising a first surface and a second surface opposite to the first surface. The electronic device may include a second housing comprising a third surface and a fourth surface opposite to the third surface. The electronic device may include at least one processor, at least one camera, at least one distance sensing sensor, and a flexible display. The electronic device may include a hinge structure configured to provide an unfolded state in which the first surface and the third surface face in the same direction, a folded state in which the first surface and the third surface face each other, or an intermediate state in which the first surface and the third surface form an angle between the angle in the unfolded state and the angle in the folded state, by rotatably connecting the first housing and the second housing with respect to a folding axis. The electronic device may include a folding sensing sensor for detecting an angle between the first surface and the third surface of the flexible display. The at least one processor can display a first image on a first display area of the flexible display in the unfolded state. The at least one processor can obtain first coordinate information of the first display area of the flexible display corresponding to the unfolded state. The at least one processor can obtain folding information corresponding to the angle between the first surface and the third surface of the flexible display based on detecting the intermediate state. The at least one processor can obtain user gaze information based on the at least one distance sensing sensor and the at least one camera. Based on the user gaze information, the folding information, and the first coordinate information of the first display area, the second coordinate information of the second display area corresponding to the intermediate state can be obtained.The at least one processor can generate a second image based on the first image and the second coordinate information of the second display area. The at least one processor can display the second image in the second display area of the flexible display in the intermediate state.
[0006] A method performed by an electronic device according to one embodiment may include an operation of displaying a first image on a first display area of a flexible display in an unfolded state. The method may include an operation of acquiring first coordinate information of the first display area of the flexible display corresponding to the unfolded state. The method may include an operation of acquiring folding information corresponding to an angle between a first surface and a third surface of the flexible display based on detecting the intermediate state. The method may include an operation of acquiring user gaze information based on at least one distance sensing sensor and at least one camera. The method may include an operation of acquiring second coordinate information of a second display area corresponding to the intermediate state based on the user gaze information, the folding information, and the first coordinate information of the first display area. The method may include an operation of generating a second image based on the first image and the second coordinate information of the second display area. The method may include an operation of displaying the second image on the second display area of the flexible display in the intermediate state. Brief explanation of the drawing
[0008] FIG. 1 is a block diagram of an electronic device in a network environment according to one embodiment. FIG. 2 illustrates an example of an unfolded state of an electronic device according to one embodiment. FIG. 3 illustrates an example of a folding state of an electronic device according to one embodiment. FIG. 4 illustrates an example of distortion correction of a display according to one embodiment. FIG. 5a illustrates an example of distortion correction performed based on a user's line of sight located on the central axis of a display, according to one embodiment. FIG. 5b illustrates an example of x-axis distortion correction performed based on a user's line of sight located on the central axis of a display, according to one embodiment. FIG. 5c illustrates an example of y-axis distortion correction performed based on a user's line of sight located on the central axis of a display, according to one embodiment. FIG. 5d illustrates an example of a distortion-corrected screen performed based on a user's line of sight located on the central axis of a display, according to one embodiment. FIG. 6a illustrates an example of distortion correction performed based on a projection plane having the same angle with respect to the first housing and the second housing, according to one embodiment. FIG. 6b illustrates an example of x-axis distortion correction performed based on a projection plane having the same angle with the first housing and the second housing, according to one embodiment. FIG. 6c illustrates an example of a distortion-corrected screen based on a projection plane that has the same angle with the first housing and the second housing, according to one embodiment. FIG. 7a illustrates an example of distortion correction of an electronic device based on a projection plane perpendicular to the user's line of sight, according to one embodiment. FIG. 7b illustrates an example of x-axis distortion correction performed based on a projection plane perpendicular to the user's line of sight, according to one embodiment. FIG. 7c illustrates an example of a distortion-corrected screen performed based on a projection plane perpendicular to the user's line of sight, according to one embodiment. FIG. 8 illustrates an example of margin utilization by distortion correction for displaying information according to one embodiment. Figure 9 illustrates an example of image loss that may occur during distortion correction. FIG. 10 illustrates an example of a notification for indicating image loss during distortion correction according to one embodiment. FIG. 11 illustrates an example of image reduction during distortion correction according to one embodiment. FIG. 12 illustrates an example of image enlargement during distortion correction according to one embodiment. FIG. 13 illustrates an example of distortion correction applied to only a portion of the display area according to one embodiment. FIG. 14 illustrates an example of selecting a display area to apply distortion correction according to one embodiment. FIG. 15 illustrates an example of whether distortion correction is changed based on a change in the user's gaze according to one embodiment. FIG. 16 illustrates the flow of an operation for displaying a distortion-corrected image based on user gaze information, folding information, and a first display area, according to one embodiment. Specific details for implementing the invention
[0009] The terms used in this disclosure are used merely to describe specific embodiments and are not intended to limit the scope of other embodiments. A singular expression may include a plural expression unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as generally understood by those skilled in the art described in this disclosure. Terms used in this disclosure that are defined in a general dictionary may be interpreted as having the same or similar meaning as they have in the context of the relevant technology, and are not to be interpreted in an ideal or overly formal sense unless explicitly defined in this disclosure. In some cases, even terms defined in this disclosure are not to be interpreted to exclude the embodiments of this disclosure.
[0010] In the various embodiments of the present disclosure described below, a hardware-based approach is described as an example. However, since the various embodiments of the present disclosure include techniques using both hardware and software, the various embodiments of the present disclosure do not exclude a software-based approach.
[0011] Terms referring to distortion compensation used in the following description (e.g., distortion compensation, changing, switching), terms referring to the first display area (e.g., first display area, display area before distortion compensation, display area displaying the first image, display area displaying image before distortion compensation), terms referring to the second display area (display area displaying after distortion compensation, display area displaying the second image, display area displaying image after distortion compensation), terms referring to the image before distortion compensation (the first image, original image), terms referring to the image after distortion compensation (the second image, compensation image, changing image, switching Terms referring to the margin part of the display (e.g., margin part, surplus part), etc., are examples provided for the convenience of explanation.Accordingly, the present disclosure is not limited to the terms described below, and other terms having equivalent technical meanings may be used. Additionally, terms such as '...part', '...device', '...body' used below may refer to at least one shape structure or a unit that processes a function.
[0012] Additionally, in this disclosure, expressions of "greater than" or "less than" may be used to determine whether a specific condition is satisfied or fulfilled; however, this is merely for the purpose of expressing an example and does not exclude descriptions of "greater than" or "less than." Conditions described as "greater than" may be replaced with "greater than," conditions described as "less than" may be replaced with "less than," and conditions described as "greater than and less than" may be replaced with "greater than and less than." Furthermore, "A" to "B" below refer to at least one of the elements from A (including A) to B (including B).
[0013] Before describing the embodiments of the present disclosure, terms necessary to describe the operations of the electronic device according to the embodiments are defined. A foldable electronic device may refer to an electronic device comprising a hinge structure configured to provide a folding state, an unfolded state, and an intermediate state. The foldable electronic device may include a flexible display. The foldable electronic device may include a first housing comprising a first surface and a second surface opposite to the first surface. The foldable electronic device may include a second housing comprising a third surface and a fourth surface. The unfolded state of the foldable electronic device may refer to a structure of the electronic device that is fully unfolded such that the first surface and the third surface face the same direction. The folding state of the foldable electronic device may refer to a structure of the electronic device that is folded so that the first surface and the third surface face each other. The intermediate state may refer to a structure of the electronic device that is folded such that the first surface and the third surface form an angle between the angle in the unfolded state and the angle in the folding state. The first display area may refer to the display area of the display in the unfolded state. The first display area may refer to the display area of the display before distortion correction due to perspective. The second display area may refer to the display area of the display after distortion correction due to perspective. The first coordinate information may refer to coordinates that can specify the first display area. For example, the first coordinate information may refer to the coordinates of the vertices of the first display area and the coordinates of the ends of the folding axis. The second coordinate information may refer to coordinates that can specify the second display area. For example, the second coordinate information may refer to the coordinates of the vertices of the second display area and the coordinates of the ends of the folding axis. The folding axis may refer to the axis along which the first housing and the second housing are folded.The central axis may refer to a plane that is perpendicular to the flexible display in the unfolded state and includes the folding axis. The user's gaze may refer to a point corresponding to the pupil of the user observing the display. According to one embodiment, the user's gaze may refer to a point corresponding to the pupil on the side the user primarily uses. According to one embodiment, the user's gaze may refer to a point corresponding to the pupil on the side designated by the user. According to one embodiment, the user's gaze may refer to a point corresponding to the pupil on the side closer to the display. As the distance from the display increases, the degree of change in the display area for distortion correction becomes greater, which may result in a large amount of blank space. According to one embodiment, the user's gaze may refer to a point corresponding to the center of both pupils of the user. User gaze information may include the coordinates of the user's gaze.
[0015] FIG. 1 is a block diagram of an electronic device in a network environment according to embodiments.
[0016] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through a server (108). According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display module (160), audio module (170), sensor module (176), interface (177), connection terminal (178), haptic module (179), camera module (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (176), camera module (180), or antenna module (197)) may be integrated into a single component (e.g., display module (160)).
[0017] The processor (120) can control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., a program (140)), for example, and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in volatile memory (132), process the commands or data stored in volatile memory (132), and store the resulting data in non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use lower power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.
[0018] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display module (160), sensor module (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.
[0019] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor module (176)). The data may include, for example, software (e.g., program (140)) and input or output data for related commands. The memory (130) may include volatile memory (132) or non-volatile memory (134).
[0020] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).
[0021] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).
[0022] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.
[0023] The display module (160) can visually provide information to an external (e.g., user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.
[0024] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).
[0025] The sensor module (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.
[0026] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.
[0027] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).
[0028] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that can be perceived by the user through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.
[0029] The camera module (180) can capture still images and video. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.
[0030] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).
[0031] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.
[0032] The communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication processors that operate independently of the processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).
[0033] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one embodiment, the wireless communication module (192) may support a Peak data rate (e.g., 20 Gbps or more) for eMBB realization, loss coverage (e.g., 164 dB or less) for mMTC realization, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for URLLC realization.
[0034] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).
[0035] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.
[0036] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.
[0037] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104, or 108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (101) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or neural networks. According to one embodiment, the external electronic device (104) or the server (108) may be included within a second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.
[0039] FIG. 2 illustrates an example of an unfolded state of an electronic device according to embodiments.
[0040] FIG. 3 illustrates an example of a folding state of an electronic device according to embodiments.
[0041] According to one embodiment, a foldable electronic device (e.g., the electronic device (101) of FIG. 1) may provide various states. For example, the electronic device (101) may provide an unfolded state, an intermediate state, and a folded state.
[0042] Referring to FIG. 2, the electronic device (101) may be in an unfolded state (200) in which the first housing (220) and the second housing (210) are fully folded out through a hinge structure contained within a folding housing (e.g., the folding housing (303) of FIG. 3).
[0043] According to one embodiment, the first housing (220) may include a first surface (221), a second surface (not shown) opposite to the first surface (221), a first side (233) and a second side (234) between the first surface (221) and the second surface.
[0044] According to one embodiment, the second housing (210) may include a third surface (211), a fourth surface opposite to the third surface (211) (e.g., the fourth surface (307) of FIG. 3), a third side (231) and a fourth side (232) between the third surface (211) and the fourth surface.
[0045] According to one embodiment, the folding housing may include a hinge structure that rotatably connects a first side (233) of a first housing (220) and a third side (231) of a second housing (210) facing the first side (233) with respect to a folding axis (235).
[0046] According to one embodiment, the state (200) may mean a state in which the first direction (202) toward which the first surface (221) of the first housing (220) faces corresponds to the second direction (201) toward which the third surface (211) of the second housing (210) faces. For example, within the state (200), the first direction (202) may be substantially parallel to the second direction (201). For example, within the state (200), the first direction (202) may be the same as the second direction (201).
[0047] According to one embodiment, within the state (200), the first surface (221) may form substantially one flat surface with the third surface (211). For example, within the state (200), the angle (203) between the first surface (221) and the third surface (211) may be 180 degrees. For example, the state (200) may mean a state in which the entire display area of the first display (240) can be provided on substantially one flat surface. According to one embodiment, within the state (200), the display area of the first display (240) may not include a curved surface. The unfolded state may be referred to as an outspread state or outspreading state.
[0048] Referring to FIG. 3, an electronic device (e.g., the electronic device (101) of FIG. 1) can provide a folded state (300) in which the first housing (220) and the second housing (210) are folded in through a hinge structure within the folding housing (303).
[0049] According to one embodiment, a folding state including state (300) may mean a state in which a first direction (202) toward a first surface (e.g., the first surface (221) of FIG. 2) (not shown in FIG. 3) is distinguished from a second direction (201) toward a third surface (e.g., the third surface (211) of FIG. 2) (not shown in FIG. 3). For example, in state (300), the angle between the first direction (202) and the second direction (201) may be substantially 180 degrees, and the first direction (202) and the second direction (201) may be distinguished from each other. For example, in state (300), the angle (305) between the first surface (221) and the third surface (211) may be substantially 0 degrees. The folding state may be referred to as a folded state. For example, the electronic device (101) can provide a state (300) in which the first surface (221) and the third surface (211) face each other through a hinge structure within a folding housing (303), so that the display area of the first display (240) corresponding to the third surface (211) substantially completely overlaps the display area of the first display (240) corresponding to the first surface (221). For example, the electronic device (101) can provide a state (300) in which the first direction (202) is substantially opposite to the second direction (201). As another example, in the state (300), the display area of the first display (240) may be obscured within the field of view of a user looking at the electronic device (101). However, it is not limited thereto.
[0050] In one embodiment, the first display (240) may be bent by rotation provided through a hinge structure within the folding housing (303). For example, in state (300), a portion of the display area of the first display (240) may be bent. For example, the portion of the display area of the first display (240) may be in a curved state to prevent damage to the first display (240) in the folding state. However, it is not limited thereto.
[0051] For example, the processor (120) can identify the angle between the first direction (202) in which the first surface (221) of the first housing (220) faces and the second direction (201) in which the third surface (211) of the second housing (210) faces, through at least one of a Hall sensor in the electronic device (101), a first sensor in the electronic device (101), a rotation sensor in the folding housing (303), and a stretch sensor in the electronic device (101).
[0052] Meanwhile, in one embodiment, the second housing (210) may include a second display (350) on a fourth surface (307) opposite to the third surface (211). For example, the second display (350) may be used to provide visual information in a folded state where the display area of the first display (240) is not visible.
[0053] According to one embodiment, the electronic device (101) may include at least one antenna formed within at least a portion of the second side (234) of the first housing (220). The electronic device (101) may include at least one antenna formed within at least a portion of the fourth side (232) of the second housing (210). For example, at least one antenna formed within at least a portion of the second side (234) of the first housing (220) may include a first antenna. At least one antenna formed within at least a portion of the fourth side (232) of the second housing (210) may include a second antenna.
[0055] FIG. 4 illustrates an example of distortion correction of a display according to one embodiment. An electronic device (e.g., electronic device (101)) can perform distortion correction of a display (e.g., display module (160)) depending on the state of the electronic device (101). The state of the electronic device (101) may be an unfolded state (e.g., state (200)), an intermediate state, or a folded state (e.g., state (300)). The display may be a flexible display.
[0056] Referring to FIG. 4, the state change diagram (400) illustrates the screen of a foldable electronic device (411, 431) depending on whether distortion correction of the display is performed. The foldable electronic device (411) and the foldable electronic device (431) exemplify the electronic device (101) of FIG. 1.
[0057] The foldable electronic device (411) can display an image before distortion correction in the unfolded state (410). The foldable electronic device (411) may include a hinge structure for forming an unfolded state (e.g., the state (200) of FIG. 2), a folded state (e.g., the state (300) of FIG. 3), or an intermediate state. The image before distortion correction (413) in the unfolded state (410) can be displayed on the display of the foldable electronic device (411) in the unfolded state (410). Hereinafter, the image before distortion correction (413) may be referred to as the first image.
[0058] The foldable electronic device (411) can display an image before distortion correction in an intermediate state (420). A user looking at the foldable electronic device (411) from the front can see the image projected onto the projection surface. The image before distortion correction (413) may be visually distorted by perspective as the foldable electronic device (411) is folded. Therefore, a user looking at the scene of the foldable electronic device (411) can observe an image distorted by perspective. The part of the screen of the display that is close to the user's line of sight (e.g., the ends of the car) may appear relatively large. The part of the screen of the display that is close to the user's line of sight (e.g., the middle of the car) may appear relatively small.
[0059] Unlike the foldable electronic device (411), the foldable electronic device (431) can perform a distortion correction function. The foldable electronic device (431) can display the image before distortion correction (413) in the unfolded state (430). The foldable electronic device (431) may include a hinge structure for forming an unfolded state, a folded state, or an intermediate state. If the state of the foldable electronic device (431) changes from the unfolded state to an intermediate state or a folded state, distortion may occur in the image before distortion correction (413) as described above. Therefore, the foldable electronic device (431) can perform distortion correction. The foldable electronic device (431) can change the image before distortion correction (413) into an image after distortion correction (433) through distortion correction. Hereinafter, the image after distortion correction (433) may be referred to as the second image, as the image after distortion correction.
[0060] The above-described foldable electronic device (431) can display a distortion-corrected image (433) in an intermediate state (440). A user looking at the above-described foldable electronic device (431) can see the image projected onto the projection surface. As the above-described foldable electronic device is folded, the distortion-corrected image (433) may be visually distorted compared to the distortion-corrected image (413) due to perspective. However, because the foldable electronic device (431) distorts the distortion-corrected image (413) in advance, the distortion-corrected image (433) displayed in the intermediate state (440) may not be distorted from the user's perspective. That is, through advance distortion, the foldable electronic device (431) can provide the user with a relatively less distorted or unde-distorted image. The user can observe from the distortion-corrected image (433) the same view as the view of the distortion-corrected image (413) in the unfolded state (430).
[0061] Hereinafter, distortion correction methods are described for a foldable electronic device (101) to change an image (413) (e.g., a first image) before distortion correction into an image (433) (e.g., a second image) after distortion correction. The image (433) after distortion correction can be generated based on folding information corresponding to the angle formed by the first surface of the first housing and the third surface of the second housing of the foldable electronic device (101). The folding information may be a folding angle, which is the angle formed by the first surface and the third surface. The at least one processor (120) may acquire the folding information through a folding angle recognition sensor. The folding angle recognition sensor may be a plurality of 6-axis sensors or strain sensors. The plurality of 6-axis sensors may be sensors that detect tilt. The plurality of 6-axis sensors may be placed on a printed circuit board (PCB) in the first housing and second housing regions, respectively. Folding information of the foldable electronic device (101) can be obtained based on the difference in tilt measured by the plurality of 6-axis sensors. The strain sensor can be connected via a flexible printed circuit board (FPCB). When the foldable electronic device (101) is folded, the flexible printed circuit board (FPCB) can be bent. The foldable electronic device (101) can obtain the angle at which the flexible printed circuit board (FPCB) is bent through the strain sensor. Therefore, the at least one processor can obtain folding information through the strain sensor.
[0063] FIG. 5a illustrates an example of distortion correction performed based on a user's line of sight located on the central axis of a display according to one embodiment. An electronic device (e.g., electronic device (101)) can perform distortion correction of a display (e.g., display module (160)) in a state of the electronic device (e.g., intermediate state). Hereinafter, the display may be a flexible display. A distortion correction method performed based on a user's line of sight located on the central axis of a display may be referred to as a first distortion correction method (500).
[0064] Referring to FIG. 5a, a foldable electronic device (501) (e.g., the electronic device (101) of FIG. 1) can perform a first distortion correction method (500). The first distortion correction method (500) illustrates the principle of distortion correction performed based on a user's line of sight located at the central axis of the display of the foldable electronic device (501).
[0065] A foldable electronic device (501) (e.g., foldable electronic device (431)) in an intermediate state (e.g., intermediate state (440)) can be folded such that the first surface of the first housing (507) and the third surface of the second housing (509) form an angle between the angle in the unfolded state (e.g., 180 degrees) and the angle in the folded state (e.g., 0 degrees). The folding axis (521) may be the axis along which the foldable electronic device (501) is folded. The user's line of sight (505) may be located on the central axis of the display included in the foldable electronic device (501). The central axis may be perpendicular to the display in the unfolded state and may be a surface including the folding axis (521).
[0066] In the unfolded state, the foldable electronic device (501) can display a first display area. The first display area can be specified by first coordinate information. The first coordinate information may include end coordinates of the first display area. The first coordinate information may include coordinates (513), (515), (517), and (519). The first coordinate information may include end coordinates of the folding axis. The first coordinate information may include coordinates (531) and (533). In the intermediate state, the foldable electronic device (501) can display a second display area. The second display area can be specified by second coordinate information. The second coordinate information may include end coordinates of the second display area. The second coordinate information may include coordinates (523), (525), (527), and (529). The second coordinate information may include end coordinates of the folding axis. The second coordinate information may include coordinates (531) and coordinates (533). The first coordinate information and the second coordinate information may share the coordinates of the folding axis (e.g., coordinates (531) and coordinates (533)).
[0067] In the unfolded state (e.g., unfolded state (430)), the foldable electronic device (501) (e.g., foldable electronic device (431)) may have the first surface of the first housing (507) and the third surface of the second housing (509) facing in the same direction. The angle (511) may be a value formed by the first housing and the surface that is perpendicular to the display and includes the folding axis in the unfolded state. The angle (511) may be a value formed by the second housing and the surface that is perpendicular to the display and includes the folding axis in the unfolded state.
[0068] To correct visual distortion caused by perspective occurring in the user's gaze (505), an image may be displayed on a second display area within the second coordinate information. The user's gaze (505) may be a point corresponding to the pupil of the user observing the display. According to one embodiment, the user's gaze (505) may be a point corresponding to the pupil of the side the user primarily uses. For example, the user's gaze (505) may point to a point corresponding to the pupil of the left eye. As another example, the user's gaze (505) may point to a point corresponding to the pupil of the right eye. According to one embodiment, the user's gaze (505) may be a point corresponding to the pupil of the side designated by the user. According to one embodiment, the user's gaze (505) may be a point corresponding to the pupil of the side closer to the display. As the distance from the display increases, the degree of change in the display area for distortion correction increases, and a large amount of blank space may occur. According to one embodiment, the user's gaze (505) may be a point corresponding to the space between the user's two pupils (e.g., the center).
[0069] According to one embodiment, the processor (120) of the electronic device (101) may obtain second coordinate information based on the user's gaze (505) and first coordinate information of the first display area. The user's gaze (505) may be directed toward a vertex of the first display area. The coordinates of the point where the flexible display of the foldable electronic device (501) meets the line connecting the user's gaze (505) and the vertex of the first display area (coordinates (513), (515), (517), and (519)) may be determined as the coordinates of the second display area. For example, the foldable electronic device (501) may identify the coordinates (523) of the point where the line connecting the user's gaze (505) and the coordinate (513) meets the first surface. The foldable electronic device (501) may identify the coordinates (525) of the point where the line connecting the user's gaze (505) and the coordinate (515) meets the first surface. The foldable electronic device (501) can identify the coordinates (527) of the point where the line connecting the user's line of sight (505) and the coordinates (517) meets the third surface. The foldable electronic device (501) can identify the coordinates (529) of the point where the line connecting the user's line of sight (505) and the coordinates (519) meets the third surface.
[0070] The second coordinate information may be the coordinates of the points where the first surface and the third surface of the foldable electronic device (501) in an intermediate state meet, and the surface connecting the vertices of the first display area of the foldable electronic device (501) in an unfolded state and the user's line of sight.
[0071] User gaze information may include the coordinates of the user gaze (505). According to one embodiment, the user gaze information may be obtained through a camera and a distance sensing sensor. The user gaze information may determine the position of the user gaze (e.g., x-axis and y-axis coordinates) through the camera and determine the distance to the user gaze (e.g., z-axis coordinate) through the distance sensing sensor. The distance sensing sensor may be a time of flight (TOF). The distance sensing sensor may be a light detection and ranging sensor.
[0073] FIG. 5b illustrates an example (540) of x-axis distortion correction performed based on a user's line of sight located at the center axis of a display according to one embodiment. FIG. 5b may be the result of a user viewing the first distortion correction method (500) from the y-axis direction.
[0074] Referring to FIG. 5b, the first value (541) (e.g., x) may correspond to a length corresponding to the end (551, 553) of the first display area from the folding axis. The first value (541) may be a given value. The second value (543) (e.g., a) may correspond to a first angle formed by the first housing and the surface including the folding axis, which is perpendicular to the display in the unfolded state. The second value (543) may be obtained based on a folding detection sensor. The third value (545) (e.g., b) may correspond to the angle formed by the direction perpendicular to the display in the unfolded state and including the folding axis, and the direction from the user's line of sight looking toward the end (551) of the first display area within the first housing in the unfolded state. The third value (545) may be obtained based on the first value and the fifth value. The fourth value (547) (e.g., b) may correspond to the angle formed by the direction perpendicular to the flexible display in the unfolded state and including the folding axis, and the direction from the user's line of sight looking toward the end (553) of the first display area within the second housing in the unfolded state. The fourth value (547) may be obtained through at least one camera and at least one distance sensing sensor. The fifth value (549) (e.g., z) may represent the vertical distance between the user's line of sight and the display in the unfolded state. The fifth value (549) may be obtained through a camera and a distance sensing sensor.
[0075] The x1 value (555) may be the distance between the projected end point of the second display area within the second housing in the intermediate state and the end point of the first display area in the unfolded state. The x2 value (557) may be the distance between the projected end point of the second display area within the second housing in the intermediate state and the projected point of the folding axis of the foldable electronic device. The x3 value (559) may be the distance between the folding axis of the foldable electronic device and the end point of the second display area within the second housing in the intermediate state. The x4 value (561) may be the vertical distance between the projected plane and the end point of the second display area within the second housing in the intermediate state. The following mathematical formulas can be derived by trigonometric functions.
[0076]
[0077]
[0078]
[0079]
[0080]
[0081]
[0082] The x1 value (555), the x2 value (557), the x3 value (559), and the x4 value (561) can be obtained by solving the above mathematical equations simultaneously. The x3 value (559) may be the distance between the end of the second display area of the second housing and the folding axis of the foldable electronic device in the intermediate state. In other words, the x3 value (559) may be the x-axis coordinate of the second display area of the second housing in the intermediate state. The x3 value can be derived as in mathematical equation 7. The x4 value can be derived as in mathematical equation 8.
[0083]
[0084]
[0085] When the folding axis is the y-axis, the x-axis distortion correction coordinate can be the x3 value. The x4 value can be used when calculating the y-axis distortion correction coordinate.
[0087] FIG. 5c illustrates an example of y-axis distortion correction performed based on a user's line of sight located on the central axis of a display according to one embodiment. FIG. 5c may show the first distortion correction method (500) as viewed by a user in the x-axis direction.
[0088] Referring to FIG. 5c, the coordinates (571) of the user's line of sight may be determined by the position of the user's line of sight looking at the display. The projection plane (573) may be a surface on which the screen of the foldable electronic device in the intermediate state (e.g., intermediate state (440)) is projected. The projection plane may be a surface formed by the foldable electronic device in the unfolded state (e.g., the foldable electronic device (501) in the unfolded state of FIG. 5a).
[0089] The first value (575) (e.g., z) may be the vertical distance between the user's line of sight and the display in the unfolded state. The first value (575) (e.g., the fifth value (549) in FIG. 5b) may be obtained through a camera and a distance sensing sensor. The second value (577) (e.g., the x4 value (561) in FIG. 5b) (e.g., x4) may be the vertical distance between the end point of the second display area in the second housing in the intermediate state and the projection plane. The third value (579) (e.g., y1) may be the distance between the end point of the projection plane in the folding axis direction and the point where the user's line of sight is projected. The third value (579) may be obtained through a camera and a distance sensing sensor. The y2 value (581) may be the distance from one end point in the folding axis direction of the second display area of the foldable electronic device in the intermediate state to the point where the user's line of sight is projected. The y3 value (583) may be the distance from the other end of the folding axis direction of the second display area of the foldable electronic device in the intermediate state to the point where the user's line of sight is projected. By proportion, the following mathematical equations can be derived.
[0090]
[0091]
[0092] By the above mathematical formulas 9 and 10, the y2 value (581) and the y3 value (583) can be obtained. The y2 value (581) may be the y-axis coordinate of one vertex of the second display area in the intermediate state. The y3 value (583) may be the y-axis coordinate of the other vertex of the second display area in the intermediate state.
[0093] Therefore, the foldable electronic device (101) can obtain the range of the second display area after distortion correction by using FIG. 5b and FIG. 5c.
[0095] FIG. 5d illustrates an example of a distortion-corrected screen performed based on a user's line of sight located on the central axis of a display, according to one embodiment.
[0096] Referring to FIG. 5d, the display area change diagram (590) may show a first display area and a second display area before and after distortion correction. The first display area may be the display area of the foldable electronic device before distortion correction. The second display area may be the display area of the foldable electronic device after distortion correction. The coordinates of the vertices of the first display area (coordinates (591), (592), (593), (594)) and the coordinates of the end of the folding axis (599) may be included in the first coordinate information of the first display area before distortion correction. In an intermediate state, the vertices of the display area may be changed from the coordinates of the vertices of the first display area (coordinates (591), (592), (593), (594)) to the coordinates of the vertices of the second display area (coordinates (595), (596), (597), (598)) based on the first distortion correction method. The coordinates of the vertices of the second display area (coordinates (595), (596), (597), and (598)) and the coordinates of the end of the folding axis (599) may be included in the second coordinate information of the second display area after distortion correction.
[0097] Unlike the display area change diagrams (690, 790) to be examined below, the second display area may be symmetrical because the position of the user's gaze is on the central axis.
[0099] FIG. 6a illustrates an example of distortion correction performed based on a projection plane having the same angle with respect to a first housing and a second housing, according to one embodiment. An electronic device (e.g., electronic device (101)) can perform distortion correction of a display (e.g., display module (160)) in a state of the electronic device (e.g., intermediate state). In FIG. 6a, the user's line of sight is not on the central axis but may be at any position. Hereinafter, the display may be a flexible display. The distortion correction method performed based on the user's line of sight at any position may be referred to as a second distortion correction method (600).
[0100] Referring to FIG. 6a, a foldable electronic device (601) (e.g., the electronic device (101) of FIG. 1) can perform a second distortion correction method (600). Through the second distortion correction method (600), the principle of distortion correction of the foldable electronic device (601) is illustrated based on a projection plane in which the angle formed with the first housing and the second housing is the same.
[0101] A foldable electronic device (601) (e.g., foldable electronic device (431)) in an intermediate state (e.g., intermediate state (440)) can be folded such that the first surface of the first housing (607) and the third surface of the second housing (609) form an angle between the angle in the unfolded state (e.g., 180 degrees) and the angle in the folded state (e.g., 0 degrees). The folding axis (621) may refer to the axis along which the foldable electronic device (601) is folded. The user's line of sight (605) may be located at any position.
[0102] In the unfolded state, the foldable electronic device (601) can display a first display area. The first display area can be specified by first coordinate information. The first coordinate information may include end coordinates of the first display area. The first coordinate information may include coordinates (613), (615), (617), and (619). The first coordinate information may include end coordinates of the folding axis. The first coordinate information may include coordinates (631) and (633). In the intermediate state, the foldable electronic device (601) can display a second display area. The second display area can be specified by second coordinate information. The second coordinate information may include end coordinates of the second display area. The second coordinate information may include coordinates (623), (625), (627), and (629). The second coordinate information may include end coordinates of the folding axis. The second coordinate information may include coordinates (631) and coordinates (633). The first coordinate information and the second coordinate information may share the coordinates of the folding axis (e.g., coordinates (631) and coordinates (633)).
[0103] In the unfolded state (e.g., unfolded state (410) and unfolded state (430) of FIG. 4), the foldable electronic device (601) (e.g., foldable electronic device (431)) may have the first surface of the first housing (607) and the third surface of the second housing (609) facing in the same direction. The angle a (611) may be a value formed by the first housing and the surface that is perpendicular to the display and includes the folding axis in the unfolded state. The angle a (611) may be a value formed by the second housing and the surface that is perpendicular to the display and includes the folding axis in the unfolded state.
[0104] To correct visual distortion caused by perspective occurring in the user's gaze (605), an image may be displayed on a second display area within the second coordinate information. The user's gaze (605) may be a point corresponding to the pupil of the user observing the display. According to one embodiment, the user's gaze (605) may be a point corresponding to the pupil of the side the user primarily uses. For example, the user's gaze (605) may point to a point corresponding to the pupil of the left eye. As another example, the user's gaze (605) may point to a point corresponding to the pupil of the right eye. According to one embodiment, the user's gaze (605) may be a point corresponding to the pupil of the side designated by the user. According to one embodiment, the user's gaze (605) may be a point corresponding to the pupil of the side closer to the display. As the distance from the display increases, the degree of change in the display area for distortion correction increases, which may result in a large amount of blank space. According to one embodiment, the user's gaze (605) may be a point corresponding to the space between the user's two pupils (e.g., the center).
[0105] According to one embodiment, the processor (120) of the electronic device (101) may obtain second coordinate information based on the user's gaze (605) and first coordinate information of the first display area. The user's gaze (605) may be directed toward a vertex of the first display area. The coordinates of the point where the line connecting the user's gaze (605) and the vertex of the first display area meets the flexible display of the foldable electronic device (601) may be determined as the coordinates of the second display area. For example, the foldable electronic device (601) may identify the coordinates (623) of the point where the line connecting the user's gaze (605) and the coordinate (613) meets the first surface. The foldable electronic device (601) may identify the coordinates (625) of the point where the line connecting the user's gaze (605) and the coordinate (615) meets the first surface. The foldable electronic device (601) can identify the coordinates (627) of the point where the line connecting the user's line of sight (605) and the coordinates (617) meets the third surface. The foldable electronic device (601) can identify the coordinates (629) of the point where the line connecting the user's line of sight (605) and the coordinates (619) meets the third surface.
[0106] The second coordinate information may be the coordinates of the points where the line connecting the vertices (coordinates (613), (615), (617), and (619)) of the first display area of the foldable electronic device (601) in the unfolded state and the user's line of sight (605) meets the first and third sides of the foldable electronic device (601) in the intermediate state.
[0107] User gaze information may include the coordinates of the user's gaze. According to one embodiment, the user gaze information may be acquired through a camera and a distance sensing sensor. The user gaze information may determine the position of the user's gaze (e.g., x-axis and y-axis coordinates) through the camera and determine the distance to the user's gaze (e.g., z-axis coordinate) through the distance sensing sensor. The distance sensing sensor may be a time of flight (TOF). The distance sensing sensor may be a light detection and ranging (Lidar).
[0109] FIG. 6b illustrates an example of x-axis distortion correction performed based on a projection plane that has the same angle with the first housing and the second housing according to one embodiment. The second distortion correction method (600) may be viewed by a user in the y-axis direction.
[0110] Referring to FIG. 6b, the first value (641) (e.g., x) may correspond to a length from the folding axis to the end (651, 653) of the first display area. The first value (641) may be a given value. The second value (643) (e.g., a) may correspond to a first angle formed by the second housing and the plane containing the folding axis, which is perpendicular to the display in the unfolded state. The second value (643) may be obtained based on a folding detection sensor. The third value (645) (e.g., b1) may correspond to the angle formed by the direction containing the folding axis, which is perpendicular to the display in the unfolded state, and the direction looking from the user's line of sight toward the end (651) of the first display area within the first housing in the unfolded state. The third value (645) may be obtained based on the first value (641), the fifth value (649), and the x5 value (673). The fourth value (647) (e.g., b2) may correspond to the angle formed by the direction perpendicular to the flexible display in the unfolded state and including the folding axis, and the direction from the user's line of sight looking toward the end (653) of the first display area within the second housing in the unfolded state. The fifth value (649) (e.g., z) may be the vertical distance between the user's line of sight and the display in the unfolded state. The fifth value (649) may be obtained through a camera and a distance sensing sensor. The sixth value (648) may be the vertical distance between the first virtual user's line of sight for obtaining the second coordinate information of the first housing and the display in the unfolded state. The seventh value (650) may be the vertical distance between the second virtual user's line of sight for obtaining the second coordinate information of the second housing and the display in the unfolded state.
[0111] Hereinafter, the x-axis coordinates of the distortion-corrected second display area in the first housing are derived. The x1 value (665) may be the distance between the point where the end point of the second display area in the first housing of the intermediate state is projected and the end point of the first display area in the unfolded state. The x2 value (667) may be the distance between the point where the end point of the second display area in the second housing of the intermediate state is projected and the point where the folding axis of the foldable electronic device is projected. The x3 value (669) may be the distance between the folding axis of the foldable electronic device and the end point of the second display area in the second housing of the intermediate state. The x4 value (671) may be the vertical distance between the end point of the second display area in the second housing of the intermediate state and the projection plane. The following mathematical formulas can be derived by trigonometric functions. The z' value (648) may represent the vertical distance between the first virtual user's line of sight (652) and the display in the unfolded state. The x5 value (673) may represent the distance between the user's line of sight and the center axis of the display.
[0112] <Mathematical Formula 1>
[0113]
[0114] <Mathematical Formula 2>
[0115]
[0116] <Mathematical Formula 3>
[0117]
[0118] <Mathematical Formula 4>
[0119]
[0120] <Mathematical Formula 5>
[0121]
[0122] <Mathematical Formula 6>
[0123]
[0124]
[0125] By solving the above mathematical equations simultaneously, the x1 value (665), the x2 value (667), the x3 value (669), the x4 value (671), the x5 value (673), and the b1 value (645) can be obtained. The x3 value (669) may be the distance between the end of the second display area of the second housing and the folding axis of the foldable electronic device in the intermediate state. In other words, the x3 value (669) may be the absolute value of the x-axis coordinate of the second display area of the second housing in the intermediate state. The x3 value can be derived as in mathematical equation 7. The x4 value can be derived as in mathematical equation 8.
[0126] <Mathematical Formula 7>
[0127]
[0128] <Mathematical Formula 8>
[0129]
[0130] When the folding axis is the y-axis, the x-axis distortion correction coordinate can be -x3. The x4 value can be used when calculating the y-axis distortion correction coordinate. However, if the x5 value is x, the z' value cannot be obtained in Equation 11. Therefore, the x-axis distortion correction coordinate -x3 value cannot be obtained either. Therefore, at least one processor can obtain the distortion correction coordinate by substituting an adjacent value for x5 when the x5 value is x.
[0131] The y-axis coordinates of the distortion-corrected second display area in the first housing and the second housing can be derived in the same way as the y-axis distortion correction method (e.g., FIG. 5c) performed based on the user's line of sight located on the folding axis.
[0132] The virtual user gaze may be a virtual gaze on the center axis of the foldable electronic device assumed to derive the second distortion correction method (600) from the first distortion correction method (500). The coordinates after distortion correction of the first housing may be the same as the coordinates after distortion correction when the user gaze is at the position of the first virtual user gaze (652). Accordingly, the second coordinate information of the second display area of the first housing can be obtained by changing the fifth value (e.g., z) in the first distortion correction method (500). The coordinates after distortion correction of the second housing may be the same as the coordinates after distortion correction when the user gaze is at the position of the second virtual user gaze (654). Accordingly, the second coordinate information of the second display area of the second housing can be obtained by changing the z value in the first distortion correction method (500).
[0133] The x-axis coordinates of the distortion-corrected second display area in the second housing are derived below. The x1 value (655) may be the distance between the point where the end point of the second display area in the first housing of the intermediate state is projected and the end point of the first display area in the unfolded state. The x2 value (657) may be the distance between the point where the end point of the second display area in the second housing of the intermediate state is projected and the point where the folding axis of the foldable electronic device is projected. The x3 value (659) may be the distance between the folding axis of the foldable electronic device and the end point of the second display area in the second housing of the intermediate state. The x4 value (661) may be the vertical distance between the end point of the second display area in the second housing of the intermediate state and the projection plane. The following mathematical formulas can be derived by trigonometric functions. The z' value (650) may represent the vertical distance between the second virtual user's line of sight (654) and the display in the unfolded state. The x5 value (663) may represent the distance between the user's line of sight and the center axis of the display.
[0134] <Mathematical Formula 1>
[0135]
[0136] <Mathematical Formula 2>
[0137]
[0138] <Mathematical Formula 3>
[0139]
[0140] <Mathematical Formula 4>
[0141]
[0142] <Mathematical Formula 5>
[0143]
[0144] <Mathematical Formula 6>
[0145]
[0146]
[0147] By solving the above mathematical equations simultaneously, the x1 value (655), the x2 value (657), the x3 value (659), the x4 value (661), the x5 value (663), and the b2 value (647) can be obtained. The x3 value (659) may be the distance between the end of the second display area of the second housing and the folding axis of the foldable electronic device in the intermediate state. In other words, the x3 value (659) may be the x-axis coordinate of the second display area of the second housing in the intermediate state. The x3 value can be derived as in Equation 7. The x4 value can be derived as in Equation 8.
[0148] <Mathematical Formula 7>
[0149]
[0150] <Mathematical Formula 8>
[0151]
[0152] When the folding axis is the y-axis, the x-axis distortion correction coordinate can be the x3 value. The x4 value can be used when calculating the y-axis distortion correction coordinate. However, if the x5 value is -x, the z' value cannot be obtained in Equation 12. Therefore, the x3 value, which is the x-axis distortion correction coordinate, cannot be obtained either. Therefore, at least one processor can obtain the distortion correction coordinate by substituting an adjacent value for the x5 value instead of -x when the x5 value is -x.
[0154] FIG. 6c illustrates an example of a distortion-corrected screen based on a projection plane that has the same angle with the first housing and the second housing, according to one embodiment.
[0155] Referring to FIG. 6c, the display area change diagram (690) may show a first display area before distortion correction and a second display area after distortion correction. The first display area may be the display area of the foldable electronic device before distortion correction. The second display area may be the display area of the foldable electronic device after distortion correction. The coordinates of the vertices of the first display area (coordinates (691), (692), (693), (694)) and the coordinates of the end of the folding axis (699) may be included in the first coordinate information of the first display area before distortion correction. In an intermediate state, the vertices of the display area may be changed from the coordinates of the vertices of the first display area (coordinates (691), (692), (693), (694)) to the coordinates of the vertices of the second display area (coordinates (695), (696), (697), (698)) based on the first distortion correction method. The coordinates of the vertices of the second display area (coordinates (695), (696), (697), and (698)) and the coordinates of the end of the folding axis (699) may be included in the second coordinate information of the second display area after distortion correction.
[0156] Unlike the previously reviewed display area change diagram (590), the second display area may not be symmetrical because the position of the user's gaze is not on the central axis.
[0158] FIG. 7a illustrates an example of distortion correction of an electronic device based on a projection plane perpendicular to the user's line of sight, according to one embodiment. An electronic device (e.g., electronic device (101)) can perform distortion correction of a display (e.g., display module (160)) in a state of the electronic device (e.g., intermediate state). In FIG. 7a, the user's line of sight is not on the central axis but may be at any position. Hereinafter, the display may be a flexible display. A distortion correction method performed based on the user's line of sight at any position may be referred to as a third distortion correction method (700).
[0159] Referring to FIG. 7a, a foldable electronic device (701) (e.g., the electronic device (101) of FIG. 1) can perform a third distortion correction method (700). The third distortion correction method (700) illustrates the principle of distortion correction performed based on a projection plane perpendicular to the user's line of sight.
[0160] A foldable electronic device (701) (e.g., the foldable electronic device (431) of FIG. 4) in an intermediate state (e.g., the intermediate state (440) of FIG. 4) can be folded such that the first surface of the first housing (707) and the third surface of the second housing (709) form an angle between the angle in the unfolded state (e.g., 180 degrees) and the angle in the folded state (e.g., 0 degrees). The folding axis (721) may be the axis along which the foldable electronic device (701) is folded. The user's line of sight (705) may be located at any position.
[0161] In the unfolded state, the foldable electronic device (701) can display a first display area. The first display area can be specified by first coordinate information. The first coordinate information may include end coordinates of the first display area. The first coordinate information may include coordinates (713), (715), (717), and (719). The first coordinate information may include end coordinates of the folding axis. The first coordinate information may include coordinates (731) and (733). In the intermediate state, the foldable electronic device (701) can display a second display area. The second display area can be specified by second coordinate information. The second coordinate information may include end coordinates of the second display area. The second coordinate information may include coordinates (723), (725), (727), and (729). The second coordinate information may include end coordinates of the folding axis. The second coordinate information may include coordinates (731) and coordinates (733). The first coordinate information and the second coordinate information may share the coordinates of the folding axis (e.g., coordinates (731) and coordinates (733)).
[0162] The projection plane may be perpendicular to the user's line of sight. The projection plane may be an electronic device in an unfolded state that is unfolded perpendicular to the user's line of sight. Angle a (711) may be half the angle formed by the first housing and the second housing. Angle c (735) may be the angle value between the plane containing the folding axis, which is perpendicular to the projection plane that is equal to the angle formed by the first housing and the second housing, and the plane containing the folding axis and the user's line of sight.
[0163] To correct visual distortion caused by perspective occurring in the user's gaze (705), an image may be displayed on a second display area within the second coordinate information. The user's gaze (705) may be a point corresponding to the pupil of the user observing the display. According to one embodiment, the user's gaze (705) may be a point corresponding to the pupil of the side the user primarily uses. For example, the user's gaze (705) may point to a point corresponding to the pupil of the left eye. As another example, the user's gaze (705) may point to a point corresponding to the pupil of the right eye. According to one embodiment, the user's gaze (705) may be a point corresponding to the pupil of the side designated by the user. According to one embodiment, the user's gaze (705) may be a point corresponding to the pupil of the side closer to the display. As the distance from the display increases, the degree of change in the display area for distortion correction increases, which may result in a large amount of blank space. According to one embodiment, the user's gaze (705) may be a point corresponding to the space between the user's two pupils (e.g., the center).
[0164] According to one embodiment, the processor (120) of the electronic device (101) may obtain second coordinate information based on the user's gaze (705) and first coordinate information of the first display area. The user's gaze (705) may be directed toward a vertex of the first display area. The coordinates of the point where the line connecting the user's gaze (705) and the vertex of the first display area meets the flexible display of the foldable electronic device (701) may be determined as the coordinates of the second display area. For example, the foldable electronic device (701) may identify the coordinates (723) of the point where the line connecting the user's gaze (705) and the coordinate (713) meets the first surface. The foldable electronic device (701) may identify the coordinates (725) of the point where the line connecting the user's gaze (705) and the coordinate (715) meets the first surface. The foldable electronic device (701) can identify the coordinates (727) of the point where the line connecting the user's line of sight (705) and the coordinates (717) meets the third surface. The foldable electronic device (701) can identify the coordinates (729) of the point where the line connecting the user's line of sight (705) and the coordinates (719) meets the third surface.
[0165] The second coordinate information may be the coordinate information of the points where the line connecting the vertices (coordinates (713), (715), (717), and (719)) of the first display area of the foldable electronic device (701) in the unfolded state and the user's line of sight (705) meets the first and third surfaces of the foldable electronic device (701) in the intermediate state.
[0166] User gaze information may include the coordinates of the user gaze (705). According to one embodiment, the user gaze information may be obtained through a camera and a distance sensing sensor. The user gaze information may determine the position of the user gaze (e.g., x-axis and y-axis coordinates) through the camera and determine the distance to the user gaze (e.g., z-axis coordinate) through the distance sensing sensor. The distance sensing sensor may be a time of flight (TOF). The distance sensing sensor may be a light detection and ranging (Lidar).
[0168] FIG. 7b illustrates an example (740) of x-axis distortion correction performed based on a projection plane perpendicular to the user's line of sight according to one embodiment. The second distortion correction method (700) may be viewed by the user in the y-axis direction.
[0169] Referring to FIG. 7b, the first value (741) (e.g., x) may correspond to a length corresponding to the end (751, 753) of the first display area from the folding axis. The first value (741) may be a given value. The second value (743) (e.g., ac or a+c) may correspond to the first angle formed by the second housing and the plane containing the folding axis, which is perpendicular to the projection plane that is equal to the angle formed by the first housing and the second housing. The second value (743) may be obtained based on a folding detection sensor, at least one camera, and at least one distance detection sensor. The third value (745) (e.g., b) may correspond to the angle formed by the plane containing the folding axis, which is perpendicular to the projection plane that is equal to the angle formed by the first housing and the second housing, and the direction in which the user's line of sight looks toward the end (753) of the first display area within the second housing in the unfolded state. The third value (745) can be obtained based on the first value, the fifth value, and the angle c. The fourth value (749) (e.g., z) may represent the vertical distance between the user's line of sight and the display in the unfolded state. The fourth value (749) can be obtained through a camera and a distance sensing sensor.
[0170] The above-described foldable electronic device can obtain second coordinate information of a second display area of a second housing by changing a second value in the first distortion correction method (500). The coordinates of the first housing after distortion correction may be the same as the coordinates after distortion correction when the second value is the sum of a value and a value and the user's line of sight is on the central axis. Accordingly, the second coordinate information of the second display area of the first housing can be obtained by changing a second value in the first distortion correction method (500).
[0171] The x-axis coordinates of the distortion-corrected second display area in the second housing are calculated below. The x1 value (755) may be the distance between the point where the end point of the second display area in the first housing of the intermediate state is projected and the end point of the first display area in the unfolded state. The x2 value (757) may be the distance between the point where the end point of the second display area in the second housing of the intermediate state is projected and the point where the folding axis of the foldable electronic device is projected. The x3 value (759) may be the distance between the folding axis of the foldable electronic device and the end point of the second display area in the second housing of the intermediate state. The x4 value (761) may be the vertical distance between the end point of the second display area in the second housing of the intermediate state and the projection plane. The following mathematical formulas can be derived by trigonometric functions.
[0172] <Mathematical Formula 1>
[0173]
[0174] <Mathematical Formula 2>
[0175]
[0176] <Mathematical Formula 3>
[0177]
[0178] <Mathematical Formula 4>
[0179]
[0180] <Mathematical Formula 5>
[0181]
[0182] <Mathematical Formula 6>
[0183]
[0185] By solving the above mathematical equations simultaneously, the x1 value (755), the x2 value (757), the x3 value (759), the x4 value (761), the b value (745), and the c value (747) can be obtained. The x3 value (759) may be the distance between the end of the second display area of the second housing and the folding axis of the foldable electronic device in the intermediate state. In other words, the x3 value (759) may be the x-axis coordinate of the second display area of the second housing in the intermediate state. The x3 value can be derived as in Equation 7. The x4 value can be derived as in Equation 8.
[0186] <Mathematical Formula 7>
[0187]
[0188] <Mathematical Formula 8>
[0189]
[0190] When the folding axis is the y-axis, the x-axis distortion correction coordinate can be the x3 value. The x4 value can be used when calculating the y-axis distortion correction coordinate.
[0191] The y-axis coordinates of the distortion-corrected second display area in the first housing and the second housing can be derived in the same way as the y-axis distortion correction method (e.g., FIG. 5c) performed based on the user's line of sight located on the folding axis.
[0193] FIG. 7c illustrates an example of a distortion-corrected screen performed based on a projection plane perpendicular to the user's line of sight, according to one embodiment.
[0194] Referring to FIG. 7c, the display area change diagram (790) may show a first display area and a second display area before and after distortion correction. The first display area may be the display area of the foldable electronic device before distortion correction. The second display area may be the display area of the foldable electronic device after distortion correction. The coordinates of the vertices of the first display area (coordinates (791), (792), (793), (794)) and the coordinates of the end of the folding axis (799) may be included in the first coordinate information of the first display area before distortion correction. In an intermediate state, the coordinates of the vertices of the first display area (coordinates (791), (792), (793), (794)) may be changed to the coordinates of the vertices of the second display area (coordinates (795), (796), (797), (798)) based on the first distortion correction method. The coordinates of the vertices of the second display area (coordinates (795), (796), (797), and (798)) and the coordinates of the end of the folding axis (799) may be included in the second coordinate information of the second display area after distortion correction.
[0195] Unlike the previously reviewed display area change diagram (590), the second display area may not be symmetrical because the position of the user's gaze is not on the central axis.
[0197] FIG. 8 illustrates an example of margin utilization by distortion correction for displaying information according to one embodiment.
[0198] Referring to FIG. 8, a foldable electronic device (800) in an unfolded state may display an image (801) (e.g., a first image) before distortion correction. A foldable electronic device (803) in an intermediate state may include an image (805) (e.g., a second image) after distortion correction. A margin portion (807) of the display may be created by distortion correction. The margin portion (807) of the display may refer to a portion of the display that is within the first display area and simultaneously outside the second display area. According to embodiments, text, an image, or a user interface (UI) may be displayed in the margin portion (807) to display information about the image (805) after distortion correction. According to embodiments, text may be displayed in the margin portion (807) to display information about the image (805) after distortion correction. For example, the text may display the shooting time and capacity of the image (805) after distortion correction. According to embodiments, an image for displaying information about the image (805) after distortion correction may be displayed in the blank portion (807). For example, the image may represent the type of the image (805) after distortion correction. According to embodiments, a user interface (UI) for displaying information about the image (805) after distortion correction may be displayed in the blank portion (807). For example, the user interface may be a button to search for the image (805) after distortion correction on the internet.
[0199] FIG. 8 illustrates that text, images, or a user interface (UI) may be displayed in the blank area (807) only to display information regarding the image after distortion correction, but embodiments of the present disclosure are not limited thereto. According to one embodiment, text, images, or a user interface (UI) may be displayed in the blank area (807) to display information related to the image after distortion correction (805). For example, if the image after distortion correction (805) is a sky photograph, information regarding the weather on the shooting date may be displayed. According to one embodiment, text, images, or a user interface (UI) may be displayed in the blank area (807) to display information unrelated to the image after distortion correction (805). For example, text, images, or a user interface (UI) may be displayed in the blank area (807) to display the remaining battery capacity.
[0201] Figure 9 illustrates an example of image loss that may occur during distortion correction.
[0202] Referring to FIG. 9, the first display area (901) may be a display area of a foldable electronic device in an unfolded state before distortion correction. The virtual display area (903) may be an area within the line where the surface connecting the user's line of sight to the end of the first display area meets the surface of the first housing or the surface of the second housing. An image loss portion (905) may occur when the length from the folding axis to the end of the virtual display area is greater than the length from the folding axis to the end of the first display area. In FIG. 10 and FIG. 11 below, we examine how to handle the case where an image loss portion (905) occurs.
[0204] FIG. 10 illustrates an example of a notification for indicating image loss during distortion correction according to one embodiment.
[0205] Referring to FIG. 10, the foldable electronic device (1001) can display a distortion-corrected image (1003) on the display in an intermediate state. A notification (1005) may indicate image loss.
[0206] In an intermediate state, the foldable electronic device (1001) may have a length from the folding axis to the end of the virtual display area greater than the length from the folding axis to the end of the first display area. Thus, an image loss portion (e.g., the image loss portion (905) of FIG. 9) may occur. According to embodiments, the foldable electronic device (1001) (e.g., the electronic device (101) of FIG. 1) may display a notification (1005) to indicate the image loss on the display of the foldable electronic device (1001) closest to the image loss portion (905). For example, the notification (1005) to indicate the image loss may be displayed by indicating a color. For example, the notification (1005) to indicate the image loss may be displayed by repeatedly blinking the color indicated on the display of the foldable electronic device (1001) closest to the image loss portion (905). For example, the above notification (1005) for indicating image loss may be displayed through a notification window that appears on the display of the foldable electronic device (1001) closest to the image loss portion (905).
[0208] FIG. 11 illustrates an example of image reduction during distortion correction according to one embodiment. When the virtual display area is larger than the area of the display included in the foldable electronic device in an intermediate state, the foldable electronic device may reduce the virtual display area to obtain a second display area in order to display the entire image on the display.
[0209] Referring to FIG. 11, the foldable electronic device (1101) can display a distortion-corrected image on a display in an intermediate state. The first display area may be a part of the display area before distortion correction of the display. The virtual display area (1103) may be an area where a plane connecting the user's line of sight to the end of the first display area meets a plane including a first housing or a plane including a second housing. The second display area (1105, 1107, 1109) may be a part of the display area after distortion correction of the display. The second display area (1105, 1107, 1109) may be a part obtained based on the virtual display area.
[0210] According to one embodiment, the at least one processor may reduce the entire image to display on the display when the length from the folding axis to the end of the virtual display area (1103) is longer than the length from the folding axis of the foldable electronic device (1101) to the end of the display area. The at least one processor may reduce the length from the end of the virtual display area (1103) to the folding axis such that the length from the folding axis of the foldable electronic device to the end of the display area is the same as the length from the folding axis of the foldable electronic device. The at least one processor may reduce the virtual display area (1103) by the same ratio with respect to the X-axis and y-axis to obtain a second display area (1105, 1107, 1109). When the at least one processor obtains the second display area in which the virtual display area (1103) has been reduced, a portion other than the second display area may occur on the folding axis. According to one embodiment, the second display area may vary depending on the placement position of the portion other than the second display area on the folding axis. For example, when a portion other than the second display area on the folding axis is placed at the bottom of the folding axis, the second display area (1105) can be obtained. For example, when a portion other than the second display area on the folding axis is placed evenly at the top and bottom of the folding axis, the second display area (1107) can be obtained. For example, when a portion other than the second display area on the folding axis is placed at the top of the folding axis, the second display area (1109) can be obtained.
[0211] According to one embodiment, the change in the second display area may be performed gradually to reduce the sense of unfamiliarity caused by abrupt size changes. According to one embodiment, when the second display area is changed, a notification may be displayed to indicate the change in the display area.
[0213] FIG. 12 illustrates an example of image enlargement during distortion correction according to one embodiment.
[0214] Referring to FIG. 12, the foldable electronic device (1201) in an intermediate state can display an image after distortion correction on a display. The first display area of the foldable electronic device (1201) before distortion correction may be the display area of the foldable electronic device in an unfolded state. The virtual display area (1203) may be an area where a surface connecting the end of the first display area from the user's line of sight meets a surface including a first housing or a surface including a second housing. The second display area (1205, 1207, 1209) may be a portion on the display that displays an image after distortion correction. The second display area (1205, 1207, 1209) after distortion correction of the electronic device may be the display area in an intermediate state. Due to distortion correction, the length in the direction perpendicular to the folding axis of the second display area may be smaller than the length in the direction perpendicular to the folding axis of the first display area.
[0215] According to one embodiment, if the length from the folding axis to the end of the virtual display area (1203) is shorter than the length from the folding axis of the foldable electronic device (1201) to the end of the display area, the image can be enlarged to increase the usability of the display. The at least one processor can enlarge the virtual display area (1203) so that the length from the end of the virtual display area (1203) to the folding axis becomes the length from the folding axis of the foldable electronic device to the end of the display area. The at least one processor can obtain a second display area (1205, 1207, 1209) by enlarging the virtual display area (1203) by the same ratio with respect to the X-axis and y-axis. When the at least one processor obtains the second display area in which the virtual display area (1203) is enlarged, a margin of the enlarged virtual display area may occur on the folding axis. The margin of the enlarged virtual display area may not be displayed on the display. This is because the margin of the enlarged virtual display area extends beyond the area of the display. According to one embodiment, the second display area may vary depending on the placement position of the margin portion of the enlarged virtual display area. For example, when the margin portion of the enlarged virtual display area is placed at the bottom of the folding axis, the second display area (1205) may be obtained. For example, when the margin portion of the enlarged virtual display area is placed evenly at the top and bottom of the folding axis, the second display area (1207) may be obtained. For example, when the margin portion of the enlarged virtual display area is placed at the top of the folding axis, the second display area (1209) may be obtained. Additionally, to minimize image loss, the at least one processor may minimize the margin portion of the enlarged virtual display area. For example, the at least one processor may identify the width of the margin portion of the enlarged virtual display area and obtain a second display area in which the width of the margin portion is minimized.For example, the width of the margin portion lost in the second display area (1205) may be smaller than the width of the margin portion lost in the second display area (1207). Additionally, the width of the margin portion lost in the second display area (1205) may be smaller than the width of the margin portion lost in the second display area (1209). At this time, the at least one processor may display an image within the second display area (1205) where the width of the lost margin portion is minimized.
[0217] FIG. 13 illustrates an example of distortion correction applied to only a portion of the display area according to one embodiment.
[0218] Referring to FIG. 13, the foldable electronic device (1301) in an intermediate state can display an image before distortion correction in a first display area (1303) and a third display area (1305). The foldable electronic device (1301) in the intermediate state can display an image after distortion correction (e.g., a second image) in a second display area (1311) where the first display area has been changed. The foldable electronic device (1301) in the intermediate state can display an image before distortion correction in a third display area (1313) where the image before and after distortion correction is the same.
[0219] According to the embodiments, when a third image is displayed in a third display area of the display in an unfolded state, the third image can be displayed in the third display area of the display in an intermediate state.
[0220] According to one embodiment, the third image in the third display area (1305) may have a lower visibility importance than the image in the first display area (1303). For example, when multimedia such as images and / or videos is displayed in the first display area (1303), the at least one processor may perform distortion correction on the image in the first display area (1303). The at least one processor may not perform distortion correction on the image in the third display area (1305) where multimedia is not displayed. By distinguishing between the first display area (1303) and the third display area (1305) and performing distortion correction only on the image in the first display area, the efficiency of the processor may be increased.
[0221] According to one embodiment, the third image in the third display area (1305) may have a higher importance for receiving touch input than the image in the first display area (1303). For example, when an image containing content with a high touch frequency is displayed in the third display area (1305), the at least one processor may not perform distortion correction on the image in the third display area (1305). The at least one processor may perform distortion correction on the image in the first display area (1303) containing content with a low touch input frequency. If the at least one processor distinguishes between the first display area (1303) and the third display area (1305) and performs distortion correction only on the image in the first display area, the efficiency of the processor may be increased.
[0223] FIG. 14 illustrates an example of selecting a display area to apply distortion correction according to one embodiment. The user can select a display area to apply distortion correction through user input.
[0224] Referring to FIG. 14, in operation (1401), a foldable electronic device (e.g., the electronic device (101) of FIG. 1) can display a first image on a display in an unfolded state.
[0225] In operation (1403), the user can select (1405) a first display area (1407) to correct the distortion of the intermediate state and a third display area (1409) not to correct the distortion of the intermediate state.
[0226] In operation (1411), the foldable electronic device (101) can change the first display area (1407) into a second display area (1413) by distortion correction. The foldable electronic device (101) may not perform distortion correction on the third display area (1415).
[0227] In FIG. 14, only the case where a display area to apply distortion correction is selected is illustrated, but embodiments of the present disclosure are not limited thereto.
[0228] According to one embodiment, the at least one processor can identify a first display area (1407) and a third display area (1409) in an intermediate state according to a specified setting. The at least one processor can perform distortion correction on the first display area (1407) in an intermediate state according to a specified setting.
[0229] For example, the specified settings may be set according to visibility importance. The at least one processor may identify a first display area (1407) and a third display area (1409) according to visibility importance. When multimedia, such as images and / or videos, is displayed in the first display area (1407), the at least one processor may perform distortion correction on the images in the first display area (1303). The at least one processor may not perform distortion correction on the images in the third display area (1409) where multimedia is not displayed.
[0230] For example, the specified setting may be set according to the touch input frequency. The at least one processor may identify a first display area (1407) and a third display area (1409) according to the touch input frequency. When an image containing content with a high touch frequency is displayed in the third display area (1409), the at least one processor may not perform distortion correction on the image in the third display area (1409). The at least one processor may perform distortion correction on the image in the first display area (1407) containing content with a low touch input frequency.
[0231] For example, the above-mentioned specified settings may be set by a software application. The software application may set criteria for designating the first display area (1407) and the third display area (1409) according to visibility importance. The software application may set criteria for designating the first display area (1407) and the third display area (1409) according to touch input frequency.
[0233] FIG. 15 illustrates an example of whether distortion correction is changed based on a change in the user's gaze according to one embodiment.
[0234] Referring to FIG. 15, the foldable electronic device (1501) may be in an intermediate state. While the foldable electronic device (1501) is in an intermediate state, a change in user gaze (1503) may occur. After the foldable electronic device (1501) displays a second image based on the first user gaze information, the foldable electronic device (1501) may acquire the second user gaze information.
[0235] According to one embodiment, the foldable electronic device (1501) can perform distortion correction based on a designated user gaze, such as a second display area (1511) after distortion correction. Regardless of a change in the user gaze (1503), the second image modified based on the first user gaze information can continue to be displayed.
[0236] According to one embodiment, the foldable electronic device (1501) may display a second image generated based on each user gaze information according to a user gaze change (1503). The second display area of the foldable electronic device (1501) after distortion correction may be changed to a second display area (1521), a second display area (1523), and a second display area (1525) according to a user gaze change (1503). According to one embodiment, the second display area (1521) may correspond to the first user gaze information (1505). The second display area (1523) may correspond to the second user gaze information (1507). The second display area (1525) may correspond to the third user gaze information (1509).
[0237] In FIG. 15, only the case where the user's gaze changes while the foldable electronic device (1501) is in an intermediate state is illustrated, but embodiments of the present disclosure are not limited thereto.
[0238] According to one embodiment, folding information may be changed while the user's gaze is maintained. The folding information may be a folding angle, which is the angle formed by the first surface of the first housing and the third surface of the second housing of the foldable electronic device (101). The folding angle may be changed from 10 degrees to 20 degrees. For example, the at least one processor (120) may change a second display area (e.g., second display area (1523)) based on the changed folding information (e.g., changing the folding angle from 10 degrees to 20 degrees) and the fixed user gaze.
[0240] FIG. 16 illustrates the flow of an operation for displaying a distortion-corrected image based on user gaze information, folding information, and a first display area, according to one embodiment.
[0241] Referring to FIG. 16, in operation (1601), at least one processor (e.g., processor (120) of FIG. 2) can display a first image on a first display area of a flexible display in an unfolded state. In operation (1603), the at least one processor (120) can obtain first coordinate information of the first display area of the flexible display corresponding to the unfolded state. In operation (1605), the at least one processor (120) can obtain folding information corresponding to the angle between the first surface and the third surface of the flexible display based on detecting an intermediate state. In operation (1607), the at least one processor (120) can obtain second coordinate information of a second display area based on user gaze information, the folding information, and the first coordinate information of the first display area. In operation (1609), the at least one processor (120) can generate a second image based on the first image and the second coordinate information of the second display area. In operation (1611), the at least one processor (120) can display the second image in the second display area of the flexible display in an intermediate state.
[0243] An electronic device including a flexible display according to the embodiments aims to correct screen distortion caused by changes in the state of the electronic device without relying on an external camera that photographs the electronic device. Visual distortion caused by changes in the state of the flexible display can be corrected by using user gaze information and coordinate information without using a separate camera. Through the distortion correction, the user experience can be enhanced by correcting visual distortion that occurs due to changes in the state of the flexible display. In addition, efficient utilization of the flexible display can be achieved by utilizing the blank screen generated during the visual distortion correction process.
[0245] In the drawings above, user gaze information is described as being acquired based on at least one camera and at least one distance sensing sensor, but embodiments of the present disclosure are not limited thereto. For example, the user may manually input user gaze coordinates.
[0247] As described above, according to one embodiment, an electronic device may include a first housing comprising a first surface and a second surface opposite to the first surface. The electronic device may include a second housing comprising a third surface and a fourth surface opposite to the third surface. The electronic device may include at least one processor, at least one camera, at least one distance sensing sensor, and a flexible display. The electronic device may include a hinge structure configured to provide an unfolded state in which the first surface and the third surface face in the same direction, a folded state in which the first surface and the third surface face each other, or an intermediate state in which the first surface and the third surface form an angle between the angle in the unfolded state and the angle in the folded state by rotatably connecting the first housing and the second housing with respect to a folding axis. The electronic device may include a folding sensing sensor for detecting the angle between the first surface and the third surface of the flexible display. The at least one processor can display a first image on a first display area of the flexible display in the unfolded state. The at least one processor can obtain first coordinate information of the first display area of the flexible display corresponding to the unfolded state. The at least one processor can obtain folding information corresponding to the angle between the first surface and the third surface of the flexible display based on detecting the intermediate state. The at least one processor can obtain user gaze information based on the at least one distance sensing sensor and the at least one camera. Based on the user gaze information, the folding information, and the first coordinate information of the first display area, the second coordinate information of the second display area corresponding to the intermediate state can be obtained.The at least one processor can generate a second image based on the first image and the second coordinate information of the second display area. The at least one processor can display the second image in the second display area of the flexible display in the intermediate state.
[0248] According to one embodiment, the first coordinate information may include a first coordinate and a second coordinate with respect to the folding axis, a third coordinate and a fourth coordinate with respect to the vertices of the first display area within the first housing in the unfolded state, and a fifth coordinate and a sixth coordinate with respect to the vertices of the first display area within the second housing in the unfolded state. According to one embodiment, the second coordinate information may include a first coordinate and a second coordinate with respect to the folding axis, a seventh coordinate and an eighth coordinate corresponding to the vertices of the second display area within the first housing in the intermediate state, and a ninth coordinate and a tenth coordinate corresponding to the vertices of the second display area within the second housing.
[0249] According to one embodiment, the second coordinate information of the second display area may be obtained based on a first value corresponding to the length from the folding axis to the end of the first display area. The second coordinate information of the second display area may be obtained based on a second value corresponding to a first angle formed by the first housing and a surface that is perpendicular to the flexible display and includes the folding axis in the unfolded state. The second coordinate information of the second display area may be obtained based on a third value corresponding to the angle formed by the direction that is perpendicular to the flexible display and includes the folding axis in the unfolded state and the direction from the user's line of sight looking toward the end of the first display area within the first housing in the unfolded state. The second coordinate information of the second display area may be obtained based on a fourth value corresponding to the angle formed by the direction that is perpendicular to the flexible display and includes the folding axis in the unfolded state and the direction from the user's line of sight looking toward the end of the first display area within the second housing in the unfolded state. The second coordinate information of the second display area can be obtained based on a fifth value, which is the vertical distance between the user's line of sight and the flexible display in the unfolded state.
[0250] According to one embodiment, the second coordinate information of the second display area can be obtained based on a sixth value that corresponds to the angle formed by a direction including the folding axis and a direction from the folding axis to the user's line of sight, which is perpendicular to the flexible display in the unfolded state.
[0251] According to one embodiment, in order to obtain the second coordinate information, the at least one processor may obtain third coordinate information of a virtual display area, which is a change of the first display area, based on the user gaze information, the folding information, and the first coordinate information of the first display area. The virtual display area may be an area where a surface connecting the user gaze to the end of the first display area meets a surface including a first housing or a surface including a second housing. If the length from the folding axis to the end of the virtual display area is greater than the length from the folding axis to the end of the first display area, the at least one processor may obtain the second coordinate information by changing the third coordinate information of the virtual display area to be closer to the folding axis.
[0252] According to one embodiment, in order to obtain the second coordinate information, the at least one processor may obtain third coordinate information of a virtual display area, which is a change of the first display area, based on the user gaze information, the folding information, and the first coordinate information of the first display area. The virtual display area may be an area where a surface connecting the user gaze to the end of the first display area meets a surface including a first housing or a surface including a second housing. If the length from the folding axis to the end of the virtual display area is shorter than the length from the folding axis to the end of the first display area, the second coordinate information may be obtained by transforming the third coordinate information of the virtual display area so that the length from the folding axis to the end of the virtual display area becomes the length from the folding axis to the end of the first display area.
[0253] According to one embodiment, the at least one processor may obtain third coordinate information of a virtual display area, which is a change of the first display area, based on the user gaze information, the folding information, and the first coordinate information of the first display area. The at least one processor may determine that the virtual display area is an area where a plane connecting the user gaze to the end of the first display area meets the plane of the first housing or the plane of the second housing. If the length from the folding axis to the end of the virtual display area is greater than the length from the folding axis to the end of the first display area, a notification indicating image loss may be additionally displayed.
[0254] According to one embodiment, the at least one processor may additionally display text, an image, or a UI (user interface) for displaying information about the second image on a portion of the flexible display within the first display area and a portion of the flexible display outside the second display area.
[0255] According to one embodiment, the at least one processor may additionally display a third image on a third display area of the flexible display in the unfolded state. The at least one processor may additionally display the third image on the third display area of the flexible display in the intermediate state.
[0256] According to one embodiment, the at least one processor can acquire first user gaze information based on the at least one distance sensing sensor and the at least one camera. The at least one processor can acquire second coordinate information of a second display area corresponding to the intermediate state based on the first user gaze information, the folding information, and the first coordinate information of the first display area. The at least one processor can generate the second image based on the first image and the second coordinate information of the second display area. The second image can be displayed on the second display area of the flexible display in the intermediate state. The at least one processor can additionally acquire second user gaze information based on the at least one distance sensing sensor and the at least one camera. The at least one processor can additionally acquire second coordinate information of the second display area corresponding to the intermediate state based on the first user gaze information, the folding information, and the first coordinate information of the first display area. The at least one processor may additionally generate the second image based on the first image and the second coordinate information of the second display area. The at least one processor may additionally display the second image in the second display area of the flexible display in the intermediate state.
[0257] As described above, a method performed by an electronic device according to one embodiment may include an operation of displaying a first image on a first display area of a flexible display in an unfolded state. The method may include an operation of acquiring first coordinate information of the first display area of the flexible display corresponding to the unfolded state. The method may include an operation of acquiring folding information corresponding to an angle between a first surface and a third surface of the flexible display based on detecting an intermediate state. The method may include an operation of acquiring user gaze information based on at least one distance sensing sensor and at least one camera. The method may include an operation of acquiring second coordinate information of a second display area corresponding to the intermediate state based on the user gaze information, the folding information, and the first coordinate information of the first display area. The method may include an operation of generating a second image based on the first image and the second coordinate information of the second display area. The above method may include an operation of displaying the second image in the second display area of the flexible display in the intermediate state.
[0258] According to one embodiment, the first coordinate information may include a first coordinate and a second coordinate with respect to the folding axis. In the unfolded state, the first coordinate information may include a third coordinate and a fourth coordinate with respect to the vertices of the first display area within the first housing. In the unfolded state, the first coordinate information may include a fifth coordinate and a sixth coordinate with respect to the vertices of the first display area within the second housing. The second coordinate information may include a first coordinate and a second coordinate with respect to the folding axis. In the intermediate state, the second coordinate information may include a seventh coordinate and an eighth coordinate corresponding to the vertices of the second display area within the first housing. The second coordinate information may include a ninth coordinate and a tenth coordinate corresponding to the vertices of the second display area within the second housing.
[0259] According to one embodiment, the second coordinate information of the second display area may be obtained based on a first value corresponding to the length from the folding axis to the end of the first display area. The second coordinate information of the second display area may be obtained based on a second value corresponding to a first angle formed by a surface perpendicular to the flexible display and including the folding axis in the unfolded state and the first housing. The second coordinate information of the second display area may be obtained based on a third value corresponding to the angle formed by a direction perpendicular to the flexible display and including the folding axis in the unfolded state and a direction from the user's line of sight looking toward the end of the first display area within the first housing in the unfolded state. The second coordinate information of the second display area may be obtained based on a fourth value corresponding to the angle formed by a direction perpendicular to the flexible display and including the folding axis in the unfolded state and a direction from the user's line of sight looking toward the end of the first display area within the second housing in the unfolded state. The second coordinate information of the second display area can be obtained based on a fifth value, which is the vertical distance between the user's line of sight and the flexible display in the unfolded state.
[0260] According to one embodiment, the second coordinate information of the second display area can be obtained based on a sixth value corresponding to the angle formed by a direction including the folding axis and a direction from the folding axis to the user's line of sight, which is perpendicular to the flexible display in the unfolded state.
[0261] According to one embodiment, the operation of acquiring the second coordinate information may include the operation of acquiring third coordinate information of a virtual display area, which is a change of the first display area, based on the user gaze information, the folding information, and the first coordinate information of the first display area. The virtual display area may be an area where a surface connecting the user gaze to the end of the first display area meets a surface including a first housing or a surface including a second housing. The operation of acquiring the second coordinate information may include the operation of acquiring the second coordinate information by changing the third coordinate information of the virtual display area closer to the folding axis when the length from the folding axis to the end of the virtual display area is greater than the length from the folding axis to the end of the first display area.
[0262] According to one embodiment, the operation of acquiring the second coordinate information may include the operation of acquiring third coordinate information of a virtual display area, which is a change of the first display area, based on the user gaze information, the folding information, and the first coordinate information of the first display area. The virtual display area may be an area where a surface connecting the user gaze to the end of the first display area meets a surface including a first housing or a surface including a second housing. The operation of acquiring the second coordinate information may include the operation of acquiring the second coordinate information by transforming the third coordinate information of the virtual display area so that, when the length from the folding axis to the end of the virtual display area is shorter than the length from the folding axis to the end of the first display area, the length from the folding axis to the end of the virtual display area becomes the length from the folding axis to the end of the first display area.
[0263] According to one embodiment, the method may additionally include an operation of obtaining third coordinate information of a virtual display area, which is a change of the first display area, based on the user gaze information, the folding information, and the first coordinate information of the first display area. The virtual display area may be an area where a surface connecting the user gaze to the end of the first display area meets a surface of the first housing or a surface of the second housing. The method may additionally include an operation of displaying a notification to indicate image loss when the length from the folding axis to the end of the virtual display area is greater than the length from the folding axis to the end of the first display area.
[0264] According to one embodiment, the method may additionally include an operation of displaying text, an image, or a UI (user interface) for displaying information about the second image on a portion of the flexible display within a first display area and a portion of the flexible display outside the second display area.
[0265] According to one embodiment, the method may additionally include an operation of displaying a third image on a third display area of the flexible display in the unfolded state. The method may additionally include an operation of displaying the third image on the third display area of the flexible display in the intermediate state.
[0266] According to one embodiment, the method may additionally include an operation of acquiring first user gaze information based on the at least one distance sensing sensor and the at least one camera. The method may additionally include an operation of acquiring second coordinate information of a second display area corresponding to the intermediate state based on the first user gaze information, the folding information, and the first coordinate information of the first display area. The method may additionally include an operation of generating the second image based on the first image and the second coordinate information of the second display area. The method may additionally include an operation of displaying the second image on the second display area of the flexible display in the intermediate state. The method may additionally include an operation of acquiring second user gaze information based on the at least one distance sensing sensor and the at least one camera. The method may additionally include an operation of acquiring second coordinate information of the second display area corresponding to the intermediate state based on the first user gaze information, the folding information, and the first coordinate information of the first display area. The above method may additionally include an operation of generating the second image based on the first image and the second coordinate information of the second display area. The above method may additionally include an operation of displaying the second image in the second display area of the flexible display in the intermediate state.
[0268] The electronic devices according to the various embodiments disclosed in this document may be of various forms. The electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, electronic devices, or consumer electronics. The electronic devices according to the embodiments of this document are not limited to the devices described above.
[0269] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as "coupled" or "connected" to another (e.g., 2nd) component, with or without the terms "functionally" or "communicationly," it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.
[0270] The term “module” as used in the various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).
[0271] Various embodiments of the present document may be implemented as software (e.g., program (140)) comprising one or more instructions stored in a storage medium (e.g., internal memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101)). For example, a processor (e.g., processor (120)) of the machine (e.g., electronic device (101)) may call at least one of the one or more instructions stored in the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.
[0272] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.
[0273] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to the integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.
Claims
Claim 1 An electronic device comprises: a first housing including a first surface and a second surface opposite to the first surface; a second housing including a third surface and a fourth surface opposite to the third surface; at least one processor; at least one camera; at least one distance sensing sensor; a flexible display; and a hinge structure configured to provide an unfolded state in which the first surface and the third surface face in the same direction, a folded state in which the first surface and the third surface face each other, or an intermediate state in which the first surface and the third surface form an angle between the angle in the unfolded state and the angle in the folded state, by rotatably connecting the first housing and the second housing with respect to a folding axis.and includes a folding detection sensor for detecting an angle between the first surface and the third surface of the flexible display, wherein the at least one processor displays a first image on a first display area of the flexible display in the unfolded state, obtains first coordinate information of the first display area of the flexible display corresponding to the unfolded state, obtains folding information corresponding to an angle between the first surface and the third surface of the flexible display based on detecting the intermediate state, obtains user gaze information based on the at least one distance detection sensor and the at least one camera, obtains second coordinate information of a second display area corresponding to the intermediate state based on the user gaze information, the folding information, and the first coordinate information of the first display area, generates a second image based on the first image and the second coordinate information of the second display area, displays the second image on the second display area of the flexible display in the intermediate state, and the first coordinate information is such that the folding axis and the first edge of the first display area An electronic device comprising: a first coordinate that intersects; a second coordinate at which the folding axis and a second edge facing the first edge intersect; and a plurality of third coordinates associated with the first display area, wherein the second coordinate information comprises a plurality of fourth coordinates determined based on the user gaze information and the folding information from the first coordinate, the second coordinate, and the plurality of third coordinates. Claim 2 An electronic device, wherein, in claim 1, the plurality of third coordinates include a fifth coordinate and a sixth coordinate for the vertices of the first display area within the first housing in the unfolded state; and a seventh coordinate and an eighth coordinate for the vertices of the first display area within the second housing in the unfolded state, and the plurality of fourth coordinates include a ninth coordinate and a tenth coordinate corresponding to the vertices of the second display area within the first housing in the intermediate state; and an eleventh coordinate and a twelveth coordinate corresponding to the vertices of the second display area within the second housing. Claim 3 An electronic device, wherein, in claim 1, the second coordinate information of the second display area is obtained based on: a first value corresponding to the length from the folding axis to the edge of the first display area parallel to the folding axis; a second value corresponding to the first angle formed by the first housing and a plane perpendicular to the flexible display and including the folding axis in the unfolded state; a third value corresponding to the angle formed by the direction perpendicular to the flexible display in the unfolded state and the direction extending vertically from a point on the edge within the first housing in the unfolded state to the user's line of sight; a fourth value corresponding to the angle formed by the direction perpendicular to the flexible display in the unfolded state and the direction extending vertically from a point on the edge within the second housing in the unfolded state to the user's line of sight; and a fifth value which is the vertical distance between the user's line of sight and the flexible display in the unfolded state. Claim 4 An electronic device, wherein, in claim 3, the second coordinate information of the second display area is obtained based on a sixth value corresponding to the angle formed by the direction perpendicular to the flexible display in the unfolded state and the direction from the folding axis toward the user's line of sight. Claim 5 An electronic device configured such that, in order to obtain the second coordinate information in claim 1, the at least one processor obtains the third coordinate information of a virtual display area, which is a change of the first display area, based on the first coordinate information of the first display area based on the user gaze information and the folding information, and the virtual display area is an area where a surface connecting the user gaze to the end of the first display area meets a surface including a first housing or a surface including a second housing, and when the length from the folding axis to the end of the virtual display area is greater than the length from the folding axis to the end of the first display area, the third coordinate information of the virtual display area is changed to be closer to the folding axis to obtain the second coordinate information. Claim 6 An electronic device configured such that, in order to obtain the second coordinate information in claim 1, the at least one processor obtains third coordinate information of a virtual display area, which is a change of the first display area, based on the user gaze information, the folding information, and the first coordinate information of the first display area, wherein the virtual display area is an area where a surface connecting the user gaze to the end of the first display area meets a surface including a first housing or a surface including a second housing, and when the length from the folding axis to the end of the virtual display area is smaller than the length from the folding axis to the end of the first display area, the third coordinate information of the virtual display area is transformed so that the length from the folding axis to the end of the virtual display area becomes the length from the folding axis to the end of the first display area, thereby obtaining the second coordinate information. Claim 7 An electronic device according to claim 1, wherein the at least one processor obtains third coordinate information of a virtual display area, which is a change of the first display area, based on the user gaze information, the folding information, and the first coordinate information of the first display area, wherein the virtual display area is an area where a plane connecting the user gaze to the end of the first display area meets the plane of the first housing or the plane of the second housing, and additionally displays a notification to indicate image loss when the length from the folding axis to the end of the virtual display area is greater than the length from the folding axis to the end of the first display area. Claim 8 An electronic device in claim 1, wherein the at least one processor is additionally configured to display text, an image, or a user interface (UI) for displaying information about the second image on a portion of the flexible display within the first display area and a portion of the flexible display outside the second display area. Claim 9 An electronic device in claim 1, wherein the at least one processor is additionally configured to display a third image on a third display area of the flexible display in the unfolded state and to display the third image on the third display area of the flexible display in the intermediate state. Claim 10 An electronic device further configured in claim 1, wherein the at least one processor acquires first user gaze information based on the at least one distance sensing sensor and the at least one camera, acquires second coordinate information of a second display area corresponding to the intermediate state based on the first user gaze information, the folding information, and the first coordinate information of the first display area, generates the second image based on the first image and the second coordinate information of the second display area, displays the second image on the second display area of the flexible display in the intermediate state, acquires second user gaze information based on the at least one distance sensing sensor and the at least one camera, acquires second coordinate information of the second display area corresponding to the intermediate state based on the first user gaze information, the folding information, and the first coordinate information of the first display area, generates the second image based on the first image and the second coordinate information of the second display area, and displays the second image on the second display area of the flexible display in the intermediate state. Claim 11 A method performed by an electronic device comprises: an operation of displaying a first image on a first display area of a flexible display in an unfolded state; an operation of acquiring first coordinate information of the first display area of the flexible display corresponding to the unfolded state; an operation of acquiring folding information corresponding to an angle between a first surface and a third surface of the flexible display based on detecting an intermediate state; an operation of acquiring user gaze information based on at least one distance sensing sensor and at least one camera; an operation of acquiring second coordinate information of a second display area corresponding to the intermediate state based on the user gaze information, the folding information, and the first coordinate information of the first display area; an operation of generating a second image based on the first image and the second coordinate information of the second display area; and an operation of displaying the second image on the second display area of the flexible display in the intermediate state, wherein the first coordinate information comprises a first coordinate where a folding axis and a first edge of the first display area intersect, and the folding axis and the first edge A method comprising a second coordinate where a second edge facing a seat intersects, and a plurality of third coordinates associated with the first display area, wherein the second coordinate information comprises a plurality of fourth coordinates determined based on the user gaze information and the folding information from the first coordinate, the second coordinate, and the plurality of third coordinates. Claim 12 A method in claim 11, wherein the plurality of third coordinates include, in the unfolded state, fifth and sixth coordinates for the vertices of the first display area within the first housing; and in the unfolded state, seventh and eighth coordinates for the vertices of the first display area within the second housing; and the plurality of fourth coordinates include, in the intermediate state, ninth and tenth coordinates corresponding to the vertices of the second display area within the first housing; and eleventh and twelveth coordinates corresponding to the vertices of the second display area within the second housing. Claim 13 In claim 11, the second coordinate information of the second display area is obtained based on: a first value corresponding to the length from the folding axis to the edge of the first display area parallel to the folding axis; a second value corresponding to the first angle formed by the first housing and a surface perpendicular to the flexible display and including the folding axis in the unfolded state; a third value corresponding to the angle formed by the direction perpendicular to the flexible display in the unfolded state and the direction extending vertically from a point on the edge within the first housing in the unfolded state to the user's line of sight; a fourth value corresponding to the angle formed by the direction perpendicular to the flexible display in the unfolded state and the direction extending vertically from a point on the edge within the second housing in the unfolded state to the user's line of sight; and a fifth value which is the vertical distance between the user's line of sight and the flexible display in the unfolded state. Claim 14 In claim 13, the second coordinate information of the second display area is obtained based on a sixth value corresponding to the angle formed by the direction perpendicular to the flexible display in the unfolded state and the direction from the folding axis toward the user's line of sight. Claim 15 In claim 11, the operation of acquiring the second coordinate information comprises the operation of acquiring the third coordinate information of a virtual display area, which is a change of the first display area, based on the user gaze information, the folding information, and the first coordinate information of the first display area, wherein the virtual display area is an area where a surface connecting the user gaze to the end of the first display area meets a surface including a first housing or a surface including a second housing, and when the length from the folding axis to the end of the virtual display area is greater than the length from the folding axis to the end of the first display area, the operation of acquiring the second coordinate information by changing the third coordinate information of the virtual display area closer to the folding axis. Claim 16 In claim 11, the operation of obtaining the second coordinate information comprises: an operation of obtaining the third coordinate information of a virtual display area, which is a change of the first display area, based on the user gaze information, the folding information, and the first coordinate information of the first display area; wherein the virtual display area is an area where a surface connecting the user gaze to the end of the first display area meets a surface including a first housing or a surface including a second housing, and when the length from the folding axis to the end of the virtual display area is smaller than the length from the folding axis to the end of the first display area, the operation of obtaining the second coordinate information by converting the third coordinate information of the virtual display area so that the length from the folding axis to the end of the virtual display area becomes the length from the folding axis to the end of the first display area. Claim 17 A method further comprising, in claim 11, an operation of obtaining third coordinate information of a virtual display area, which is a change of the first display area, based on the user gaze information, the folding information, and the first coordinate information of the first display area, wherein the virtual display area is an area where a surface connecting the user gaze to the end of the first display area meets the surface of the first housing or the surface of the second housing, and an operation of displaying a notification to indicate image loss when the length from the folding axis to the end of the virtual display area is greater than the length from the folding axis to the end of the first display area. Claim 18 A method in claim 11 further comprising the operation of displaying text, an image, or a UI (user interface) for displaying information about the second image on a portion of the flexible display within the first display area and a portion of the flexible display outside the second display area. Claim 19 A method in claim 11, further comprising an operation of displaying a third image on a third display area of the flexible display in the unfolded state and an operation of displaying the third image on the third display area of the flexible display in the intermediate state. Claim 20 A method further comprising, in claim 11, an operation of acquiring first user gaze information based on at least one distance sensing sensor and at least one camera; an operation of acquiring second coordinate information of a second display area corresponding to the intermediate state based on the first user gaze information, the folding information, and the first coordinate information of the first display area; an operation of generating the second image based on the first image and the second coordinate information of the second display area; an operation of displaying the second image on the second display area of the flexible display in the intermediate state; an operation of acquiring second user gaze information based on at least one distance sensing sensor and at least one camera; an operation of acquiring second coordinate information of the second display area corresponding to the intermediate state based on the first user gaze information, the folding information, and the first coordinate information of the first display area; an operation of generating the second image based on the first image and the second coordinate information of the second display area; and an operation of displaying the second image on the second display area of the flexible display in the intermediate state.
Citation Information
Patent Citations
Electronic apparatus and displaying method thereof
KR1020220060926A
Cited By
Display apparatus, method of driving the same and electronic apparatus including the same
US12700366B2
