Flexible display appparutus and controlling method thereof

Abstract

A flexible display device and a method for controlling the same are disclosed. The flexible display device includes a flexible display, a sensor, a memory, and a processor. The processor displays a first content on the flexible display and detects a user's gaze located on the displayed first content through a sensor. When an event occurs in which the shape of the flexible display changes, the processor identifies the last position of the detected user's gaze, stores the identified last position and the screen state in which the first content is displayed in memory, changes the layout to correspond to the changed shape of the flexible display to display the first content, and displays an indicator at a position on the changed first content where the layout corresponding to the stored last position is located.

Description

Technology Field

[0001] The present disclosure relates to a flexible display device and a method for controlling the same, and specifically to a flexible display device and a method for controlling the same that identify a content area that a user reads / views. Background Technology

[0002] Recently, various flexible display devices are being developed. Flexible display devices have the advantage of being portable because their shape can be changed (e.g., folding, rolling), and they can display content on a wide screen when in use.

[0003] In addition, flexible display devices can display content on screens of various sizes depending on their shape and other factors. Depending on the screen size of the flexible display device, the displayed content can also be displayed in various sizes. When the size of the displayed content changes, the layout of the content may change. At this time, the location of the area where the user reads or views may also change. When the location of the area where the user reads or views changes, the user experiences the inconvenience of having to find the reading or viewing area again within the content with the changed layout. Therefore, there is a need for technology that allows the user to quickly find the reading or viewing area even when the layout of the content changes. The problem to be solved

[0004] The present disclosure is intended to solve the aforementioned problems, and the purpose of the present disclosure is to provide a flexible display device and a control method thereof that recognizes an area where a user is reading / viewing and displays the recognized reading / viewing area when the layout of the content is changed. means of solving the problem

[0005] A flexible display device according to one embodiment of the present disclosure includes a flexible display, a sensor, a memory, and a processor, wherein the processor displays a first content on the flexible display and detects a user’s gaze located on the displayed first content through the sensor, and when an event occurs in which the shape of the flexible display changes, the processor identifies the last position of the detected user’s gaze, stores the identified last position and the screen state in which the first content is displayed in the memory, changes the layout to correspond to the changed shape of the flexible display to display the first content, and displays an indicator at a position on the first content in which the layout has changed, corresponding to the stored last position.

[0006] A control method for a flexible display device according to one embodiment of the present disclosure includes the steps of: displaying a first content on a flexible display; detecting a user’s gaze located on the displayed first content; identifying the last position of the detected user’s gaze when an event occurs in which the shape of the flexible display changes; storing the identified last position and the screen state in which the first content is displayed; changing the layout to correspond to the changed shape of the flexible display and displaying the first content on the flexible display; and displaying an indicator at a position on the first content with the changed layout corresponding to the stored last position. Brief explanation of the drawing

[0007] FIG. 1 is a block diagram illustrating the configuration of a flexible display device according to one embodiment of the present disclosure. FIG. 2 is a block diagram illustrating the specific configuration of a flexible display device according to one embodiment of the present disclosure. FIGS. 3a to 3c are drawings illustrating embodiments in which the layout of the content is changed. FIG. 4 is a flowchart illustrating a control method for a flexible display device according to one embodiment of the present disclosure. FIG. 5 is a flowchart illustrating the process of indicating the last position in new content according to one embodiment of the present disclosure. FIG. 6 is a diagram illustrating the clustering of content according to one embodiment of the present disclosure. FIGS. 7a and 7b are drawings illustrating an embodiment of a process for calculating a user's reading speed. FIG. 8 is a drawing illustrating the process of identifying the last position of an image area according to one embodiment of the present disclosure. FIG. 9 is a drawing illustrating the process of moving the last position to the user's current line of sight area according to one embodiment of the present disclosure. FIG. 10 is a diagram illustrating the process of transmitting information related to the last location to an external device according to one embodiment of the present disclosure. Specific details for implementing the invention

[0008] Hereinafter, various embodiments are described in more detail with reference to the attached drawings. The embodiments described in this specification may be modified in various ways. Specific embodiments may be depicted in the drawings and described in detail in the detailed description. However, specific embodiments disclosed in the attached drawings are intended only to facilitate understanding of various embodiments. Accordingly, the technical concept is not limited by specific embodiments disclosed in the attached drawings, and it should be understood that it includes all equivalents or substitutions that fall within the concept and scope of the disclosure.

[0009] Terms including ordinal numbers, such as first, second, etc., may be used to describe various components, but these components are not limited by the aforementioned terms. The aforementioned terms are used solely for the purpose of distinguishing one component from another.

[0010] In this specification, terms such as “comprising” or “having” are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof. When a component is described as being “connected” or “connected” to another component, it should be understood that it may be directly connected to or connected to that other component, or that there may be other components in between. On the other hand, when a component is described as being “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between.

[0011] Meanwhile, a "module" or "part" for a component as used in this specification performs at least one function or operation. Furthermore, a "module" or "part" may perform a function or operation by hardware, software, or a combination of hardware and software. Additionally, a plurality of "modules" or a plurality of "parts," excluding a "module" or "part" that must be performed on specific hardware or on at least one processor, may be integrated into at least one module. A singular expression includes a plural expression unless the context clearly indicates otherwise.

[0012] In describing the present disclosure, the order of each step should be understood as non-limiting, unless the preceding step must logically and temporally be performed prior to the subsequent step. That is, except for such exceptional cases, the essence of the disclosure is not affected even if the process described in the subsequent step is performed prior to the process described in the preceding step, and the scope of the rights should be defined regardless of the order of the steps. Furthermore, the designation "A or B" in this specification is defined to mean not only selectively referring to either A or B, but also including both A and B. Additionally, the term "included" in this specification has a meaning that includes additional components beyond the elements listed as included.

[0013] This specification describes only the essential components necessary for explaining the present disclosure and does not mention components unrelated to the essence of the present disclosure. Furthermore, the mentioned components should not be interpreted in an exclusive sense to include only those components, but should be interpreted in a non-exclusive sense to include other components as well.

[0014] Furthermore, in describing the present disclosure, if it is determined that a detailed description of related known functions or configurations could unnecessarily obscure the essence of the present disclosure, such detailed description is abbreviated or omitted. Meanwhile, each embodiment may be implemented or operated independently, but each embodiment may also be implemented or operated in combination.

[0015] FIG. 1 is a block diagram illustrating the configuration of a flexible display device according to one embodiment of the present disclosure.

[0016] Referring to FIG. 1, the flexible display device (100) may include a flexible display (110), a sensor (120), a processor (130), and a memory (140).

[0017] The flexible display (110) can display content. For example, the content displayed on the flexible display (110) may be content received from an external device or content stored in the flexible display device (100). The flexible display (100) may have a deformed shape. For example, if the flexible display device (100) is of a foldable type, the flexible display (110) may be folded or unfolded. Or, if the flexible display device (100) is of a rollable type, the flexible display (110) may be rolled or unrolled. Also, the flexible display (110) may change the layout of the displayed content to correspond to the deformed shape. For example, the display (130) may be implemented as an LCD (Liquid Crystal Display), OLED (Organic Light Emitting Diode), touch screen, etc. When the flexible display (110) is implemented as a touch screen, the flexible display device (100) can receive control commands through the touch screen.

[0018] The sensor (120) can detect the user's gaze. The processor (130) can determine the position of the user's gaze on the displayed content based on the detected user's gaze. Additionally, the sensor (120) can detect a change in the shape of the flexible display device (100). The processor (130) can modify the layout of the content, etc., based on the detected shape change. Alternatively, the sensor (120) can detect objects around the flexible display device (100). The processor (130) can recognize a control command based on the detected signal and perform a control operation corresponding to the recognized control command. Furthermore, the sensor (120) can detect surrounding environment information of the flexible display device (100). The processor (130) can perform a corresponding control operation based on the surrounding environment information detected by the sensor (120). For example, the sensor (120) may include an image sensor, a tracking sensor, an angle sensor, an accelerometer, a gravity sensor, a gyroscope, a geomagnetic sensor, a direction sensor, a motion recognition sensor, a proximity sensor, a voltmeter, an ammeter, a barometer, a hygrometer, a thermometer, an illuminance sensor, a heat detection sensor, a touch sensor, an infrared sensor, an ultrasonic sensor, etc.

[0019] The processor (130) can control each component of the flexible display device (100). For example, the processor (130) can control the flexible display (110) to display content, control the sensor (120) to detect the user's gaze, and control the memory (140) to store data or information.

[0020] The processor (130) can display content on the flexible display (110) in a layout corresponding to the shape of the flexible display (110) and detect the gaze of a user located on the displayed content through a sensor (120). The processor (130) can cluster the displayed content into text areas or image areas, etc., according to the attributes of the content area.

[0021] The flexible display device (100) can have its shape changed by a user. For example, the flexible display device (100) can be unfolded from a folded state or folded from an unfolded state. Alternatively, the flexible display device (100) can be unrolled from a rolling state or rolled from an unrolled state. When an event occurs in which the shape of the flexible display device (100) changes, the processor (130) can determine the last position of the detected user's gaze. Then, the processor (130) can store the determined last position of the user's gaze and the screen state in which the content is displayed in memory (140).

[0022] The processor (130) can change the layout of the content to correspond to the changed shape of the flexible display (110) and display the content of the changed layout on the flexible display (110).

[0023] The processor (130) can determine the last position the user is reading / viewing based on the screen state where the stored content is displayed and the last position of the user's gaze. The processor (130) can display an indicator in the area of ​​the last position of the user's gaze on the content displayed with the layout changed.

[0024] Alternatively, an event in which the user's gaze is lost may occur. For example, an event in which the user's gaze is lost may include failure to detect the user's gaze, termination of execution of content, quick switching of content, or change in the size of the window in which the content is displayed. An event in which the user's gaze is lost in a broader sense may also include an event in which the shape of the flexible display device (100) described above changes. When an event in which the user's gaze is lost occurs, the processor (130) may store the last position of the user's gaze and the state of the screen in which the content is displayed in memory (140), similar to the above description.

[0025] The flexible display device (100) can change the layout of the displayed content and display it. Alternatively, the flexible display device (10) can execute new content and display the executed content by a user's command. For example, when the flexible display device (100) changes the layout of the currently displayed content and displays it again, it can perform an indicator display process similar to when a shape change event of the flexible display device (100) described above occurs.

[0026] When a flexible display device (100) executes new content and displays the executed content, the processor (130) can determine whether the previously displayed old content and the newly displayed new content are the same content. When the new content is executed, the processor (130) can determine the similarity between the old content and the new content. If the similarity between the old content and the new content is greater than or equal to a preset value, the processor (130) can identify a location corresponding to the last position of the user's gaze stored in the new content. Then, the processor (130) can display an indicator at a location on the new content corresponding to the identified last position.

[0027] In one embodiment, when the last position of the user's gaze is located in a clustered text area, the processor (130) can identify a preset number of consecutive words in the old content that include the word at the last position of the user's gaze based on the stored screen state. Then, the processor (130) can determine similarity based on whether the identified preset number of consecutive words are included in the new content. In one embodiment, the preset number of words is 5, and the flexible display device can determine that the last position of the user's gaze is located at 'orange' in the text “I like fruits. My favorite fruits are grape, orange, cherry, pear and peach.” The flexible display device can compare the two words, including 'orange', which is the word at the last position of the user's gaze. That is, the flexible display device can determine similarity based on whether the words are, grape, orange, cherry, and pear are included in the new content.

[0028] When the last position of the user's gaze is located in a clustered image area, the processor (130) can extract image features of the last position of the user's gaze from the old content based on the stored screen state. Then, the processor (130) can determine similarity based on whether the extracted image features are included in the new content.

[0029] The memory (140) can store data, algorithms, etc. that perform the functions of the flexible display device (100), and can store programs, commands, etc. that run on the flexible display device (100). The algorithms stored in the memory (140) can be loaded into the processor (130) under the control of the processor (130) to perform data processing. In addition, the memory (140) can store the last position of the detected user's gaze and the screen state in which content is displayed when an event occurs. For example, the memory (140) can be implemented as a type such as ROM, RAM, HDD, SSD, memory card, etc.

[0030] FIG. 2 is a block diagram illustrating the specific configuration of a flexible display device according to one embodiment of the present disclosure.

[0031] Referring to FIG. 2, the flexible display device (100) may include a flexible display (110), a sensor (120), a processor (130), a memory (140), an input interface (150), a communication interface (160), a camera (170), a microphone (180), and a speaker (190). Since the flexible display (110), the sensor (120), and the memory (140) are the same as those described in FIG. 1, a detailed description is omitted.

[0032] The input interface (150) can receive commands from a user. For example, the input interface (150) may include a keyboard, a keypad, a touchpad, a mouse, etc. Alternatively, the input interface (150) may be implemented as a sensor (120) that detects the user's proximity or gesture, a camera (170) that photographs the user, or a microphone (180) that receives the user's voice. In one embodiment, the input interface (150) that receives commands from a user, etc., may be referred to as an input device, an input unit, an input module, etc.

[0033] The communication interface (160) can communicate with an external device using a wired or wireless communication method. The communication interface (160) may include a module capable of communicating via methods such as 3G, LTE (Long Term Evolution), 5G, Wi-Fi, Bluetooth, DMB (Digital Multimedia Broadcasting), ATSC (Advanced Television Systems Committee), DVB (Digital Video Broadcasting), LAN (Local Area Network), etc. The external device may include a server, a cloud, a terminal device, etc. For example, the communication interface (160) may receive content or information from an external device. Alternatively, the communication interface (160) may transmit information regarding the last position of the user's gaze and the screen status where content is displayed to an external device, or receive information from an external device. The communication interface (160) that communicates with an external device may be referred to as a communication device, a communication unit, a communication module, a transceiver, etc.

[0034] The camera (170) can capture the surrounding environment of the flexible display device (100). Alternatively, the camera (170) can capture the user's facial expressions, movements, gaze, etc. The processor (130) can recognize a control command or a reading / viewing area based on the captured user's facial expressions, movements, gaze, etc., and perform a control operation corresponding to the recognized control command or the recognized area. For example, the camera (170) may include a CCD sensor or a CMOS sensor. Additionally, the camera (170) may include an RGB camera or a depth camera.

[0035] The microphone (180) can receive the user's voice. The processor (130) can recognize a control command based on the input voice and perform a control operation corresponding to the recognized control command.

[0036] The speaker (190) outputs a voice signal that has undergone voice processing. Additionally, the speaker (190) can output user input commands, information regarding the status or operation of the flexible display device (100), etc., as voice or notification sounds.

[0037] The flexible display device (100) may include all of the configurations described above, or it may include some of the configurations. In addition, the flexible display device (100) may include other configurations that perform various functions in addition to the configurations described above. The configuration of the flexible display device (100) has been described so far. Below, an example is described in which the flexible display device (100) changes the layout of content and displays an indicator on the content.

[0038] FIGS. 3a to 3c are drawings illustrating embodiments in which the layout of the content is changed.

[0039] Referring to FIG. 3a, a foldable type flexible display device (100a) is illustrated. For example, the flexible display of the foldable type flexible display device (100a) may include a first area (11) and a second area (12). The flexible display device (100a) may be folded or unfolded to change its shape. Depending on the folding shape, the flexible display device (100a) may display content in the first area (11), the second area (12), or the entire area (11, 12).

[0040] For example, as illustrated in FIG. 3a, if the angle between the first area (11) and the second area (12) of the flexible display device (100a) is folded to approximately 360 degrees, the flexible display device (100a) can display content only in the first area (11). And, the user can read / view the content displayed in the first area (11). The flexible display device (100a) can detect the gaze of the user reading / viewing the content.

[0041] The user can unfold the flexible display device (100a). When the angle between the first region (11) and the second region (12) of the flexible display device (100a) becomes approximately 180 degrees (or, when unfolded), the flexible display device (100a) can display content over the entire area including the first region (11) and the second region (12).

[0042] The flexible display device (100a) can display content by changing the layout of the content as the size of the displayed area has changed. As the layout of the content has changed, the position of the area that the user reads / views can also change. When an event occurs in which the shape of the flexible display changes, the flexible display device (100a) can identify the last position of the detected user's gaze. And, the flexible display device (100a) can save the identified last position of the user's gaze and the display screen of the content before the layout change. The flexible display device (100a) can display the content by changing the layout in response to the change in the shape of the flexible display.

[0043] The flexible display device (100a) can determine an area (or point) of content corresponding to the last position of the user's gaze based on the last position of the user's gaze stored and the display screen of the content before the layout change. The flexible display device (100a) can search for a location corresponding to the determined area of ​​content on the content with the changed layout. Then, the flexible display device (100a) can display an indicator (1) at the searched location of the content displayed with the changed layout. For example, the indicator (1) may include a preset color, a preset shape, a preset pattern, a preset blinking method, a highlight, etc. The user can quickly recognize the position they were reading / viewing on the content even when the layout of the content has changed.

[0044] Referring to FIG. 3b, a rollable type flexible display device (100b) is illustrated. For example, the flexible display of the rollable type flexible display device (100b) can change the size of the display area according to rolling or unrolling. The flexible display device (100b) can change the layout of the content and display it in correspondence with the size of the display area of ​​the flexible display.

[0045] For example, as illustrated in FIG. 3b, if the flexible display device (100b) is in a rolled state, the flexible display device (100b) can display content only in a certain area exposed to the outside. And, the user can read / view the content displayed in the certain area. The flexible display device (100b) can detect the gaze of the user reading / viewing the content.

[0046] The user can unroll the flexible display device (100b). When the flexible display device (100b) is completely unrolled, the flexible display device (100b) can display content over the entire area.

[0047] The flexible display device (100b) can display content by changing the layout of the content as the size of the displayed area has changed. As the layout of the content has changed, the position of the area that the user reads / views may also change. When an event occurs in which the shape of the flexible display changes, the flexible display device (100b) can identify the last position of the detected user's gaze. And, the flexible display device (100b) can save the last position of the identified user's gaze and the display screen of the content before the layout change. The flexible display device (100b) can display the content by changing the layout in response to the change in the shape of the flexible display.

[0048] The flexible display device (100b) can determine an area (or point) of content corresponding to the last position of the user's gaze based on the last position of the user's gaze stored and the display screen of the content before the layout change. The flexible display device (100b) can search for a location corresponding to the determined area of ​​content on the content with the changed layout. Then, the flexible display device (100b) can display an indicator (1) at the searched location of the content displayed with the changed layout.

[0049] Alternatively, the flexible display device may display an indicator (1) on the content according to an event in which the user's gaze is lost. For example, as illustrated in FIG. 3c, a foldable type flexible display device (100a) in an unfolded state may display a first content in a first area (11) and a second content in a second area (12). The user may read / view the second content displayed in the second area (12), and the flexible display device (100a) may detect the gaze of the user reading / viewing.

[0050] The user may input a command to change the display area of ​​the second content. The flexible display device (100a) may display the second content, which is displayed only in the second area (12), in the entire area (11, 12) in response to the user's command. That is, the flexible display device (100a) may change the layout of the second content currently displayed. When the layout of the content is changed, it is difficult for the user to recognize the reading / viewing position. Therefore, the flexible display device (100a) may display an indicator (1) on the second content with the changed layout that indicates the reading / viewing position of the user.

[0051] That is, the flexible display device (100) can display an indicator in response to the occurrence of an event in which the shape of the flexible display device changes as well as an event in which the user's gaze is lost. For example, an event in which the user's gaze is lost may include events such as failure to detect the user's gaze, termination of execution of content, quick switching of content, and change in the size of the window in which the content is displayed. Since the process in which the flexible display device (100) detects the user's gaze and displays an indicator on the content with a changed layout is the same as the process described above, a detailed explanation is omitted.

[0052] FIG. 4 is a flowchart illustrating a control method for a flexible display device according to one embodiment of the present disclosure.

[0053] Referring to FIG. 4, the flexible display device displays content on the flexible display (S410) and detects the gaze of a user located on the displayed content (S420). For example, the flexible display device can use a sensor to detect which area (or point) of the displayed content the user's gaze is located on.

[0054] When an event occurs in which the shape of the flexible display changes, the flexible display device identifies the last position of the detected user's gaze (S430), and saves the identified last position and the screen state in which the content is displayed (S440). Then, the flexible display device changes the layout to correspond to the changed shape of the flexible display and displays the content on the flexible display (S450).

[0055] For example, a flexible display device can cluster displayed content into text areas, image areas, etc., based on the attributes of the content area. The flexible display device can identify the location where the user is reading / viewing based on the identified last location and the screen state where the content is displayed. Alternatively, the flexible display device can identify the last location where the detected user's gaze was located when an event occurs in which the user's gaze is lost over the displayed content. For example, an event in which the user's gaze is lost may include the non-detection of the user's gaze, the termination of execution of the content, quick switching of the content, or a change in the size of the window in which the content is displayed.

[0056] A flexible display device can identify the position where a user is reading / viewing based on the last position of the user's gaze and the screen state of the content when the layout of the currently displayed content changes. However, if the first content (old content) that was being executed ends and the second content (new content) is executed after a certain period of time, the flexible display device must determine whether the first content and the second content are the same content. If the first content and the second content are the same content, the flexible display device can display an indicator on the second content at the position where the user was reading / viewing (the last position of the user's gaze on the first content). Thus, the flexible display device can determine the similarity between the first content and the second content. For example, when determining similarity regarding the text area of ​​the content, the flexible display device can identify a preset number of consecutive words including the word at the last position of the first content based on the stored screen state. Then, the flexible display device can determine similarity based on whether the identified preset number of consecutive words are included in the second content. Alternatively, when the flexible display device determines similarity for an image area of ​​content, it may extract image features of the last position of the first content based on a stored screen state and identify similarity based on whether the extracted image features are included in the second content.

[0057] The flexible display device displays an indicator at a location on the content with a changed layout corresponding to the last stored location (S460). Alternatively, if the similarity between the first content and the second content is greater than or equal to a preset value, the flexible display device may identify a location on the second content corresponding to the last stored location and display an indicator at the identified location on the second content.

[0058] Meanwhile, the flexible display device can move the area of ​​the content with a changed layout, where an indicator is displayed, to an area adjacent to the detected user's current gaze position. Alternatively, the flexible display device can transmit the last stored position and the screen state where the content is displayed to an external device where the content is displayed. The above-described embodiment will be explained in detail later.

[0059] FIG. 5 is a flowchart illustrating the process of indicating the last position in new content according to one embodiment of the present disclosure.

[0060] Referring to FIG. 5, the flexible display device displays a first content (S510) and can cluster areas of the first content (S520). For example, the flexible display device can cluster the first content into text areas, image areas, etc.

[0061] The flexible display device can collect user data (S530). For example, the flexible display device can detect the user's gaze and determine the position of the user's gaze on the content.

[0062] The flexible display device can determine whether an event has occurred (S540). For example, the event may include an event in which the shape of the flexible display changes (e.g., rolling, folding, etc.), an event in which the user's gaze is lost, etc.

[0063] If no event occurs, the flexible display device may repeat the clustering process of the first content. If an event occurs, the flexible display device may store reading information (or viewing information) (S550). For example, the reading information may include the detected position of the user's gaze, the screen state of the first content currently displayed, etc.

[0064] The flexible display device may display a second content upon the occurrence of an event. In some cases, the flexible display device may calculate the similarity between the first content (old content) and the second content (new content) (S560). If the calculated similarity is less than or equal to a set value (or less than) (S570-N), the flexible display device may determine that the first content and the second content are different content and terminate the indicator display process. If the calculated similarity exceeds the set value (or is greater than or equal to) (S570-Y), the flexible display device may determine that the first content and the second content are the same content (or similar content) and search for a location on the second content corresponding to the last position of the user's gaze (S580). The flexible display device may display an indicator at the searched location on the second content (S590).

[0065] So far, we have explained the process of displaying indicators on content with changed layouts or new content. Below, we explain the clustering process for content areas.

[0066] FIG. 6 is a diagram illustrating the clustering of content according to one embodiment of the present disclosure.

[0067] Referring to FIG. 6, content including text and images is illustrated. As an example, as shown in FIG. 6, the top and bottom regions of the content may include text regions (3a, 3b), and the center and left regions of the content may include image regions (5a, 5b). A flexible display device can cluster the content into text regions or image regions according to the region attributes of the content. Accordingly, a flexible display device can cluster the top and bottom regions of the content shown in FIG. 6 into text regions (31) and cluster the center and left regions of the content into image regions (51). The flexible display device can determine the similarity of the clustered text regions based on whether a preset number of consecutive words match, and determine the similarity of the clustered image regions based on whether a preset extracted feature matches.

[0068] Meanwhile, a flexible display device can determine the area on the content that the user is reading or viewing by detecting the user's gaze using a sensor. Alternatively, the flexible display device may determine the area the user is reading by determining the reading speed of the text area. Reading speed is explained below.

[0069] FIGS. 7a and 7b are drawings illustrating an embodiment of a process for calculating a user's reading speed.

[0070] As described above, the flexible display device can determine the area being read / viewed by the user based on the detected position of the user's gaze. Alternatively, the flexible display device can determine the area being read / viewed by the user based on the detected position of the user's gaze and a calculated reading speed. Alternatively, if the flexible display device does not include a sensor, it can determine the area being read by the user based on a calculated reading speed. The flexible display device can determine the user's reading speed when the user's gaze is located in a clustered text area or in the case of content containing only text, and can identify the position of the user's gaze based on the determined reading speed.

[0071] Referring to Fig. 7a, Swipe area refers to the text area, and Wread refers to the number of words the user's gaze has scanned. Tstable refers to the time it takes for the screen to stabilize after the initial content is displayed. For example, Tstable may be a short time of about 2-3 seconds. Tswipe refers to the time the user scans the screen with their eyes.

[0072] Referring to FIG. 7b, in one embodiment, the reading time Tread may be the time obtained by subtracting the Tstable and Tnotfocus times from the Tswipe time. Tnotfocus refers to the time when the user does not look at the text area. That is, the reading time may be the time taken for the screen to stabilize and the time when the user's gaze does not stay on the text area, subtracted from the time the user scans the text area with their eyes.

[0073] ARStemp (Average Reading Speed, temporary) refers to a temporary reading speed. The temporary reading speed can be calculated by dividing the number of words read by the reading time. If the temporary reading speed is too large or too small (outside a preset range), the flexible display device may ignore the calculated temporary reading speed.

[0074] The average reading speed of end users can be calculated by adding the currently calculated temporary reading speed (ARStemp) and the average reading speed of existing users (ARSuser), and dividing by the number of times (Count_calculate) calculated to determine the overall average reading speed. The calculated average reading speed can then be stored in memory and updated.

[0075] A flexible display device can determine the user's last gaze position based on the user's average reading speed calculated for a text area.

[0076] FIG. 8 is a drawing illustrating the process of identifying the last position of an image area according to one embodiment of the present disclosure.

[0077] Referring to FIG. 8, a flexible display device can display text containing an image. A user can view the image, and the flexible display device can determine the last position (7) of the user's gaze based on the user's gaze.

[0078] When an event occurs, the flexible display device can store the last position of the user's gaze and the screen state of the content. Based on the stored screen state, the flexible display device can extract image features of the last position of the old content. Then, the flexible display device can identify similarity based on whether the extracted image features are included in the new content. If the similarity is greater than a certain value, the flexible display device can display an indicator at the position of the new content corresponding to the last position of gaze.

[0079] FIG. 9 is a drawing illustrating the process of moving the last position to the user's current line of sight area according to one embodiment of the present disclosure.

[0080] Referring to FIG. 9, content with a changed layout is illustrated. As described above, the flexible display device can display an indicator (1) on the content with a changed layout that corresponds to the last position of the user's gaze. However, the initial position of the user's gaze (7) on the content with a changed layout may differ from the position where the indicator (1) is displayed. The flexible display device can move the area of ​​the indicator (1) displayed on the content with a changed layout to an area adjacent to the detected user's current position of gaze.

[0081] FIG. 10 is a diagram illustrating the process of transmitting information related to the last location to an external device according to one embodiment of the present disclosure.

[0082] Referring to FIG. 10, a first terminal device (100-1) and a second terminal device (100-2) are illustrated. At least one of the first terminal device (100-1) and the second terminal device (100-2) may be a flexible display device. For example, if the first terminal device (100-1) is a flexible display device, the first terminal device (100-1) can store the last position of the user's gaze and the screen state in which the content is displayed through the process described above. The first terminal device (100-1) can transmit the last position of the user's gaze and the screen state in which the content is displayed to the second terminal device (100-2) via an external device (1000). For example, the external device (1000) may include a server, a cloud, etc. Thus, the user can quickly recognize the last position they were reading / viewing even when using another terminal device.

[0083] A control method for a flexible display device according to the various embodiments described above may be provided as a computer program product. The computer program product may include the S / W program itself or a non-transitory computer readable medium on which the S / W program is stored.

[0084] A non-transient readable medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specifically, the various applications or programs described above may be stored and provided on non-transient readable media such as CDs, DVDs, hard disks, Blu-ray discs, USBs, memory cards, and ROMs.

[0085] Furthermore, although preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above. It is understood that various modifications can be made by those skilled in the art without departing from the essence of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical spirit or perspective of the present disclosure. Explanation of the symbols

[0086] 100: Flexible display device 110: Flexible display 120: Sensor 130: Processor 140: Memory

Claims

Claim 1 A flexible display device comprising: a flexible display; a sensor; a memory; and a processor; wherein the processor displays a first content on the flexible display and detects a user’s gaze located on the displayed first content through the sensor; when an event occurs in which the shape of the flexible display changes, identifies the last position of the detected user’s gaze and stores the identified last position and the screen state in which the first content is displayed in the memory; changes the layout to correspond to the changed shape of the flexible display and displays the first content on the flexible display, and displays an indicator at a position on the first content with the changed layout corresponding to the stored last position; when a second content is executed, identifies the similarity between the first content and the second content, and if the similarity between the first content and the second content is greater than or equal to a preset value, identifies a position on the second content corresponding to the stored last position and displays an indicator at the identified position on the second content. Claim 2 A flexible display device according to claim 1, wherein the processor identifies the last location where the detected user's gaze was located when an event occurs in which the user's gaze located on the first displayed content is lost. Claim 3 A flexible display device according to paragraph 2, wherein the event in which the user's gaze is lost includes at least one of failure to detect the user's gaze, termination of execution of the content, quick switching of the content, and change in the size of the window in which the content is displayed. Claim 4 delete Claim 5 A flexible display device according to claim 1, wherein the processor identifies a preset number of consecutive words including the word at the last position of the first content based on the stored screen state, and identifies the similarity based on whether the identified preset number of consecutive words are included in the second content. Claim 6 A flexible display device according to claim 1, wherein the processor extracts image features of the last position of the first content based on the stored screen state, and identifies the similarity based on whether the extracted image features are included in the second content. Claim 7 A flexible display device comprising: a flexible display; a sensor; a memory; and a processor; wherein the processor displays a first content on the flexible display and detects a user’s gaze located on the displayed first content through the sensor; when an event occurs in which the shape of the flexible display changes, identifies the last position of the detected user’s gaze and stores the identified last position and the screen state in which the first content is displayed in the memory; changes the layout to correspond to the changed shape of the flexible display and displays the first content on the flexible display; displays an indicator at a position on the first content with the changed layout corresponding to the stored last position; clusters the displayed first content into at least one of a text area and an image area according to the attributes of the content area; identifies the user’s reading speed when the user’s gaze is located in the clustered text area; and identifies the last position based on at least one of the detected user’s gaze and the identified reading speed. Claim 8 In claim 7, the identified reading speed is an average reading speed, and the processor stores and updates the identified reading speed in the memory, a flexible display device. Claim 9 A flexible display device according to claim 1, wherein the processor moves an indicator area displayed on the first content with the changed layout to an area adjacent to the detected user's current gaze position. Claim 10 A flexible display device according to claim 1, further comprising a communication interface; wherein the processor transmits the stored last position and the screen state in which the first content is displayed to an external device in which the first content is displayed through the communication interface. Claim 11 A method for controlling a flexible display device comprising: a step of displaying a first content on a flexible display; a step of detecting a user’s gaze located on the displayed first content; a step of identifying the last position of the detected user’s gaze when an event occurs in which the shape of the flexible display changes; a step of storing the identified last position and the screen state in which the first content is displayed; a step of displaying the first content on the flexible display by changing the layout to correspond to the changed shape of the flexible display; a step of displaying an indicator at a position on the first content with the changed layout corresponding to the stored last position; a step of executing a second content; a step of identifying the similarity between the first content and the second content; a step of identifying a position on the second content corresponding to the stored last position when the similarity between the first content and the second content is greater than or equal to a preset value; and a step of displaying an indicator at the identified position on the second content. Claim 12 A method for controlling a flexible display device according to claim 11, further comprising the step of identifying the last location where the detected user's gaze was located when an event occurs in which the user's gaze located on the first displayed content is lost. Claim 13 A method for controlling a flexible display device according to claim 12, wherein the event of loss of the user's gaze includes at least one of failure to detect the user's gaze, termination of execution of the content, quick switching of the content, and change of the size of the window in which the content is displayed. Claim 14 delete Claim 15 A method for controlling a flexible display device, wherein the step of identifying similarity comprises identifying a preset number of consecutive words including the word at the last position of the first content based on the stored screen state, and identifying the similarity based on whether the identified preset number of consecutive words are included in the second content. Claim 16 In claim 11, the step of identifying the similarity comprises extracting image features of the last position of the first content based on the stored screen state, and identifying the similarity based on whether the extracted image features are included in the second content, a method for controlling a flexible display device. Claim 17 A method for controlling a flexible display device according to claim 11, wherein the step of identifying the last position comprises clustering the displayed first content into at least one of a text area and an image area according to the attributes of the content area, identifying the user's reading speed when the user's gaze is located in the clustered text area, and identifying the last position based on at least one of the detected user's gaze and the identified reading speed. Claim 18 A method for controlling a flexible display device according to claim 17, wherein the identified reading speed is an average reading speed, and further comprising the step of storing and updating the identified reading speed. Claim 19 A method for controlling a flexible display device according to claim 11, further comprising the step of moving an indicator area displayed on the first content with the changed layout to an area adjacent to the detected user's current gaze position. Claim 20 A method for controlling a flexible display device according to claim 11, further comprising the step of transmitting the stored last position and the screen state in which the first content is displayed to an external device in which the first content is displayed.

Images