Method and apparatus for managing operations on data presented on a display

Abstract

A method for managing operations on data presented on a display is proposed. The method includes: detecting data presented on a display of an electronic device; determining reading parameters associated with the data; and performing multiple operations based on the reading parameters, wherein the reading parameters include at least one of a full screen reading time taken by a user to fully read the data presented on the display of the electronic device, a partial screen reading time taken by the user to read at least a portion of the data, a scrolling distance per scrolling action on the display, or a data reading time as the time taken to fully read the data presented on the display.

Description

Technical Field

[0001] The present disclosure generally relates to a method and system for improving a user's reading experience on an electronic device. Specifically, the present disclosure relates to a method and system for improving a user's reading experience by managing operations on an electronic device based on the user's reading parameters. Background Art

[0002] Electronic devices such as portable electronic devices have been widely used and provide users with various services such as voice call services, messaging services, content presentation services, data management services, etc. Portable electronic devices for handheld use, such as smart phones, tablet computers, e-readers, laptop computers, present various contents to users through the display screens of the respective devices. The widespread use of touch-sensitive displays in portable electronic devices enables users to interact with the contents presented through the touch-sensitive displays using different types of touch gestures. Each time a user needs to perform an action on the contents displayed through the touch-sensitive display of the portable electronic device, the user needs to touch the touch-sensitive display and perform the desired touch gesture on the touch-sensitive display. Summary of the invention

[0003] [Technical solution]

[0004] The proposed method solves existing problems of controlling the operation of electronic devices to provide users with an improved reading experience.

[0005] According to one aspect of the present disclosure, a method for managing various operations on data presented by an electronic device is provided. The method includes detecting data presented on a display of the electronic device; determining reading parameters associated with the data; and performing multiple operations based on the reading parameters, wherein the reading parameters include at least one of a full screen reading time taken by a user to fully read the data presented on the display of the electronic device, a partial screen reading time taken by a user to read at least a portion of the data, a scrolling distance per scrolling action on the display, or a data reading time as the time taken to fully read the data presented on the display.

[0006] [Beneficial Effects]

[0007] Based on the proposed disclosure, management of various operations including page control or scroll control can be achieved for data items presented on the screen of an electronic device. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The method is illustrated in the accompanying drawings, in which the same reference numerals represent corresponding parts in the various drawings. From the following description with reference to the accompanying drawings, the embodiments herein will be better understood, wherein:

[0009] Figure 1A is a block diagram of an electronic device 100 for managing various operations on data presented on a screen according to an embodiment of the present disclosure;

[0010] Figure 1B is a block diagram illustrating operations for data detection and context detection according to an embodiment of the present disclosure;

[0011] Figure 1C is a block diagram illustrating an operation of detecting a scrolling interval between consecutive scrolls on a display 106 associated with an electronic device 100 and a scrolling distance on the display 106 according to an embodiment of the present disclosure;

[0012] Figure 1D is a flow chart 300 illustrating a method of generating one or more training data sets for training the one or more ML regression models using a processor 102 associated with an electronic device 100 to determine reading parameters of a user of the one or more data items presented via a display 106 of the electronic device 100 according to an embodiment of the present disclosure;

[0013] Figure 1E is a block diagram of determining screen reading time according to an embodiment of the present disclosure;

[0014] Figure 1F is a block diagram showing the training of the processor 102 associated with the electronic device 100 to predict the screen reading time of a data file using a first training data set according to an embodiment of the present disclosure;

[0015] Figure 1G is a flow chart 500 illustrating a method for predicting complete screen reading time using one or more trained ML regression models and an input vector generated in real time through one or more extracted contextual features of a data item presented on a display 106 of an electronic device 100 according to an embodiment of the present invention;

[0016] Figure 1H 1 shows the use of data items to determine the time for reading each data item presented through the display 106 or the actual data item reading time according to an embodiment of the present disclosure;

[0017] Fig. 1I is a block diagram illustrating training of the processor 102 associated with the electronic device 100 to predict the reading time of a data item of a data file using a third training data set according to an embodiment of the present invention;

[0018] Figure 1JAn operation for collecting a plurality of users' data item reading time series for various data items presented through the screens 106a, 106b, 106c, and 106d of the respective electronic devices 100a, 100b, 100c, and 100d according to an embodiment of the present disclosure is shown;

[0019] Figure 1K A flowchart for managing the operation of data presented on the display 106 according to an embodiment of the present disclosure is shown;

[0020] Figure 2 An example use case of determining a dynamic screen timeout as one of the reading parameters according to an embodiment of the present disclosure is shown;

[0021] Figure 3 An example of automatically determining a dynamic screen timeout by using the predicted complete screen reading time and presenting the dynamic screen timeout on the electronic device 100 according to an embodiment of the present disclosure is shown;

[0022] Figure 4A and Figure 4B An example of indicating automatic scrolling on the display 106 of the electronic device 100 according to an embodiment of the present disclosure is shown;

[0023] Figure 5 An example of a dynamic content refresh rate for a data item presented by a display 106 of an electronic device 100 according to an embodiment of the present disclosure is shown;

[0024] Figure 6 An example of providing intelligent subtitles on data including video content according to an embodiment of the present disclosure is shown;

[0025] Figure 7 An example of providing a time stamp when presenting a data item through a display of an electronic device according to an embodiment of the present disclosure is shown;

[0026] Figure 8 An example of identifying popular blocks in a data item according to an embodiment of the present disclosure is shown;

[0027] Fig. 9 An example of other users in a suggestion corpus discussing a data item presented through the display 106 of the electronic device 100 according to an embodiment of the present disclosure is shown;

[0028] Fig.10 An example of presenting estimated reading time of data items or individual data files in a data file presented through the display 106 of the electronic device 100 according to an embodiment of the present disclosure is shown;

[0029] Fig.11AAn example of presenting a message indicating user distraction or fatigue through the display 106 of the electronic device 100 according to an embodiment of the present disclosure is shown;

[0030] Fig. 11B An example of presenting an advertisement when a user's attention is distracted or fatigued beyond a predetermined value according to an embodiment of the present disclosure is shown; and

[0031] Fig.12 It is shown that notifications of the electronic device 100 are classified according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] According to one aspect of the present disclosure, a method for managing various operations on data presented by an electronic device is provided. The method includes: detecting data presented on a display of the electronic device; determining a reading parameter associated with the data; and performing multiple operations based on the reading parameter, wherein the reading parameter includes at least one of a full screen reading time taken by a user to fully read the data presented on the display of the electronic device, a partial screen reading time taken by a user to read at least a portion of the data, a scrolling distance per scrolling action on the display, or a data reading time as the time taken to fully read the data presented on the display.

[0033] According to another aspect of the present disclosure, a device for managing multiple operations on data presented on a screen is provided. The device includes: a display that displays data; a processor configured to detect the data presented on the display, determine a reading parameter associated with the data, and perform multiple operations based on the reading parameter, wherein the reading parameter includes at least one of a full screen reading time taken by a user to fully read the data presented on the display of the electronic device, a partial screen reading time taken by the user to read at least a portion of the data, a scrolling distance per scrolling action on the display, or a data reading time as the time taken to fully read the data presented on the display.

[0034] These and other aspects of the embodiments herein will be better understood and appreciated when considered in conjunction with the following description and accompanying drawings. However, it should be understood that the following description, while indicating preferred embodiments and many of their specific details, is given by way of illustration and not limitation. Many changes and modifications may be made within the scope of the embodiments, and the embodiments herein include all such modifications.

[0035] [Invention Mode]

[0036] The embodiments of this article and their various features and advantageous details will be explained more fully with reference to the non-limiting embodiments shown in the drawings and described in detail in the following description. In order not to unnecessarily obscure the embodiments of this article, the description of well-known components and processing techniques is omitted. In addition, the various embodiments of this article do not have to be mutually exclusive, because some embodiments can be combined with one or more other embodiments to form new embodiments. Unless otherwise stated, the term "or" used herein refers to a non-exclusive or. The examples used herein are intended only to help understand the way in which the embodiments of this article can be practiced, and further enable those skilled in the art to practice the embodiments of this article. Therefore, the embodiments should not be interpreted as limiting the scope of the embodiments of this article.

[0037] As is conventional in the art, embodiments may be described and illustrated according to blocks that perform one or more functions described. These blocks may be referred to herein as units or modules, etc., which are physically implemented by analog or digital circuits, such as logic gates, integrated circuits, microprocessors, microcontrollers, memory circuits, passive electronic components, active electronic components, optical components, hard-wired circuits, etc., and may be optionally driven by firmware and software. For example, the circuit may be implemented in one or more semiconductor chips, or implemented on substrate supports such as printed circuit boards, etc. The circuits constituting the blocks may be implemented by dedicated hardware, or by a processor (e.g., one or more programmed microprocessors and associated circuits), or by a combination of dedicated hardware and a processor: dedicated hardware performs some functions of the blocks, and the processor performs other functions of the blocks. Without departing from the scope of the present invention, the various blocks of the embodiments may be physically divided into two or more interactive and discrete blocks. Similarly, without departing from the scope of the present invention, the blocks of the embodiments may be physically combined into more complex blocks. In addition, embodiments of the present disclosure use electronic devices such as, but not limited to, smart phones, tablet computers, personal computers, desktop computers, electronic readers, etc. to perform one or more operations in the present disclosure. In one or more embodiments, the electronic device includes a touch-sensitive screen to enable a user to interact with data items presented via the screen using one or more touch gestures.

[0038] One or more embodiments of the present disclosure use an electronic device to perform one or more operations on one or more data items in a data file stored or accessed using an electronic device. The data file includes a document, a text file, a video file with subtitles, a web page, and other files that can be accessed using an electronic device and present one or more readable contents to a user through the screen of the electronic device. In some embodiments, the content in the data file presented by the screen of the electronic device is referred to as a data item. The data item includes a text portion in a data file, an image portion in a data file, a subtitle portion in a data file, or any combination of these. In some embodiments, the data file contains multiple data items, such as multiple lines of text or sentences in a document, and the multiple data items are presented through the screen of the electronic device. In some other embodiments, the multiple data items presented by the screen of the electronic device can constitute a fixed part in a page or data item. In some embodiments, the data file contains various data items, such as text, images, videos, web content, etc., which are presented to the user through the screen of the electronic device in the form of a continuous sequence or multiple parts (such as pages in a data file). By providing one or more inputs to the electronic device, the user can move from one data item to another, or from one part containing multiple data items to another. In some embodiments, the user can interact with the data items presented by the screen of the electronic device using touch to the screen or using gestures.

[0039] The accompanying drawings are used to help easily understand various technical features, and it should be understood that the embodiments presented herein are not limited by the accompanying drawings. Therefore, the present disclosure should be interpreted as extending to any changes, equivalents and substitutions in addition to those specifically set forth in the accompanying drawings. Although the terms "first", "second", etc. can be used to describe various elements in this article, these elements should not be limited by these terms. These terms are usually only used to distinguish one element from another element.

[0040] Therefore, embodiments of the present invention provide one or more methods and electronic devices for performing one or more operations on an electronic device based on the method to provide an improved reading experience to a user. Embodiments of the present invention disclose one or more methods and electronic devices configured as follows: managing one or more operations on one or more data items presented through a screen of an electronic device based on the reading parameters of the user. The method disclosed herein utilizes an electronic device to perform various operations, including detecting one or more data items (i.e., the content of one or more data items presented through a screen of an electronic device), a context corresponding to one or more data items, and creating one or more machine learning (ML) regression models to operate on the detected data items and the context of the data items. The method disclosed herein also utilizes an electronic device to perform one or more ML regression models to predict one or more reading parameters of a user by operating on the detected data items presented through the screen of the electronic device and the context of the data items. In addition, the method disclosed herein utilizes an electronic device to perform one or more operations on a data item based on the predicted reading parameters of the user to improve the overall reading experience of the user on the electronic device.

[0041] Users are increasingly relying on portable electronic devices with touch-sensitive displays to read various content such as documents, news, articles, web content, and the like. A touch-sensitive display that displays content enables a user to perform actions, such as scrolling content or moving to another content on the touch-sensitive display, by performing touch or scroll gestures on the touch-sensitive display. In addition, if a user opens a specific document for display on a touch-sensitive display of a portable electronic device and wishes to increase the font size of the text in the document to improve the readability of the document, the user needs to perform a zoom touch gesture on the touch-sensitive display. The frequent need for actions on a touch-sensitive display that presents content affects the overall reading experience of a user using a portable electronic device. Alternatively, an accidental scroll touch when reading a document when the display screen is turned on will also hinder the user's overall reading experience.

[0042] In addition, electronic devices including portable electronic devices are typically configured to automatically turn off a display screen including a touch-sensitive display within a predetermined period of time when inactive. The screen timeout that automatically turns off the screen prevents excessive battery usage in the electronic device. If a user is reading content on a touch-sensitive display of a portable electronic device and continues for a predetermined period of time without any touch gestures on the touch-sensitive display, the touch-sensitive display automatically turns off to prevent excessive battery usage of the portable electronic device. In order to continue reading the content, the user needs to turn on the touch-sensitive display by using a touch gesture or by pressing a button on the portable electronic device. Therefore, when the user is reading content on the display screen of the electronic device, whenever there is no response within a predetermined period of time, the display screen automatically turns off, and the user needs to manually turn on the display screen, which in turn affects the user's overall reading experience and productivity.

[0043] In addition, electronic devices are generally configured to provide notifications to users through display screens, including call notifications, message notifications, advertisements, alarm notifications, etc. Notifications are provided in the form of pop-up notifications on top of content presented to the user through the display screen. In the case of an electronic device with a touch-sensitive display, the user can use touch gestures to select features related to the notification, or cancel the notification from the touch-sensitive display. If the user does not respond when receiving the notification, the notification stays on the display screen for a short time and disappears automatically. If the user is reading content on the display screen of the electronic device, and if one or more received notifications are displayed on the content, the user's overall reading experience is affected. In order to continue reading on the touch-sensitive display, the user needs to manually cancel the notification using a touch gesture on the touch-sensitive display.

[0044] A gaze-based display control method is used to minimize reliance on physical interactions such as touch gestures on a touch-sensitive display of a portable electronic device. In the gaze-based display control method, a front camera of a portable electronic device is used to determine a user's gaze direction or viewing position. If the user is reading content, scrolling of the content is performed by determining the user's gaze position using the front camera of the portable electronic device. In addition, by determining changes in the user's gaze direction, the screen timeout or shutdown of the display screen is automatically controlled when reading the content. However, in order to perform the above actions by determining the user's gaze direction and position, it is necessary to continuously track the user's gaze using a front camera, which in turn reduces the battery life of the portable electronic device. Therefore, there is a need for an improved method for automatically controlling one or more operations on an electronic device when reading content on a display screen.

[0045] The present disclosure provides a method for managing one or more operations on a data item presented through a screen of an electronic device based on one or more predicted reading parameters of a user.

[0046] Another object of embodiments of the present invention is to provide a method for predicting a user's reading parameters using one or more machine learning regression models.

[0047] Another object of embodiments herein is to provide a method for predicting a user's reading parameters using the content of a data item, the context of the data item, and a machine learning regression model.

[0048] Another object of the embodiments herein is to provide a method for training a machine learning regression model to predict the scroll distance of a screen displaying data items.

[0049] Another object of embodiments herein is to provide a method for training a machine learning regression model to predict the complete screen reading time taken by a user to completely read a screen displaying a data item.

[0050] Another object of the embodiments herein is to provide a method for training a machine learning regression model to predict a data item reading time when a user reads the data item.

[0051] Another object of embodiments herein is to provide a method for determining a global data item reading time sequence of one or more data items based on the global data item reading time sequences of multiple users in a corpus.

[0052] Another object of embodiments herein is to provide a method for highlighting one or more portions in a data item based on popularity levels of the respective portions determined from a global data item reading time series of the one or more data items.

[0053] Another object of embodiments herein is to provide a method for dynamic screen timeout on an electronic device based on a user's predicted reading parameters.

[0054] Another object of embodiments herein is to provide a method for automatic scrolling on a screen displaying data items based on a user's predicted reading parameters.

[0055] Another object of embodiments herein is to provide a method for determining user fatigue while the user is reading a data item and displaying at least one recommendation associated with the fatigue to the user.

[0056] Another object of embodiments herein is to provide a method for determining when to place a notification or advertisement on a screen while a user is reading a data item on an electronic device.

[0057] Another object of the embodiments herein is to provide a method for recommending multiple other users in a corpus to a user based on reading parameters of the data item by the user and the multiple other users when the user reads the data item.

[0058] Another object of embodiments herein is to provide a method for determining a content refresh rate of one or more data items based on a user's reading parameters.

[0059] Another object of embodiments herein is to provide a method for determining random responses in a survey based on a user's reading parameters.

[0060] Another object of embodiments of this document is to provide a method for presenting subtitles based on a user's reading parameters.

[0061] It is another object of embodiments herein to provide a method for displaying a timer based on a user's reading parameters, the timer indicating an amount of time remaining to complete reading one or more portions of a data item.

[0062] Another object of embodiments herein is to provide a method for indicating a read stamp for a data item based on a user's reading parameters.

[0063] Another object of embodiments herein is to provide a method for automatically prioritizing messages or files at an electronic device based on a user's reading parameters.

[0064] Another object of embodiments herein is to provide a method for determining a plurality of global reading parameters for a particular data item based on a plurality of shared reading parameters of a user with other users in a corpus.

[0065] The method begins by detecting one or more data items displayed on a screen of an electronic device and determining a context of the one or more data items using the electronic device. One or more machine learning (ML) regression models are used to process the one or more data items and the context of the one or more data items to predict a user reading parameter of the one or more data items presented by the electronic device. The method also includes automatically performing an operation at the electronic device based on the predicted reading parameter of the user. In addition, the method of automatically performing an operation on the electronic device based on the predicted reading parameter of the user is intended to improve the overall reading experience of the user on the electronic device.

[0066] In an embodiment, the predicted reading parameters of the user include a full screen reading time taken by the user to fully read one or more data items displayed on the entire screen of the electronic device, a partial screen reading time taken by the user to read at least a portion of the one or more data items displayed on the screen of the electronic device, a scroll distance of the screen displaying at least one data item, and a data item reading time of the one or more data items. Each data item visible on the screen contributes to the full screen reading time.

[0067] In another embodiment, the context of one or more data items presented through the screen of the electronic device is determined based on at least one of the content of the one or more data items, the type of content, the user's familiarity with the content, the state of the electronic device, the user's scrolling pattern, the user's reading ability, and the time and location of the electronic device configured to present the one or more data items to the user.

[0068] Therefore, the embodiments of the present invention also disclose a method for training one or more machine learning regression models to predict the reading parameters of a user. By evenly dividing the screen displaying one or more data items in a data file into a plurality of blocks, for each block, extracting the features of the one or more data items contained in the block, creating a first training data set including the actual time taken by the user to fully read the screen displaying the one or more data items, the features of each block of the one or more data items extracted, and the context of the one or more data items, and creating a second training data set including the actual scrolling distance, the features of each block of the one or more data items extracted, and the context of the one or more data items, the method using the electronic device can train one or more machine learning regression models. In addition, the method includes training a first ML regression model in one or more ML regression models using the first training data set to predict the complete screen reading time taken by the user to fully read the screen displaying the one or more data items. In addition, the method includes training a second ML regression model in one or more ML regression models using the second training data set to predict the scrolling distance of the screen displaying the one or more data items.

[0069] Therefore, the embodiment also discloses a method for determining the visual focus distribution on a screen displaying one or more data items based on the weight pattern of one or more block-level layers of a trained ML regression model. Once the visual focus distribution on the screen displaying one or more data items is determined, the actual complete screen reading time is divided among multiple blocks based on the visual focus distribution. In addition, the method utilizes an electronic device to determine the actual data item reading time of each data item by allocating the actual complete screen reading time divided among the blocks to the data items of the blocks. The method utilizes an electronic device to determine the actual data item reading time of one or more data items based on the number of times the data items are displayed on a continuously scrolling screen. In addition, the method utilizes an electronic device to determine a data item reading time sequence of one or more data items in a data file based on the actual data item reading time of each data item.

[0070] Therefore, the embodiments of the present invention also disclose a method for creating a third training data set, the third training data set including one or more data items, the context of the data items, and the actual data item reading time of each data item. In addition, the method includes training a third ML regression model in the one or more ML regression models using the third training data set to predict the data item reading time.

[0071] Therefore, an embodiment of the present invention also discloses a method, which includes: using a server or an electronic device to collect data item reading time sequences of multiple users for one or more data items; and determining a global data item reading time sequence of one or more data items based on the collected data item reading time sequences of multiple users. In some cases, the global data item reading time sequence of one or more data items is used to determine the popularity level of each part of the one or more data items. The popularity level of each part of the data item is also used to highlight the part of one or more data items displayed on the screen of the electronic device using at least one visual indicator. In addition, the user's predicted reading parameters are used to perform one or more operations on the electronic device to provide the user with an overall improved experience. In one example, the operations performed based on the predicted reading parameters include dynamic screen timeout and automatic scrolling of content corresponding to at least a portion of at least one data item presented through the screen of the electronic device.

[0072] Therefore, an embodiment discloses a method, the method comprising: detecting one or more actual reading parameters of a user for at least one data item; identifying deviations in one or more predicted reading parameters of the user by comparing the actual reading parameters of the user with the predicted reading parameters of the user; and automatically outputting a message indicating the deviation to the user through a screen of an electronic device. The deviation of the predicted reading parameters of the user determined by the electronic device is used to determine when to place a notification or advertisement on the electronic device, and also to determine fatigue of the user, and the electronic device displays one or more recommendations associated with fatigue to the user.

[0073] Therefore, embodiments of the present invention disclose an electronic device having a memory for storing one or more data files (each data file having one or more data items), and a processor configured to perform various operations on the one or more data items based on a plurality of predicted reading parameters of a user. The processor of the electronic device is configured to detect the context of one or more data items displayed on the screen of the electronic device and one or more data items in the data file. The processor is further configured to predict the user's reading parameters of the one or more data items using the one or more data items and the context of the one or more data items on one or more machine learning (ML) regression models. The processor of the electronic device is also configured to automatically perform operations based on the predicted reading parameters of the user, thereby improving the user's overall reading experience on the electronic device.

[0074] Referring now to the drawings, and more particularly to FIGS. 1 to Fig.12 Embodiments are shown, wherein like reference numerals refer to corresponding features throughout.

[0075] Figure 1Ais a block diagram of an electronic device 100 for managing various operations on data presented on a screen according to an embodiment of the present disclosure.

[0076] The electronic device 100 includes a processor 102 configured to perform various operations on data presented on a display 106 of the electronic device. In this specification, the term "screen" may be used interchangeably with the term "display".

[0077] The electronic device 100 also includes a memory 104 and a display 106, the memory 104 is used to store various information, including electronic files or computer-readable data files corresponding to data presented on the display 106, and the display 106 is configured to present data to the user in the format of the computer-readable data file. In an embodiment, the processor 102 of the electronic device detects data that can be presented to the user in a text format through the display 106 of the electronic device 100. In an embodiment, the processor 102 detects the content of the data presented through the display 106 of the electronic device 100, detects the actual scroll interval detector 112 to detect the scroll interval between consecutive scrolls on the display 106 displaying the data, and the processor 102 detects the scroll distance on the display 106 displaying the data.

[0078] Figure 1B is a block diagram illustrating the operations of data detection and context detection according to an embodiment of the present disclosure.

[0079] In an embodiment, the electronic device 100 presents data or an electronic document associated with at least one data file. The data displayed on the display 106 of the electronic device 100 includes one or more sentences in the document and / or an image in the document. The image in the document may include, but is not limited to, a moving picture, such as video content. The processor 102 detects each data item in the document displayed on the display 106 of the electronic device 100. In an embodiment, when the data item is displayed on the display 106 of the electronic device 100, the processor 102 detects each data item in the document. In another embodiment, when the document is stored in the memory 104 of the electronic device 100, the processor 102 detects each data item in the document. The processor 102 determines the context of one or more data items based on the content of the one or more data items, the type of content, the user's familiarity with the content, the state of the electronic device 100 (e.g., the screen orientation of the electronic device 100), the user's scrolling mode, the user's reading proficiency, the time of the day, the time elapsed since the user started reading, and / or the location of the electronic device 100. Throughout the specification, a data item refers to text data, image data, or a combination thereof presented as a computer-readable document file on the display 106 of the electronic device 100 .

[0080] Figure 1Cis a block diagram illustrating an operation of detecting a scrolling interval between consecutive scrolls on a display 106 associated with an electronic device 100 and a scrolling distance on the display 106 according to an embodiment of the present disclosure.

[0081] In an embodiment, the processor 102 is configured to detect a scrolling time interval between one or more consecutive scrolls of one or more data items presented through the display 106 of the electronic device 100. In order to accurately estimate the scrolling time interval, information such as the font size, the space size between words and paragraphs, the screen size, and / or the number of lines skipped by each scroll may be predetermined. The processor 102 is configured to detect a scrolling distance of each scrolling action of the one or more data items presented through the display 106 of the electronic device 100. The processor 102 uses the scrolling time interval and the scrolling distance obtained from the one or more scrolling actions of the user to determine the actual reading parameters of the one or more data items presented through the display 106 of the electronic device 100 by the user.

[0082] In one or more embodiments, the electronic device 100 further utilizes the content of one or more data items presented and detected through the display 106 and the context of one or more data items determined by the processor 102 to predict and / or determine the user's reading parameters for the one or more data items. In an embodiment, the electronic device 100 processes the content of the one or more data items, the context of the one or more data items, and one or more outputs from one or more machine learning (ML) regression models to predict or determine the user's reading parameters for the one or more data items. In an embodiment, the reading parameters may include the full screen reading time taken by the user to fully read the data item displayed on the display 106 of the electronic device 100, the partial screen reading time taken by the user to read at least a portion of the data item displayed on the display 106 of the electronic device 100, the scrolling distance of the display 106 displaying the data item, and the data item reading time. The data item reading time may be the time taken to read the data item on the display 106. For example, due to the complexity or depth of interest of each of the blocks on the screen, the data item reading time for block 1 of the screen may be different from the data item reading time for block 2 of the screen even if blocks 1 and 2 have the same size. The data item may be at least one sentence. When the reading parameters are determined, the electronic device 100 automatically performs one or more operations associated with the reading parameters at the electronic device 100 on the data items presented via the display 106 of the electronic device 100 to provide an improved overall reading experience to the user.

[0083] In an embodiment, the full screen reading time and / or the partial screen reading time are determined by taking into account the font size used in the data item, the overall size of the display, the space size between words, and / or the space size between lines, and the space size between paragraphs. In particular, if the font size used in the data item is large, the full screen reading time can be reduced compared to data with a smaller font size. Similarly, the larger the space size between paragraphs, the smaller the full screen reading time or the partial screen reading time.

[0084] Figure 1D 300 is a flowchart illustrating a method for using a processor 102 associated with an electronic device 100 to generate one or more training data sets for training the one or more ML regression models to determine a user's reading parameters for one or more data items presented via a display 106 of the electronic device 100 according to an embodiment of the present invention.

[0085] In operation 302, the method for generating the first training data set and the second training data set includes detecting by the processor 102 whether a reading application is used to present the data items as a computer-readable document file on the electronic device 100. In operation 304, the processor 102 captures the data items on the current screen content or display 106. In operation 306, the processor 102 separates the captured data items from the display 106 based on the data format such as text, image or other types of formats. In operation 308, the processor 102 vectorizes the separated data items in text format by extracting corresponding text features. In machine learning, a feature vector is an n-dimensional vector representing the digital features of an object. The machine learning model understands data in digital form, so the text features extracted from the text data are converted into vectors (a mathematical way of representing multiple numbers) to uniquely represent each text feature. A vector is a matrix with a number of rows or columns equal to 1. Text features can be extracted from the data items on the screen, such as multiple words included in each block or on the screen, word bags, topic modeling features that capture user interests and / or article types, language proficiency (word familiarity) that indicates whether the word is known to the reader (by storing the most frequently used words), POS tags, TF, IDF. The word bag defines a dictionary of unique words contained in the data item, and then finds the counts of each word in the data item. For example, if a user wants to collect a list of unique words from the "Harry Potter" series 1, and then split the complete list into words for each chapter, the user will get an array with each row corresponding to a chapter and the columns being the counts of each word. Therefore, "vectorizing the data items" means extracting and / or calculating text features from the content (data items) on the screen and representing them as row / column vectors.

[0086] In operation 310, the processor 102 converts the separated data items in the image format captured from the display 106 into a matrix vector. The RGB image is stored as an m×n×3 data array that defines the red, green, and blue components for each individual pixel. A simple feature representation of an image is the raw intensity value of each pixel. The intensity values ​​form an m×n×3 matrix that is input to the neural network ML regression model. In an embodiment, the m×n×3 values ​​are expanded into row / column vectors to be used as feature vectors of the image in the ML regression model. The feature matrix is ​​passed to a convolutional neural network (ML regression model), which further extracts graphic features such as shape, object type, color, texture, action, etc., and uses them to generate the desired predictions / outputs based on training.

[0087] In operation 312, the processor 102 extracts contextual features associated with the captured data items in text format and image format. Factors other than the content presented on the screen can affect the time spent on the screen before scrolling. These factors are captured as contextual features, such as the time elapsed since the start of the reading activity, the orientation of the screen, the multitasking state of the application (whether background music is playing, the number of multi-windows / floating windows), the current time, the current location of the electronic device, the font size of the data item, the font style of the data item, the user's level of engagement, etc.

[0088] In operation 314, processor 102 connects the extracted context features and generates an input vector. In other words, feature vectors such as text feature vectors, context feature vectors, and (image) matrix vectors are grouped to form a single input feature instance and are input to the ML regression model for learning and training. In an embodiment, all feature vectors can be cascaded / connected together to form an input feature vector. If a vector is 1 column by P rows, other vectors are 1 column by Q rows. Then, after the cascade result, the vector will be 1 column by P+Q rows.

[0089] In operation 316, when the processor 102 detects a scrolling event on the display 106, it makes at least one reading time-input vector pair for the first training data set and at least one scrolling distance-input vector pair for the second training data set. The training data set may include pairs of input vectors (feature vectors) and corresponding output vectors (labels / targets). In the process of training the ML regression model, the current model is run using the training data set and a result is generated, which is then compared with the target for each input vector in the training data set. For each scrolling action of the user, the input feature vector of the screen (a combination of the text feature vector (image) matrix vector and the context feature vector of the screen) and the reading time of the screen (the time elapsed from the last scroll to the current scrolling action) are paired to form an {input feature vector-screen reading time} pair of the display screen. The pair is a training instance of the training data set. Similarly, the input feature vector and the scrolling amount / distance of the current scrolling action are paired to form an {input feature vector-scrolling distance} pair of the display screen.

[0090] In operation 318, the processor 102 adds the reading time-input vector pair to the reading time data set or the first training data set, and adds the scrolling distance-input vector pair to the scrolling distance data set or the second training data set. In operation 320, the processor 102 detects whether the reading application is inactive in the electronic device 100, and exits the training data set generation process in operation 322. The first and second training data sets generated in this way are used as inputs of the processor 102 for further processing. The generated {input feature vector-screen reading time} pairs and {input feature vector-scrolling distance} pairs are collected for each displayed screen during each scrolling action. The data set formed by collecting the {input feature vector-reading time} pairs is called the first training data set, and the data set formed by collecting the {input feature vector-scrolling distance} pairs is called the second training data set.

[0091] Figure 1E is a block diagram of determining screen reading time according to an embodiment of the present disclosure.

[0092] Reference Figure 1EThe architecture of a neural network ML regression model is described and used to predict screen reading time. In an embodiment, the processor 102 of the electronic device 100 divides the screen of the display 106 displaying the data item into multiple blocks: block 1, block 2, block 3...block N, where N is an integer. In addition, the processor 102 extracts various features of the contextual features of the data items included in each block in the screen of the display 106. The processor 102 generates a first training data set, which includes the actual complete screen reading time (i.e., the time taken by the user to actually read the data item presented on the display 106), the features extracted from each screen block, and the context of the data item. The processor 102 or a remote server (not shown) uses the first training data set to train a first ML regression model to predict or determine the complete screen reading time taken by the user to completely read the data item presented on the display 106. The predicted or determined complete screen reading time will be as equal to the actual complete screen reading time as possible. Generate training data including a set of training examples. Each training example is a pair consisting of an input vector extracted from the screen content or data item and an expected output value of the actual complete screen reading time. The training data is fed to a first ML regression model, such as a deep neural network, to use the first ML regression model ( Figure 1E The ML regression model is trained using the training weights of the initial block-level network shown in Figure 1. Once the ML regression model is trained, the trained ML regression model can output screen reading time (time spent on the screen before scrolling) for any given content on the screen based on the input, i.e., the input feature vector (a numerical representation of the screen content). The ML regression model can be composed of artificial neurons (mathematical functions) and weights associated with the neurons. The activation function of the artificial neurons can be linear, sigmoid, rectified, etc. Figure 1E The architecture in shows how inputs are given to the proposed neural network and how the various layers / neurons are interconnected. At the beginning of training, the weights of the ML regression model associated with the interconnections between neurons are randomly initialized. When training an ML regression model, the current model with the current weights is run using the input features of the training dataset and produces results. The results are then compared to the labels in the dataset to determine how to adjust the weights so that the results become closer to the labels / targets. The iterative process of gradually adjusting the weights can be done using algorithms such as stochastic gradient decay, RMSProp, etc. Figure 1E The architecture in is the same for the screen reading time prediction model and the scrolling distance prediction model. For the scrolling distance prediction model, the second training dataset consisting of {input feature vector-scrolling distance} pairs is trained with Figure 1E Neural networks of the same structure. For any given screen content, the trained ML regression model can output the amount of scrolling distance the user has to scroll to the next content based on the input, namely the input feature vector (a numerical representation of the current screen content).

[0093] In an embodiment, the processor 102 generates a block-level network and a page or screen-level network for data items presented on the display 106 of the electronic device 100. For each block, the Figure 1D The extracted text feature vector discussed. Further, the processor 102 maps the input vector, i.e., the vector of the extracted text features, to the corresponding blocks and pages of the data item presented on the display 106 of the electronic device 100. In an embodiment, for block 1, block 2, block 3, ... block n of the page of the data item presented on the display 106, the extracted text features are provided to the corresponding block-level network. Figure 1E , the entire page presented on the display 106 is divided into block 1, block 2, block 3, ... block n. The page can be divided evenly. For example, each rectangle in the screen 126 corresponds to each block with a fixed width.

[0094] In addition, the processor 102 generates an image network by mapping the matrix vectors to the corresponding images 136 in the data items. In addition, the block-level layer weights are initialized according to a normal distribution. Figure 1E , "m" represents the number of layers in the block-level network 151, "n" represents the number of layers in the page or screen-level network 153, and "p" represents the number of layers in the combined network 157. In addition, the processor 102 fully connects the block-level network to the page or screen-level network, and the screen-level network is further combined with the image network to incorporate each contribution of the content or data item presented on the display 106 of the electronic device 100.

[0095] Figure 1F is a block diagram illustrating training of the processor 102 associated with the electronic device 100 to predict the screen reading time of a data file using a first training data set according to an embodiment of the present disclosure.

[0096] The data file includes one or more pages, each page having one or more data items presented through one or more screens of the electronic device 100: screen 1, screen 2, screen 3, ... screen N (N is an integer). In the first training data set, the extracted feature vectors of each data item presented in screens 1 to N of the electronic device 100 and the context of the one or more data items presented in screens 1 to N are mapped to the corresponding actual scrolling intervals T1, T2, T3, ... T detected using the processor 102. N. As previously described, the feature vector may include a text feature vector, a context feature vector, and an image matrix vector. In an embodiment, the screen reading time prediction model takes as input a feature vector derived from the content and context of the screen during the training phase of the model and the prediction phase. The screen reading time prediction model is trained with a first training data set of mapping pairs of feature vectors and actual screen reading times containing data items on the screen. The screen reading time prediction model may take the feature vector as input and, as output, predict the screen reading time. The screen reading time prediction model is trained to learn the weights of the model to match the predicted screen reading time with the actual reading time in the training data set.

[0097] like Figure 1F As shown, feature vectors 1, 2, 3, ... N in the feature vectors of the extracted data items correspond to screen 1, screen 2, screen 3, ... screen N, respectively. The mapping of screens displaying data items, contexts, and actual scrolling intervals between screens including screen 1 to screen N thus obtained is processed using an ML regression model to predict the complete screen reading time taken by a user to completely read the screens including screen 1 to screen N and display the data items.

[0098] Figure 1G 500 is a flowchart illustrating a method for predicting complete screen reading time using one or more trained ML regression models and an input vector generated in real time by one or more extracted contextual features of a data item presented on a display 106 of an electronic device 100 according to an embodiment of the present invention.

[0099] The method for predicting the complete screen reading time includes, in operation 502, using the processor 102 to detect whether a reading application has opened a data file having one or more data items. The data file may be a document file containing one or more data items such as text data and / or image data. In operation 504, the processor 102 captures the current screen content or data item presented by the display 106. In operation 506, the processor 102 separates various types of data formats in the data item. The type of the data format may be at least one of text or image. In operation 508, the processor 102 vectorizes the separated text data by extracting text features, and in operation 510, saves the text vector, the image vector, and any other context feature vector as an input vector to evaluate the screen reading time predictor 116. The screen reading time predictor 116 may be implemented as the processor 102. In operation 512, the processor 102 uses the input vector as one of the inputs of the trained first ML regression model to predict or determine the complete screen reading time. In operation 514, when the user reads content presented via the entire screen, the screen timeout of the entire screen is dynamically adjusted using the processor 102 using the predicted or determined complete screen reading time obtained for a specific content or data item presented on the display 106. When the processor 102 detects that the reading application is inactive in operation 516, the processor 102 controls the stop operation in operation 518.

[0100] In one embodiment, the processor 102 of the electronic device 100 uses a reference Figure 1D Generating a second training data set. The second training data set may include the actual scroll distance, the extracted features of each block, and the context of the data item to train a second ML regression model in the one or more ML regression models to predict the scroll distance on the display 106 displaying the one or more data items. The actual scroll distance is the difference in vertical position of the same portion of the data item in two consecutive screens in response to a scroll action.

[0101] In an embodiment, the processor 102 trains a second ML regression model using a second training data set to predict the scrolling distance between the current scroll and the next scroll of a given screen displaying a data item. In addition, the processor 102 uses the same input vector in the trained second ML regression model to predict or determine the scrolling distance. The processor 102 automatically scrolls the display 106 displaying the data item using the obtained predicted or determined scrolling distance. Based on the trained second ML regression model, the predicted or determined scrolling distance will be as close as possible to the actual scrolling distance. In an embodiment, training data including a set of training examples is generated. Each training example is a pair including an input vector extracted from the screen content and an expected output value of the scrolling distance. The training data can be fed to the trained second ML regression model for training, and the next scrolling distance is determined based on user behavior and the trained second ML regression model.

[0102] Figure 1H Utilization of data items according to an embodiment of the present disclosure to determine a time for reading respective data items presented through the display 106 or an actual data item reading time is shown.

[0103] Instead of reading the data items from top to bottom, the user usually reads a portion of the data items presented by the display 106, and scrolls the screen to bring the desired content or data items into the continuous screen or page of the data file according to the user's reading mode or visual focus distribution. The visual focus distribution can represent the degree of attention of the user about the various blocks of the screen or the distribution of the attention level. The visual focus distribution may not be associated with the data items or content included in the various blocks, but with the physical area of ​​the screen. Instead of reading the screen from top to bottom, the user usually reads a portion of the screen and keeps scrolling to bring the content to the portion according to his reading mode and visual focus distribution. In addition, the user's visual focus mode or visual focus distribution is mostly constant over a period of time. Over a long period of time, there may be only a small gradual change. The visual focus distribution can be estimated based on the scrolling action and the complete screen reading time and / or the actual complete screen reading time.

[0104] The screen reading time prediction model will learn to predict the total time the user spends on the entire screen using features extracted from the data items on the screen. Due to visual focus patterns or visual focus contributions, all regions and blocks of the screen do not have equal contributions to the total screen reading time. Therefore, in the learning process of predicting the complete screen reading time, the complete screen reading time model also learns which regions or blocks of the screen have high or low contributions to the complete screen reading time. In order to be able to extract this learning of visual focus contributions, a method is introduced to divide the screen into uniform regions or blocks and extract features from data items at the block level rather than the entire page level.

[0105] Features extracted from specific areas or blocks of the screen are input only to the corresponding block-level network. Block-level networks are not interconnected with each other. Therefore, the information flow before inputting the page-level network is guided, and the block-level layer acts as a gate for the information flow. During training, the weights of the block-level network must be adjusted so that blocks with higher user focus and higher contribution to screen reading time have a larger amplitude that allows more information flow. And blocks with less focus and less contribution have a smaller amplitude for limiting information flow because it is less relevant to screen reading time. Therefore, the weight pattern between block-level networks follows the visual focus pattern. For example, if the screen is divided into P blocks, we determine P values ​​for each block. For each block-level network, take the average of the weight size of the network. This can be P values. The maximum value of these P values ​​can be found, and all P values ​​are divided by the maximum value. The resulting P values ​​are the visual focus distribution (value) F(i) of the i-th block. These P values ​​range from 0 to 1, and the sum of all values ​​is 1.

[0106] For 1≤i≤P, 0≤F(i)≤1,

[0107] For any block i, its corresponding visual focus distribution value (i) represents the fraction of the contribution of block i to the complete screen reading time. In an embodiment, the initial value of (i) can be taken as the known visual focus distribution of the general user group. And, as described above, the trained weights of the block level in the screen reading time model can be used to modify the initial value of the visual focus distribution according to the user's personal reading pattern.

[0108] In some cases, the user may happen to spend more time on certain data items, and the changes in the reading time of each data item (for each block) are captured. The visual focus distribution of the user on the display 106 indicates the focus level or attention level of the user on each block of the data item on the display 106. Initially, the standard visual focus distribution or the most common visual focus distribution F(i) on the block is used as the default visual focus distribution, where "i" represents the block position on the display 106 where the data item is displayed. In order to determine the visual focus distribution of each user, the processor 102 modifies F(i) based on the weight pattern of the block level layer of the trained ML regression model. In one example, the actual complete screen reading time of the data item displayed on the display 106 is used to determine the reading time of each block of the screen. The reading time of each block can be the block reading time spent for reading each block. The actual complete screen reading time is distributed to each block of the screen based on the visual focus distribution value F(i) to determine the reading time of each block of the screen. The actual reading time of the i-th block of the screen is determined using equation (2).

[0109] t i ab= F (i) * t n as Equation (2)

[0110] where t i ab is the actual reading time of the ith block, t n as is the total time spent on screens 1 to N, where n=1 to N.

[0111] here,

[0112] In some cases, successive screens after each scroll by the user include overlapping or repeated blocks of data items. In this case, the reading time of the overlapping blocks is determined by combining or adding the individual reading times of the overlapping blocks, t ab i1 +t ab i2 .

[0113] Once the visual focus distribution of the weight pattern of the block level layer based on the trained ML regression model is determined using the visual focus distribution detection performed by the processor 102, the processor 102 divides the actual full screen reading time between the blocks in the display 106 based on the determined visual focus distribution. In addition, the processor 102 determines the actual (data item) reading time of a specific block on the display 106 by allocating the divided actual full screen reading time to each block in the display 106. For overlapping blocks in continuous screens, the actual reading time of the data items in the overlapping blocks is determined based on the number of times the data items are displayed on the display 106 when scrolling continuously. Overlapping blocks in continuous screens means that for the data items that stay on the screen when scrolling continuously, all the reading times calculated for multiple times (overlapping) are combined or added together. For any given document read by a user, a sequence of data items and their actual data item reading times is generated and referred to as a reading time sequence of data items or a data item reading time sequence. In addition, the processor 102 determines the actual reading time sequence of data items in the data file based on the actual reading time of each block.

[0114] Once the actual reading time sequence of the data items in the data file is determined, the processor 102 generates a third training data set including the data items, the context of the data items, and the actual reading times. The processor 102 also trains a third ML regression model from the one or more ML regression models using the third training data set to predict the data item reading times in the display 106. In one example, the data items may be sentences, and the predicted data item reading times may be sentence reading times.

[0115] Fig. 1Iis a block diagram illustrating training of the processor 102 associated with the electronic device 100 to predict the reading time of a data item of a data file using a third training data set according to an embodiment of the present invention.

[0116] The processor 102 is used to predict the amount of time a user spends on a specific block (such as a line or sentence) of a data item displayed on the display 106. The processor 102 determines the actual data item reading time or the time used to read each data item, and maps the actual data item reading time to the determined feature vector as the screen reading time and scroll distance prediction are determined, which is used to train a third ML regression model to predict or determine the block reading time or sentence reading time or line reading time. The output of the data item reading time prediction performed by the processor 102 can be further used to provide intelligent subtitles when playing a video on the electronic device 100.

[0117] In an embodiment, when a user reads the contents of a data file or a data item on the display 106 , the processor 102 detects the degree of distraction of the user.

[0118] When a user reads data on the display 106 of the electronic device 100 for a long time, he or she may feel reading fatigue. After fully training the first and second ML regression models, the complete screen reading time can be accurately predicted. Then, the processor 102 compares the output, that is, the predicted complete screen reading time and the actual complete screen reading time spent by the user reading the data item of the current screen of the display 106 within a predetermined time period. Based on the comparison, it is determined that the actual complete screen reading time is continuously increased (deviation) with a gap (predetermined value) compared with the predicted complete screen reading time. In an embodiment, the deviation may indicate that the user is distracted from reading in two consecutive times. In addition, the continuous increase in the actual complete screen reading time, for example, more than three times, indicates that the user is distracted. In an embodiment, the processor 102 compares the predicted scroll distance with the actual scroll distance to determine whether a skip of a data item or page or screen presented by the electronic device 100 occurs. The output of the comparison is used to determine the degree of fatigue or distraction of the user, and to appropriately schedule a pop-up or warning notification of an advertisement to the user through the electronic device 100 based on the degree of distraction of the user. The warning notification may include a warning for stopping reading the data item and taking a break for a period of time.

[0119] Figure 1J An operation for collecting data item reading time series of a plurality of users for various data items presented through the screens 106a, 106b, 106c, and 106d of the respective electronic devices 100a, 100b, 100c, and 100d according to an embodiment of the present disclosure is illustrated.

[0120] The reading data of each user, including the reading parameters, is shared by multiple users to the server 132 via a network. In an embodiment, the processor 102 of the electronic device 100 or the server 132 is configured to enable the data item reading time series of multiple users to be collected for various data items presented through the screens 106a, 106b, 106c and 106d of the respective electronic devices 100a, 100b, 100c and 100d. The data item reading time series may represent a collection of data item reading time spent by multiple users for a specific data item.

[0121] The processor 102 or the server 132 collects the actual data item reading time of various contents or data items and stores the actual data item reading time of various contents or data items in the server 132. The server 132 stores the actual data item reading time of each user based on the type and content of the data item for multiple users. When the user starts to read a specific data item, the processor 102 retrieves the global data item reading time sequence of the specific data item and presents it to the user through a graphical user interface (GUI) on the display 106 of the electronic device 100. The global data item reading time sequence of the specific data item can be a series of average reading times for each data item collected from multiple users. That is, the data item reading sequences of multiple users of the document are combined into the global data item reading time sequence of the specific document.

[0122] The processor 102 of the electronic device 100 continuously collects the reading parameters of each user in order to analyze and further visualize various metrics through a dedicated dynamic GUI in the electronic device 100. The processor 102 collects data from the actual reading content or data items of the user and data from various other components and stores them in the memory 104. The collected data is classified and presented through the dynamic GUI on the display 106 of the electronic device 100. The dynamic GUI provides the following information, including the meaning of the new words encountered, the meaning of the infrequent words, the predicted and actual reading time during a specific time period such as a week, a month, an hour, etc., so that the user can evaluate the improvement of various reading parameters such as the full screen reading time, the partial screen reading time, the scrolling distance of each scrolling action, or the data reading time. In addition, the processor 102 enables the user to share the collected information with other users through social media or other platforms. In an embodiment, the electronic device 100 or the server 132 uses the global data item reading time series of one or more data items to determine the popularity level of each part of one or more data items. The electronic device 100 further utilizes popularity levels of various portions of one or more data items to highlight portions of the data items displayed on the display 106 using at least one visual indicator.

[0123] Figure 1KA flow chart for managing the operation of data presented on the display 106 according to an embodiment of the present disclosure is shown.

[0124] In operation 1001 , the processor 102 detects data (data items in a data file) presented on the display 106 of the electronic device 100 .

[0125] In operation 1003 , the processor 102 uses the electronic device 100 to determine a reading parameter of a user associated with the data.

[0126] In operation 1005, the processor 102 performs a plurality of operations based on the reading parameters. In an embodiment, the reading parameters may include a full screen reading time taken by a user to fully read the data presented on the display 106 of the electronic device 100, a partial screen reading time taken by a user to read at least a portion of the data, a scrolling distance of each scrolling action on the display 106, or a data reading time as the time taken to fully read the data presented on the display 106. When the data is presented on a portion of the display 106 of the electronic device 100, the data reading time may be different from the full screen reading time. The partial screen reading time may be the time taken to read a portion of the data when the screen is divided into a plurality of blocks. The plurality of blocks may be evenly divided.

[0127] Figure 2 An example use case of determining a dynamic screen timeout as one of the reading parameters according to an embodiment of the present disclosure is shown.

[0128] Predicted reading parameters are used to determine a dynamic screen timeout on the electronic device 100 when a user reads a data item on the display 106 of the electronic device 100, wherein the predicted reading parameters include a complete screen reading time predicted to be taken by the user to completely read the data item displayed by the processor 102 on the display 106 of the electronic device 100. The dynamic screen timeout also depends on the content of the screen content or data item presented by the display 106 of the electronic device 100 and the user's relationship to the content, and supervised machine learning of the processor 102 is used to calculate the amount of time the user spends on the data item presented on the display 106 before automatic scrolling. After the dynamic screen timeout has passed, the screen may be changed to display a next page of the data item including a different portion. For example, if the portion of the data item is related to a specific technology, the electronic device 100 automatically sets the dynamic screen timeout to 1 minute because content related to the specific technology may require more time to read, such as Figure 2 If the specific portion of the data item is related to sports content, the electronic device 100 automatically sets the dynamic screen timeout to 40 seconds, as shown in FIG. Figure 2(b). In addition, according to an embodiment, the electronic device 100 sets a dynamic screen timeout based on the type and size of the font in the display 106. For example, the smaller the font size, the longer the dynamic screen timeout. For example, if the font size is very small, the user reading time also increases, and the dynamic screen timeout is automatically set by the electronic device 100 to 2 minutes, as shown in FIG. Figure 2 (c) is shown. In addition, the electronic device 100 sets a dynamic screen timeout based on the size and count of the image presented on the display 106. For example, if the size of the image on the screen is Figure 2 (d) is shown as large, the amount of text content or data item is small, and the dynamic screen timeout is automatically reduced and set to 30 seconds by the electronic device 100. The dynamic screen timeout saves battery and prevents undesirable interruptions to the user when reading a specific portion of a data item on the display 106 of the electronic device 100.

[0129] Figure 3 A use case of automatically determining a dynamic screen timeout by using the predicted complete screen reading time and presenting the dynamic screen timeout on the electronic device 100 according to an embodiment of the present disclosure is shown.

[0130] Dynamic screen timeout can be presented through the GUI of electronic device 100. When the user reads the data items on display 106, such as Figure 3 (a) shows the remaining time of the dynamic screen timeout, for example, 5 minutes. In an embodiment, a graphical indicator such as a dynamic horizontal progress bar dynamically presents the remaining time for reading the entire data item presented on the display 106 of the electronic device 100, such as Figure 3 (b) As shown. In the embodiment, Figure 3 As shown in (c), a dynamic vertical progress bar is displayed that dynamically presents the remaining time for reading the entire data item presented on the display 106 of the electronic device 100.

[0131] Figure 4A and Figure 4B A use case of indicating automatic scrolling on the display 106 of the electronic device 100 according to an embodiment of the present disclosure is shown.

[0132] exist Figure 4A In (a) and (b), the GUI of the electronic device 100 highlights or changes the text color or transparency of some data items on the display 106 as a graphic indicator to the user that the highlighted portion will automatically scroll up or scroll out of the interface within a predetermined time.

[0133] exist Figure 4BIn (c) and (d), the GUI of the electronic device 100 indicates to the user that automatic scrolling is about to be performed by the electronic device 100 in the form of a bottom animation or as an edge progress bar. In an embodiment, the color of the animation or arrow changes according to the remaining time of the next automatic scrolling on the display 106 of the electronic device 100. For example, in Figure 4 (c), as the remaining time of the next automatic scrolling becomes smaller, the color of the animation becomes darker. In another example, in Figure 4 (d), as the remaining time of the next automatic scrolling becomes smaller, the arrow moves toward the bottom of the vertical side or edge of the display 106. The edge of the display can be a feature screen that curves around one or both edges of the electronic device 100.

[0134] Figure 5 An example of a dynamic content refresh rate regarding a data item presented through the display 106 of the electronic device 100 according to an embodiment of the present disclosure is shown.

[0135] The output from the screen reading time prediction is used to determine the content refresh rate of the content or data items presented by the display 106 of the electronic device 100. Figure 5 As shown, if the display 106 is displaying dynamically updated content, such as the score of a sports game, the processor 102 dynamically adjusts the refresh rate of the content in the display 106 based on the screen reading time, where the screen reading time is determined by the processor 102 using the complete screen reading time. In some cases, the processor 102 dynamically adjusts the refresh rate of the content in the display 106 based on the data item reading time, where the data item reading time is determined as the time for reading a data item or content in the display 106. When the predicted data item reading time expires, the processor 102 automatically refreshes the content.

[0136] In addition, the electronic device 100 continuously monitors the user's reading parameters and compares them with the predicted reading parameters. When the electronic device 100 detects a deviation in the predicted reading parameters such as the user's predicted scrolling distance by comparing the reading parameters including the user's actual scrolling distance with the predicted scrolling distance, the processor 102 automatically adjusts the amount of the automatic scrolling distance to provide a better reading experience for the user.

[0137] Figure 6 An example of providing intelligent subtitles with respect to data including video content according to an embodiment of the present disclosure is shown.

[0138] Based on the data item reading time determined by the processor 102 based on the trained third ML regression model, the electronic device 100 adjusts the subtitles on the video content being reproduced through the display 106 of the electronic device 100. In some cases, the subtitles based on the user's predicted data item reading time can be presented to the user through the screen of the electronic device 100 in different colors or font sizes or font types. In some other cases, the processor 102 can control the amount of subtitles displayed with the video content being played on the display 106 of the electronic device 100 based on the predicted data item reading time to avoid (for example) blocking the video image by the subtitles. The longer the predicted data item reading time, the smaller the amount of subtitles to be displayed with the video content.

[0139] Another exemplary case of using the data item reading time based on the trained third ML regression model is in a standardized test performed using the electronic device 100. The data item reading time for each user of the test is calculated using the processor 102, and the total reading time for all questions is added to the total test time limit. This allows users with dyslexia to be provided with additional time when participating in standardized tests using the electronic device 100. For example, the total time used to solve the problems in the test includes the problem reading time and the problem solving time. The present method takes into account the user's dyslexia and the problem reading time of the user with dyslexia is different from the problem reading time of other users participating in the standardized test using the electronic device 100. However, the problem solving time remains the same for all users taking standardized tests using the electronic device 100.

[0140] Figure 7 An example of providing a time stamp when presenting a data item through a display of an electronic device according to an embodiment of the present disclosure is shown.

[0141] When a user sends one or more data items as a text message to another user using the electronic device 100, a processor in the receiving device determines a predicted reading time for reading the text message, and sends the predicted reading time of the receiving device to the sending electronic device 100. Based on the predicted reading time of the receiving device, a reading timestamp or a read timestamp is presented through the GUI of the electronic device 100.

[0142] Figure 8 An example of identifying popular blocks in a data item according to an embodiment of the present disclosure is shown.

[0143] The total time spent by each user reading each data item presented on the display 106 is collected by the processor 102 or shared with the server 132. The server 132 calculates or estimates the reading time of each block of the data item and classifies each block having a portion of the data item based on the reading time. The server 132 compares the reading time of each block having a portion of the data item with the threshold value of each block and classifies each block into a "high", "medium" or "low" reading time category. The category of "high" reading time can represent the high importance of the block in the data item. The category can be more specifically divided into, for example, 1 (lowest) to 5 (highest). In an embodiment, the threshold value can be the average reading time of a specific block of the data item calculated based on the reading time of the specific block collected globally. When a user accesses a specific block of the data item, the electronic device 100 presents visual indications for the various blocks to enable the user to quickly overview the data item and skip non-important blocks of the data item indicated by the visual indication.

[0144] Fig. 9 An example of suggesting other users in a corpus to discuss a data item presented through the display 106 of the electronic device 100 according to an embodiment of the present disclosure is shown.

[0145] In an embodiment, if a user is reading a data item or a paragraph in a data item, other users who spend a high amount of time on the particular paragraph are suggested to the user to discuss the particular paragraph based on the reading time collected globally in the server 132. In an embodiment, the server 132 suggests users who have previously connected or accessed the particular paragraph or data item. The suggested users may be selected from the user's social media friends. In an embodiment, the server 132 suggests any other users to the user whose at least one of the reading parameters matches or substantially matches. In an embodiment, a graphical identifier (e.g., a facial image or a chat head) of the suggested user may be displayed on or near a particular paragraph presented on the display 106, with reference to the user's social media friends. Fig. 9 9. In an embodiment, a graphical identifier may be displayed on an edge portion of the electronic device 100 corresponding to a specific paragraph presented on the display 106. The user may touch the suggested chat head and an expanded list of friends may appear to start a conversation about the specific paragraph of the data item presented through the display 106. The specific paragraph may correspond to the reference Figure 8 a specific block.

[0146] Fig.10 An example of presenting estimated reading times of data items in a data file or respective data files presented through the display 106 of the electronic device 100 according to an embodiment of the present disclosure is shown.

[0147] The server 132 stores the total time spent by various users reading data items in a particular data file. The processor 102 is used to determine the actual total time spent reading data items in a particular data file. Based on the received actual total time spent reading the data items, the server 132 determines an estimated reading time for the data items in the data file, and then determines a global data item reading time for the particular data item by collecting the data item reading times of the various users. Then, as Fig.10 As shown in (a), the estimated reading time of the data items in the data file is presented through the GUI of the electronic device. In one embodiment, when the user reads the data file, the display 106 of the electronic device 100 presents an animation corresponding to the estimated remaining time to complete reading a specific portion of the data items in the data file, such as Fig.10 (b) as shown.

[0148] Fig.11A An example of presenting a message indicating user's distraction or fatigue through the display 106 of the electronic device 100 according to an embodiment of the present disclosure is shown.

[0149] The degree of distraction of the user when reading the data items in the display 106 of the electronic device 100 is determined by detecting the user's actual reading parameters, i.e., the actual full screen reading time and the actual scrolling distance for at least one data item, and by comparing the user's actual reading parameters with the user's predicted reading parameters, identifying the deviation from the user's predicted reading parameters (i.e., the predicted full screen reading time and the predicted scrolling distance for at least one data item). When it is determined that the deviation is higher than a predetermined value, the electronic device 100 is configured to output a message or notification 1101 indicating the deviation to the user. In addition, the processor 102 of the electronic device 100 presents one or more advertisements 1103 after or instead of the message or notification through the display 106, as shown in reference Fig. 11B shown.

[0150] Furthermore, according to an embodiment, the degree of user distraction is used to determine the time to place a notification on the electronic device 100. While reading one or more data items on the display 106 of the electronic device 100, if the degree of user distraction indicates that the user distraction level is high, the electronic device 100 automatically delays the received notification on the electronic device 100. In another example, the notification is delayed until it is determined by the processor 102 that the user has finished reading the current sentence or block of data items in the display 106. In another example, the user can preset a delay time by which the notification can be delayed while the user is reading the data items on the display 106 of the electronic device 100.

[0151] Fig.12 It is shown that notifications of the electronic device 100 are classified according to an embodiment of the present invention.

[0152] Notifications received on the electronic device 100 may be stored or categorized based on the predicted time for the user to read the notifications. For example, reading text message notifications and email notifications takes more time than reading advertisements. In an embodiment, the links or source content of the notifications are opened in the background, and the total reading time of the links is determined based on the content of the links. Accordingly, the notifications on the electronic device 100 are arranged based on the reading time of the users associated with the respective notifications.

[0153] In another embodiment, total reading time, new words encountered and their meanings, reading speed, content sources accessed are collected and displayed via a GUI presented on the display 106 of the electronic device 100. Furthermore, in some cases, data from the processor 102 can be shared with other users to support each other in reading goals.

[0154] The foregoing description of specific embodiments will fully reveal the general nature of the embodiments herein, so that others can easily modify and / or adapt such specific embodiments for various applications by applying prior art without departing from the general concept, and therefore, such adaptations and modifications should be understood to be within the meaning and scope of the equivalents of the disclosed embodiments. It should be understood that the words or terms used herein are for descriptive purposes rather than for limiting purposes. Therefore, although the embodiments herein have been described according to preferred embodiments, those skilled in the art will recognize that the embodiments herein can be practiced by modification within the scope of the embodiments described herein.

Claims

1. A method for managing operations on data presented on a display of an electronic device, the method comprising: detecting data presented on a display of the electronic device; determining reading parameters associated with the data based on at least one machine learning (ML) regression model; performing a plurality of operations based on the reading parameters; wherein the reading parameter comprises at least one of a full screen reading time taken by a user to fully read data presented on a display of the electronic device, a partial screen reading time taken by a user to read at least a portion of the data, a scrolling distance of each scrolling action on the display, or a data reading time, wherein the data reading time is the time taken to fully read the data presented on the display, and The training of the at least one ML regression model comprises: Dividing the display screen of the electronic device into a plurality of blocks; extracting features of the data for each of the plurality of blocks; and The at least one ML regression model is trained based on the extracted features.

2. The method according to claim 1, further comprising: determining the context of said data, Wherein, the determination of the reading parameters is performed based on the data and the context of the data being processed by the at least one ML regression model.

3. The method of claim 2, wherein determining the context of the data comprises: The context of the data is determined based on at least one of the content of the data, the type of content of the data, the user's familiarity with the content, the state of the electronic device, the user's scrolling pattern, the user's reading ability, the time when the data is detected, or the location of the electronic device.

4. The method according to claim 1, wherein: Training the at least one machine learning (ML) regression model based on the extracted features comprises: generating a first training data set comprising actual time taken to fully read the data presented on the display, the extracted features of each of the plurality of blocks, and a context of the data; generating a second training data set, the second training data set comprising an actual scroll distance, the extracted features of each of the plurality of blocks, and a context of the data; training a first ML regression model of the at least one ML regression model using the first training data set to determine the complete screen reading time; and A second ML regression model of the at least one ML regression model is trained using the second training data set to determine a scrolling distance of each scrolling action.

5. The method according to claim 4, wherein: Determining a reading parameter associated with the data based on at least one ML regression model includes: determining the complete screen reading time based on a trained first ML regression model; and One of the multiple operations is adjusting a screen timeout period of data presented on a display of the electronic device.

6. The method according to claim 4, wherein: Determining the reading parameters associated with the data based on at least one ML regression model includes: determining the scrolling distance of each scrolling action based on a trained second ML regression model; and One of the plurality of operations is to perform scrolling of the display based on the determined scrolling distance of each scrolling action.

7. The method according to claim 4, further comprising: determining a visual focus distribution based on the plurality of blocks; assigning an actual complete screen reading time to each of the plurality of blocks, the actual complete screen reading time indicating a time taken by a user to completely read data on the display; An actual data item reading time for each of the plurality of blocks is determined based on the allocated actual complete screen reading time for each of the plurality of blocks.

8. The method of claim 7, wherein the determining of the visual focus distribution comprises: The visual focus distribution is determined based on weight patterns of multiple block-level layers of a trained machine learning regression model.

9. The method according to claim 1, further comprising: Determine actual full screen reading time during a predetermined period; as well as comparing the complete screen reading time with the actual complete screen reading time; One of the plurality of operations is providing an advertisement based on a result of the comparing.

10. The method of claim 1, wherein one of the plurality of operations is displaying a graphical indicator indicating a progress of reading data presented on a display of the electronic device.

11. The method of claim 1, wherein one of the plurality of operations is displaying a graphical indicator indicating a portion of the data that will be scrolled out within a predetermined time.

12. The method according to claim 1, further comprising: receiving data as a text message from another electronic device; as well as A data reading time for the data as a text message is transmitted to the other electronic device, and the data reading time is displayed on a display of the other electronic device.

13. The method according to claim 1, further comprising: Evenly dividing the screen of the display of the electronic device into a plurality of blocks; estimating a block reading time for each of the plurality of blocks based on the reading times of the plurality of blocks collected from a plurality of users; as well as classifying each of the plurality of blocks into various importance levels based on the estimated block reading time, One of the plurality of operations is to display a different visual indicator for each of the plurality of blocks based on the importance level.

14. The method of claim 1, wherein one of the plurality of operations displays identifications of other users adjacent to or corresponding to an edge of a portion of the data based on at least one of the reading parameters.

15. An electronic device for managing operations on data presented on a display of an electronic device, the electronic device comprising: said display for displaying data; as well as The processor is configured as: detecting data presented on the display, determining reading parameters associated with the data based on at least one machine learning (ML) regression model, and performing a plurality of operations based on the reading parameters; wherein the reading parameter comprises at least one of a complete screen reading time taken by a user to completely read data presented on a display of the electronic device, a partial screen reading time taken by a user to read at least a portion of the data, a scrolling distance of each scrolling action on the display, or a data reading time, wherein the data reading time is the time taken to completely read the data presented on the display, The training of the at least one machine learning ML regression model includes: Dividing the display screen of the electronic device into a plurality of blocks; extracting features of the data for each of the plurality of blocks; and The at least one machine learning (ML) regression model is trained based on the extracted features.

Images