Electronic style reading position notification device

Abstract

To provide a reading position notification device of electronic style which collates sentences being read aloud to display reading positions for a user, and can avoid inconvenience that the reading positions become unknown.SOLUTION: A reading position notification device of electronic style comprises: display means which displays electronic data of sentences; voice input means; voice recognition means; reading position determination means which converts the sentence read aloud into characters, and collates it with sentence data to determine a reading position; reading position display means which displays the position on the display means in association with the sentence; and the like.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an electronic style reading position notification device that recovers the forgotten reading position of an electronic style by processing audio reading aloud, when reading an electronic style on an electronic display screen, and when the reading position of the electronic style is often lost from the user's consciousness due to unintentional forgetting or forgetting due to falling asleep for a moment, etc. This is particularly likely to occur when reading silently. [Background technology]

[0002] Looking back at prior art documents, the applicant's work can be found in Patent Documents 1 and 2. Patent Document 1 involves extracting the viewpoint by observing the face with a camera and displaying the viewpoint position on an electronic display screen. Patent Document 2 involves using a camera to photograph a light spot displayed on an electronic display screen by a light spot providing means mounted on the head, and using the camera to obtain data on the reading position. Both of these systems require at least a photographing means. Patent Documents 3 and 4 do not restore the forgotten reading position of the electronic writing style. In Patent Document 3, a gaze detection unit determines the order of parts referenced (seen) by the user, and when it matches a predetermined part order (e.g., parts proceeding in the order of P, Q, R), it issues a control such as issuing instructions necessary for the next action (e.g., displaying an arrow pointing to S). This has the disadvantage of requiring a complicated configuration, such as a gaze detection unit for a camera or the like, and a parts display position database. Patent Document 4 describes a conference relay device that identifies an area (not the text information itself) in the conference materials indicated by text information converted from the voices spoken during the conference, and highlights that area. Specifically, the contents of the text information and the intended meaning of the sentence are identified, for example, When a voice utters a phrase such as "the second line" or "red text," the sentence "second line" is highlighted on the display screen, and the portion written in "red" text is also emphasized. In this application, the written sentence itself is described as, for example, "the second line," but the position of the writing may be at the end of the sentence and not the "second line." Similarly, in this application, the written sentence itself is merely described as "red text," and the sentence itself is normally written in black text, and the "area indicated by the text information" does not refer to the text itself. In this application, the position of the electronic text being read is one location immediately before, but in Patent Document 4, it is not necessarily one location. When viewing text, drawings, charts, or other displays on a display screen, it is common to lose track of where you were looking when you realized it, resulting in wasted time and effort in rereading the text. This often occurs when viewing difficult content or when fatigued. Even when not asleep, it is often the case that people lose track of where they were looking. Therefore, a simple device is needed to recover the reading position of an electronic text style that has been forgotten due to sudden sleepiness, etc., when reading an electronic text style on an electronic display screen. This is in response to the reality that people often lose track of where they were looking, regardless of whether they are asleep or not. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent application 2021-097677 [Patent Document 2] Patent application 2021-101671 [Patent Document 3] Patent Publication No. 2006-107048 [Patent Document 4] Patent Publication No. 2011-65467 Summary of the Invention [Problem to be solved by the invention]

[0004] The objective of this application is to provide a device with a simple configuration that, when reading an electronic style on an electronic display screen, can often cause the user to forget the reading position of the electronic style due to unintentional forgetting or forgetting due to falling asleep, etc., and that can recover the forgotten reading position of the electronic style and notify the user by using and processing the voice of the reading aloud. [Means for solving the problem]

[0005] The following description is based on the claims. The invention described in claim 1 is an electronic style reading position notification device, A control means and a device controlled by the control means, a sentence data storage means for storing sentence data and layout data of the sentence data; a display means for displaying the stored sentence data as a sentence on an electronic screen; a voice input means for inputting a voice of a user reading the displayed sentence; a voice recognition means for recognizing the input voice and converting it into character string data; and a character string temporary storage means for temporarily storing the converted character string data. The system is provided with a reading position determination means for comparing the character string data with the sentence data in the sentence data storage means and determining a single reading position from the layout data, a matching result storage means for storing the matching results, a reading position display means for displaying the single reading position in association with the sentence in the display means, and a matching condition designation notification means for designating a matching condition, so that when the reading position becomes unknown, the user can easily find the reading position and resume reading, The matching condition designation notification means is capable of designating whether to issue an alarm or not when the user does not read aloud for a predetermined period of time, when the user reads aloud intermittently, or when a matching reading position is not found even after the matching, and / or designating the condition for issuing a notification to prompt the user to read aloud at appropriate time intervals. of It is characterized by: [Effects of the Invention]

[0006] As configured as described above, the electronic style reading position notification device of the present invention converts the sentence being read aloud into characters, compares the characters, determines a single reading position, and displays it to the user, thereby avoiding the inconvenience of the reading position being unknown. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a diagram showing an embodiment of the configuration of an electronic style reading position notification device according to the present invention; [Figure 2] 1 is a diagram showing an embodiment of the operation flow of an electronic style reading position notification device according to the present invention; [Figure 3] 1 is a diagram showing an embodiment of designating a matching condition for an electronic style reading position notification device according to the present invention; [Figure 4] 10 is a diagram showing an embodiment of a function for single determination of a reading position of an electronic style reading position notification device according to the present invention; FIG. [Figure 5] FIG. 10 is a diagram showing one embodiment of the collation function of the electronic style reading position notification device according to the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0008] FIG. 1 is a diagram showing an embodiment of the configuration of an electronic style reading position notification device according to the present invention. a sentence data storage means 110 for storing data on sentences and their arrangement, a display means 120 for displaying the stored sentence data as sentences on an electronic screen, a voice input means 130 for inputting the voice of a user reading the displayed sentence, a voice recognition means 140 for recognizing the input voice and converting it into character string data, and a character string temporary storage means 150 for temporarily storing the converted character string data. The system is provided with a reading position determination means 160 that compares the character string data with the sentence data in the sentence data storage means 110 and determines a single reading position from the placement data, a matching result storage means 170 that stores the matching results, a reading position display means 180 that displays the single matching result as a reading position in association with the sentence in the display means, a matching condition designation notification means 190 that designates matching conditions or notifies the user, and a control means 200 that controls these. When the user is reading a sentence on the display means 120, if the user is reading aloud, this device uniquely identifies the position of the sentence being read and displays that position on the display means 120. Therefore, even if the user loses track of where they were reading, they can easily find the reading position and return to where they were. This often happens when the user gets sleepy for a moment, or when they stop reading midway, and it is inconvenient to have to reread and find the reading position each time, but this device avoids this inconvenience. The sentence and layout data in the sentence data storage means 110 is, for example, HTML-described data for the web, and is data in a description format appropriate for the application. There are many examples of the display means 120, including LCD displays and EL displays. The voice input means 130 is a so-called microphone. The voice recognition means 140 is a voice-to-text conversion technology that has become commonplace with the advent of deep learning and the development of artificial intelligence, to the point where it is installed in smartphones, and voice recognition programs are also commercially available and can be used. The reading position determination means 160 and the matching condition designation notification means 190 will be described in detail below. The control means 200 is a computer that controls the procedures for performing the operations of the above means. The storage means is a so-called memory. It should be noted that devices that display sentences on the display means 120 generally have a means for specifying display conditions that change the display of sentences on the display means 120 in accordance with the user's preferences.

[0009] The operations of the reading position determination means 160 and the matching condition designation notification means 190 will be described with reference to Figures 2 to 5. The reading position determination means 160 collates the character-converted part of the read aloud sentence with the sentence data in the sentence data storage means 110 to determine the position of the sentence that has been read, and the matching condition designation notification means 190 specifies various matching conditions in this case or serves as an interface for notifying the user, and details will be provided below.

[0010] 2 is a diagram showing an embodiment of the operation flow of the electronic style reading position notification device according to the present invention. A) The device is started. B) The matching conditions shown in 1) and 2) of FIG. 3 are specified. C) Select the operation status shown in 3) of Figure 3. ·Select Match to start matching. ·You can also hide the display under Display. ·Select End to end the process. ·Select Go to Matching Condition Specification to return to the matching condition specification in B) and change the conditions.

[0011] 3 is a diagram showing one embodiment of the designation of the matching condition of the electronic style reading position notification device according to the present invention. 1) Choose whether to use the same conditions as last time or this time. If you select the same conditions as last time, matching will begin immediately. If you select this time, you can change the parts you want to change. If this is your first time, you can specify all conditions or select the predetermined standard conditions. Examples of specified conditions are: (1) The matching time interval can be set to either the default interval or a (specified) appropriate time interval. (2) When displaying the collation results, you can specify options such as: always, at appropriate time intervals, on demand, or no display. (3) When matching characters in an inserted image, you can specify whether to match or not match. (4) As an alarm condition, you can specify whether or not to issue an alarm when there is no reading aloud for a specified time, such as when the voice is interrupted and reading continues, when the reading aloud is disjointed, or when no matching reading position is found even after verification.You can also specify a specified time. In this way, an alarm can be issued in case of dozing off, etc. It is desirable that instead of an alarm, a notification such as "Please pronounce" can be specified as a condition to prompt the user to read aloud at appropriate intervals. In such a case, the problem can be solved by restarting the matching process after the alarm. Although it may be possible to specify whether to save or not save the state of the reading position etc. at the time of termination, it is usually sufficient to save it at the time of termination. If you select the end of the specified position, you will move to the 2) start position specification field.

[0012] 2) Specifying the start (including restart) position: You can specify it using one of the following. (1) From the xth line of the text currently displayed or from the cursor selection position (2) From the position where the previous scan ended / If you specify this, the previous scan page will automatically open and display the position. (3) From page y, line z of the electronic style currently displayed (4) Specify details from the table of contents of the electronic writing style currently displayed Table of contents → Start of section → Page forwarding → Line selection If you select "End of specification", you will move to the specification field for 3) selecting / inputting the operating status.

[0013] 3) Operation state selection / input: Can be specified as one of the following. (1) Start of verification / Verification in progress: Enter the input to start verification and display the verification in progress. (2) Specify matching conditions: Switch to specifying matching conditions. (3) Matching Suspension / Resume, End: When a user leaves his / her seat, in order to avoid the influence of other voices, the user can input the commands to suspend, resume, or end the match. · Save · No save: Specify to save the status of the reading position, etc., and notify the current status

[0014] Incidentally, 1) whether the same as the previous specification or the current specification and 2) the specification of the start position may be displayed on the entire display means 120 since collation has not yet started. Once collation has started, they may not be displayed. 3) The operation status selection may be performed while verification has already begun. In this case, it is preferable to display the status at all times, minimally, on the edge of the display screen so as not to interfere with the display of the sentence, and to make it possible to terminate the operation midway or change the conditions. In addition, the display unit for operation status selection may display a selection key called "display request," so that verification is performed, but the display is only displayed when requested. Even when the display is off, the display can be restored by a user's request at any time using the "request display" selection key. In particular, this can be done by pressing a switch or by speaking a keyword that is unlikely to appear in a sentence. Furthermore, there may be a lamp or other indicator that indicates that verification is in progress.

[0015] It is preferable that the following specifications be possible when specifying the matching conditions. b) Specify the range of the reading position display width I-1, 1 phrase specification Example: In the sentence "I went to the doctor." It is possible to specify each segment, such as [I], [I went to the doctor], and [I went to the doctor]. I-2, Multiple clause specification (punctuation clause specification) For example, in the sentence, "I had a fever since yesterday, so I went to the doctor. I got an injection, and my fever went down." It is possible to specify multiple segments, such as [I had a fever since yesterday], [I went to the doctor], and [My fever went down after I got an injection]. b) Setting the reading position display type Ro-1, Highlight display: Shown in bright colors as if marked with a highlighter pen Ro-2, White background... The background of the text becomes black and the text becomes white Ro-3, Underline: A line is drawn under the character Ro-4, Enclosure... Enclose the letters in a square c) How to move the reading position display Ha-1, current position...move by phrase (or multiple phrases) Ha-2, Add without moving... Once the position marking is displayed, it will not disappear and will be added. (It will disappear if you select Clear or on a new page.)

[0016] FIG. 4 is a diagram showing an embodiment of the function of single determination of the reading position of the electronic style reading position notification device according to the present invention. Suppose a user is reading aloud a sentence on an electronic screen. In the example in 4-A, when the letters "PQR" are read aloud, there are three corresponding reading positions, and it is not clear which position the user was reading. 4-B and 4-C are examples of a function that determines a single pronunciation. In 4-B, the function waits for the next pronunciation and adds that part to the comparison, resulting in a single pronunciation. In this case, it is best to extend the sentences pronounced before or after, as in this example, and repeat the comparison until a single pronunciation is determined. The longer the sentence is compared, the better the comparison performance, and as a result, a single pronunciation can be determined with certainty. In 4-C, when the user can determine which is the reading position from their memory of the sentence, the position is determined as a single pronunciation by selecting with a mouse or by using a specific keyword (e.g., select select 2 (meaning select the second) or by voice instruction). In 4-B, as a means of change, it is also possible to determine the same character that follows the previous reading position as a single reading position. This is possible because people usually read by moving in a certain direction. However, it is common for people to go back and reread a character even earlier than the previous one because they do not understand the meaning, so this approach can sometimes be inconvenient, but this inconvenience is a matter of frequency and can sometimes be tolerated. Next, Figure 5 shows an example of an unusual case.

[0017] 5 is a diagram showing one embodiment of the matching function of the electronic style reading position notification device according to the present invention. a) This is a normal case of matching, in which the next matching candidate is read aloud along the direction of the sentence after the current reading position. However, as shown in b) the non-normal case of matching, b-1) Sometimes you go back to a part you have already read and read it, or b-2) sometimes you jump to an unread part and read it. In this case, there may be cases where a match is not made even when the next normal matching candidate, the previous (or next) match, or even the previous (or next) match data is entered and compared. For example, the entire sentence may not be read aloud continuously, but only in bits and pieces. In such cases, a message such as "Please read aloud as a long sentence including the context" can be displayed to prompt the user to match the long sentence including the context. If this still does not work, the system will display "No match with next matching candidate, etc." and prompt the user to specify matching conditions, ·Page □□□, line △△·End of specification ·Line XX before page XX ·End of specification ·Current line before (after) ** lines·End of specification It is preferable to have the user specify the following.

[0018] Furthermore, if you return to a section that you have already read, you can immediately find it because it has been previously matched. However, if you jump to an unread section, you can specify conditions to assist in matching. This response can also be considered when the current page suddenly jumps to another page that is different from the usual page, but when reading aloud, since it is acceptable to assume that the display has changed to a different page, it is possible to start from the beginning and check the page that has changed, or if it does not match the reading position, it can instead prompt the user to "specify the reading position." This situation also applies when reading while moving back and forth between multiple sentences at the same time: when you jump from one sentence to another, the reading position in that sentence is displayed and saved, and when you move to another sentence (from sentence to sentence), the process described in 2) Specifying the start position in Figure 3 is carried out. In particular, a convenient process in this case is the ability to instantly specify the start position by specifying the Xth line or the cursor.

[0019] Since reading aloud is a necessary requirement, it is also preferable to perform partial cancellation of the match, which cancels the match result only at that point, in case the user makes or has made a sound unrelated to the sentence to be read, or in case a sound different from the user's is input. Also, the user may be warned with a message such as "Another sound has been input. Please do not input any more." Although this would be a heavy function, it would also be possible to use voiceprint detection to distinguish and remove voices of other people or to issue a warning. The intensity of the audio used for reading aloud can be normal sounds of around 60 dB, as well as soft sounds of 10 dB to 30 dB and extremely soft sounds of less than 10 dB. For reference, the intensity of breathing sounds is around 10 dB to 20 dB. The handling of soft and extremely soft sounds can be found in JP 2014-063018 and JP 2015-228002. [Industrial Applicability]

[0020] As described above, the electronic style reading position notification device of the present invention checks the sentence being read aloud to determine the reading position and displays it to the user, thereby avoiding the inconvenience of the reading position being unknown and making it extremely convenient for industrial use.

[0021] 110 Sentence data storage means 120 Display means 130 Voice input means 140 Voice Recognition Means 150 String temporary storage means 160 Reading position determining means 170 Matching result storage means 180 Reading position display means 190 Matching condition designation notification means 200 Control Means

Claims

[Claim 1] A control means and a device controlled by the control means, a sentence data storage means for storing sentence data and layout data of the sentence data; a display means for displaying the stored sentence data as a sentence on an electronic screen; a voice input means for inputting a voice of a user reading the displayed sentence; a voice recognition means for recognizing the input voice and converting it into character string data; and a character string temporary storage means for temporarily storing the converted character string data. The system is provided with a reading position determination means for comparing the character string data with the sentence data in the sentence data storage means and determining a single reading position from the layout data, a matching result storage means for storing the matching results, a reading position display means for displaying the single reading position in association with the sentence in the display means, and a matching condition designation notification means for designating a matching condition, so that when the reading position becomes unknown, the user can easily find the reading position and resume reading, The matching condition designation notification means is capable of specifying whether or not to issue an alarm if the user does not read aloud for a predetermined period of time, if the user reads aloud in a staggered manner, or if a matching reading position is not found even after the matching, and / or is capable of specifying conditions for issuing a notification to encourage reading aloud at appropriate time intervals.

Images