Character input device, character input system, character input method, and character input program

Abstract

To improve the operability of character input by line of sight.SOLUTION: A character input device includes: a line-of-sight detection unit configured to switch between a first screen including a plurality of areas related to consonants and a second screen including a plurality of areas related to vowels and cause a display device to display it, acquire information indicating a gaze point from a line-of-sight detection device that detects the line of sight of a user facing the display device and outputs the gaze point of the user, and determine that, when the gaze point indicated by the acquired information is included in a predetermined area for a predetermined gaze time or longer, the area has been selected; and a character input unit configured to accept input of characters corresponding to a combination of an area selected on the first screen and an area selected on the second screen based on the determination by the line-of-sight detection unit. The plurality of areas included in the first screen and the plurality of areas included in the second screen are displayed separated from other areas included in the same screen by a predetermined distance or more.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a character input device, a character input system, a character input method, and a character input program. [Background technology]

[0002] Conventionally, a gaze input device has been proposed that allows efficient character input by displaying characters of sufficient size (for example, Patent Document 1). In this technology, a portion of a group of multiple types of characters is displayed in a character area. Then, a character in the character area is selected and input using a gaze cursor that indicates the position of the observer's gaze. When a character that is not displayed in the character area is to be input, the gaze cursor is moved to the position of a character that is partially displayed on the edge of the character area. When the gaze cursor is moved to this position, the group of characters in the character area is scrolled and a new character is displayed in the character area. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2000-20196 A [Patent Document 2] JP 2014-158144 A [Patent Document 3] JP 2002-215288 A [Patent Document 4] Special Publication No. 2009-545802 [Patent Document 5] JP 2011-59998 A [Patent Document 6] Special Publication No. 2011-529597 [Patent Document 7] JP 2020-5114 A Summary of the Invention [Problem to be solved by the invention]

[0004] When inputting characters by eye gaze, the distance the eyes move on the screen tends to be large, which causes eye fatigue. Also, when many characters are displayed on the screen, input errors increase. On the other hand, when scrolling through character options on the screen, it takes time to wait until the desired option appears, which reduces input efficiency.

[0005] An object of the present invention is to improve the operability of character input by eye gaze. [Means for solving the problem]

[0006] The character input device includes a gaze detection unit that switches between a first screen including a plurality of regions related to consonants and a second screen including a plurality of regions related to vowels and displays them on a display device, detects the gaze of a user facing the display device, acquires information indicating the gaze point from a gaze detection device that outputs the gaze point of the user, and determines that a predetermined area has been selected when the gaze point indicated by the acquired information is included in the predetermined area for a predetermined gaze time or more, and a character input unit that accepts input of a character according to a combination of the area selected on the first screen and the area selected on the second screen based on the judgment of the gaze detection unit. Also, the plurality of areas included in the first screen and the plurality of areas included in the second screen are displayed at a distance equal to or greater than a predetermined distance from other areas included in the same screen.

[0007] When inputting characters by the user's line of sight, there tends to be a large number of gaze areas as options, but by inputting options related to consonants and options related to vowels on two separate screens, it is possible to prevent erroneous inputs such as inputting unintended characters. In addition, each of the multiple areas as options is located at a predetermined distance from other areas included in the same screen. By displaying the characters at a greater distance than the predetermined distance, input errors can also be prevented.

[0008] The gaze detection unit may provide a waiting time during which it does not determine whether the gaze point is included in a predetermined area for a predetermined gaze time or longer after switching the display of the display device between the first screen and the second screen. The waiting time can prevent erroneous input based on the user's gaze immediately after the screen is switched.

[0009] Furthermore, the multiple regions included in the first screen and the multiple regions included in the second screen may be arranged on a circular arc, so that the distance the line of sight moves to each region is approximately the same, reducing user fatigue.

[0010] The first screen may further include a plurality of areas for the user to express a predetermined intention, the plurality of areas relating to consonants being arranged on an outer arc of the double circle, and the plurality of areas for expressing a predetermined intention being arranged on an inner arc of the double circle. Being able to express, for example, affirmation or negation in parallel while inputting characters is useful when a question is suddenly asked, for example.

[0011] The gaze detection unit may also be configured to determine that a region has been selected by setting a longer gaze time for a plurality of regions for expressing a predetermined intention than for a plurality of regions corresponding to rows on the first screen. The gaze often passes through the inner regions, which are used relatively infrequently, so by setting a difference in gaze time, it is possible to prevent the occurrence of erroneous input while preventing a decrease in operability.

[0012] The first screen may further include a temporary text box for temporarily displaying the character string that has been input, and a decision area for outputting the character strings displayed in the temporary text box to another predetermined text box or outputting them as sound, and when the gaze detection unit determines that the decision area has been selected, the character input unit may output the character strings displayed in the temporary text box to another predetermined text box or output them as sound, and when the gaze detection unit determines that the area for expressing a predetermined intention has been selected, the character input unit may output according to the selected area for expressing a predetermined intention before outputting the character string displayed in the temporary text box. For example, this allows a user to express an intention, for example, affirmative or negative, in parallel while inputting characters, which is useful when a user is suddenly asked a question.

[0013] The first screen may further include a rest button area for stopping the process by the gaze detection unit when the gaze detection unit determines that the area has been selected, and when the gaze detection unit determines that the rest button area has been selected, it may stop accepting operations other than the operation to cancel the stop. This can reduce user fatigue caused by continuous operations.

[0014] Also, each of the multiple areas included in the second screen may be displayed at a position on the display device that does not overlap with the position where the selected area is displayed on the first screen. In this way, it is possible to prevent erroneous input based on the user's line of sight immediately after the screen is switched.

[0015] The first screen may include a change area for changing the gaze time or the waiting time, and the gaze detection unit may increase or decrease the gaze time or the waiting time when it is determined that the change area is selected. Since the gaze time or the waiting time can be changed depending on the user's proficiency, operability can be improved.

[0016] The predetermined distance is the expected distance from the user's eyeball to the gaze point on the display device. The distance may be l, and the angle between the line connecting the eyeball and the point of gaze and the line connecting the eyeball and the edge of the centrally visible range on the display device may be θ, where r is the radius of the centrally visible range calculated by the following formula. r=ltanθ Note that θ may be 1 to 2 degrees. Specifically, by adopting such a magnitude, it is generally possible to clearly recognize an object by central vision, and to suppress unintended control by the user due to microsaccades occurring during fixation.

[0017] The contents of the means for solving the above problems can be combined as much as possible without departing from the scope of the problems and technical ideas of the present invention. The above means may be realized as a method executed by a computer, or as a program for causing a computer to execute the above means. The program may be provided by recording it on a computer-readable recording medium. Effect of the Invention

[0018] According to the present invention, it is possible to improve the operability of character input by eye gaze. [Brief description of the drawings]

[0019] [Figure 1] FIG. 1 is a configuration diagram showing an example of a character input system according to the present embodiment. [Diagram 2] FIG. 2 is a functional block diagram showing an example of a character input system. [Diagram 3] FIG. 3 is a diagram showing an example of the first screen. [Figure 4] FIG. 4 is a diagram showing an example of the second screen. [Diagram 5] FIG. 5 is a diagram for explaining the position and size of the operation area set on the display device. [Figure 6] FIG. 6 is a diagram showing the relationship between the first screen shown in FIG. 3 and the above-mentioned radius r. [Figure 7] FIG. 7 is a diagram showing the relationship between the second screen shown in FIG. 4 and the above-mentioned radius r. [Figure 8] FIG. 8 is a process flow diagram illustrating an example of a process performed by the display device. [Figure 9] FIG. 9 is a process flow diagram illustrating an example of a process performed by the gaze detection device. [Figure 10] FIG. 10 is a process flow diagram illustrating an example of a process performed by the control device. [Figure 11] FIG. 11 is a process flow diagram illustrating an example of a process performed by the control device. [Figure 12] FIG. 12 is a process flow diagram illustrating an example of a process performed by the control device. [Figure 13] FIG. 13 is a process flow diagram illustrating an example of a process performed by the control device. [Figure 14] FIG. 14 is a diagram showing an example of the second screen according to the modified example. [Figure 15] FIG. 15 is a diagram showing an example of the second screen according to another modified example. [Figure 16] FIG. 16 is a diagram showing an example of the first screen according to another modified example. [Figure 17] FIG. 17 is a diagram showing an example of the second screen according to another modified example. [Figure 18] FIG. 18 is a diagram showing an example of the second screen according to another modified example. [Figure 19] FIG. 19 is a functional block diagram showing an example of a character input device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] Hereinafter, a character input system according to an embodiment of the present invention will be described with reference to the drawings.

[0021] <Device configuration> FIG. 1 is a configuration diagram showing an example of a character input system according to this embodiment. The character input system 1 shown in FIG. 1 includes a control device 2, a display device 3, and a gaze detection device 4. The character input system 1 is a system for performing character input based on the gaze of a user. The gaze detection device 4 includes, for example, a camera that captures infrared light, and detects the gaze based on the movement of the user's eyes. The control device 2 causes the display device 3 to display an interface for character input, and also indicates a gaze point, which is the part on the display device 3 where the user is gazing (or fixating). The control device 2 acquires information from the gaze detection device 4 and accepts input of characters selected according to the position of the gaze point. The input characters may be output as voice from a speaker, for example, and used for conversation with other users. The input characters may also be output to a text box of other software for performing text chat, for example. The control device 2 corresponds to the "character input device" in this invention.

[0022] <Functional configuration> 2 is a functional block diagram showing an example of a character input system 1. The character input system 1 in FIG.

[0023] The control device 2 is a general computer, for example, a PC (Personal Computer), The control device 2 is a smartphone, a tablet, etc. The control device 2 also includes a communication interface (I / F) 21, a storage device 22, an input / output device 23, and a processor 24, which are connected via a bus 25. The communication I / F 21 is a digital camera for performing communication using Wi-Fi (registered trademark), Bluetooth (registered trademark), or other RF (Radio Frequency) signals. The storage device 22 may be a wireless module for inputting and outputting information, a wired network card, a parallel port, a serial port, etc., and communicates with other devices based on a predetermined protocol. The storage device 22 is a main storage device such as a random access memory (RAM) or a read only memory (ROM), and an auxiliary storage device (secondary storage device) such as a hard disk drive (HDD), a solid state drive (SSD), or a flash memory. The main storage device temporarily stores programs read by the processor 24 and secures a working area for the processor 24. The auxiliary storage device stores programs to be executed by the processor 24, etc. The input / output device 23 is a user interface such as a keyboard, a mouse, or a touch panel. The input / output device 23 may also function as the display device 3. The processor 24 is an operating device such as a central processing unit (CPU), The processor 24 is a computational processing device, and executes a program to perform each process described in the present embodiment. In the example of FIG. 2, functional blocks are shown in the processor 24. Specifically, the processor 24 includes an I / F output unit 241, a gaze detection unit 242, a character input unit 243, and a setting change unit 244. The I / F output unit 241 switches between a first screen including a plurality of areas related to consonants and a second screen including a plurality of areas related to vowels, and displays them on the display device. The gaze detection unit 242 acquires information on the movement of the user's gaze from the gaze detection device 4 via the communication I / F 21, and detects the selection of a character by the user according to the position of the gaze point. The character input unit 243 accepts the input of the character selected by the user according to the position of the gaze point. The setting change unit 244 accepts a change in the setting regarding the operation of the present system according to the position of the gaze point.

[0024] The display device 3 is a video display device that acquires and displays a video signal of an image (still image or video) output by a computer or the like. In FIG. 2, functional blocks are also shown within the display device 3. The display device 3 includes a communication I / F 31 and a display unit 32. The communication I / F 31 is an interface for receiving an input of a video signal, such as an HDMI (High-Definition Multimedia Interface, registered trademark) terminal, a DVI (Digital Visual Interface) terminal, a VGA terminal, a wireless communication module, or the like. The display unit 32 is a liquid crystal display (LCD), a plasma display (PDP), an organic EL display, a cathode ray tube (CRT), or the like, and outputs an image according to the video signal input via the communication I / F 31.

[0025] The gaze detection device 4 is a device that detects the movement of the user's gaze and identifies the gaze point. In FIG. 2, functional blocks are also shown within the gaze detection device 4. The gaze detection device 4 includes a communication I / F 41, an infrared transmission / reception unit 42, and a gaze detection unit 43. The communication I / F 41 is an interface such as a USB (Universal Serial Bus), and outputs gaze point information to the control device 2. The infrared transmission / reception unit 42 transmits infrared rays to a user facing the gaze detection device 4. The line is projected onto the display device and an image of the reflected light is captured. The gaze detection unit 43 detects the direction of the gaze of the user based on the position of the pupil of the eye from the image of the user and calculates the position of the gaze point on the display device.

[0026] <User interface> In this embodiment, character input is accepted based on a combination of areas on the first screen and the second screen that the user gazes upon. The first screen is a user interface including an area related to input of consonants, and the second screen is a user interface including an area related to input of vowels. For example, the first screen may include a plurality of areas corresponding to rows in the Japanese syllabary, and the second screen may include a plurality of areas corresponding to columns.

[0027] FIG. 3 is a diagram showing an example of the first screen. The circular area in FIG. 3 is an operation area that accepts some operation when the user gazes at it for a predetermined time. In the example of FIG. 3, a plurality of operation areas arranged on double arcs are displayed near the center of the display unit 32. On the outer arc, operation areas 101 to 112 with "a", "ka", "sa", "ta", "na", "ha", "ma", "ya", "ra", "wa / -", "〟〠 small", and "×" respectively displayed in clockwise order from the left (9 o'clock direction when viewed from the center) are arranged. On the inner semi-circular arc, operation areas 113 to 116 with an upward arrow, "Yes", "No", and "Determine" respectively displayed in clockwise order from the left are arranged. Also, on the right side portion of the display unit 32, operation areas 120 to 124 with "first", "voice", an upward triangle, a downward triangle, and "break" respectively displayed are arranged. Also, the rectangular area in FIG. 3 is a text box. Near the center of the display unit 32, a text box 117 where the input string is temporarily displayed, a first prediction conversion column 118 where the prediction conversion result of the string being input is displayed, and a second prediction conversion column 119 are arranged. Also, on the left side portion of the display unit 32, a text box 125 where the input string is displayed is arranged.

[0028] The operation areas 101 to 110 with "a", "ka", "sa", "ta", "na", "ha", "ma", "ya", "ra", and "wa / -" displayed are areas related to the input of consonants. For example, "a" is an area for selecting the a row (without consonant input), "ka" is an area for selecting the ka row ( / k / ), and "sa" is an area for selecting the sa row ( / s / ). The areas related to the input of consonants are used in combination with the selection of the areas related to the input of vowels on the second screen described later to input characters. Also, the operation area 111 with "〟〠 small" displayed is used to attach a dakuten or handakuten to the character input immediately before, or to change it to a kana represented by a small character (sokushinji). The operation area 112 with "×" displayed is used to cancel the character input immediately before, etc.

[0029] FIG. 4 is a diagram showing an example of the second screen. The circular area in FIG. 4 is also an operation area that accepts some operation by the user gazing at it for a predetermined period of time. In the example of FIG. 4, a plurality of operation areas arranged on an arc are displayed near the center of the display unit 32. On the arc, operation areas 131 to 136 are arranged, in which "a", "i", "u", "e", "o" and "x" are displayed clockwise from the upper left (the 10 o'clock direction when viewed from the center). The rectangular area in FIG. 4 is also a text box. The text boxes 117 and 125 are the same as the corresponding components in FIG. 3. Note that after the first screen and the second screen are switched, a waiting time is provided during which no input operation by line of sight is accepted, to suppress erroneous input (input against the user's intention).

[0030] The operation areas 131 to 135 in which "a", "i", "u", "e" and "o" are displayed are areas related to inputting vowels. For example, "a" is an area for selecting the a row ( / a / ), "i" is an area for selecting the i row ( / i / ), and "u" is an area for selecting the u row ( / u / ). The areas related to inputting vowels are located in the areas related to inputting consonants on the first screen. It is used to input characters in combination with selection. Therefore, for example, when the ka row is selected on the first screen, "ka", "ki", "ku", "ke", and "ko" are displayed in the operation areas 131 to 135, and when the sa row is selected on the first screen, "sa", "shi", "su", "se", and "so" are displayed in the operation areas 131 to 135. When the ya row is selected on the first screen, "ya", "yu", "yo", "," (comma), and "." (full stop) may be displayed in the operation areas 131 to 135. When the wa row is selected on the first screen, "wa", "wo", "n", "ー" (macro note), and "_" (underscore) may be displayed in the operation areas 131 to 135. In addition, the operation area 136 on the second screen where "x" is displayed is used to cancel the operation being input and return to the first screen.

[0031] Returning to the description of FIG. 3, the inputted characters are displayed in the text box 117. In addition, according to the character string being inputted, the first predictive conversion field 118 and the second predictive conversion field 119 display the predicted result. Note that character strings with high frequency of use can be stored in advance and displayed as predicted results. The operation area 113 in which the upward arrow is displayed is an area for inputting the predicted result displayed in the first predictive conversion field 118 into the text box 117. At this time, the predicted result displayed in the second predictive conversion field 119 is displayed in the first predictive conversion field 118, and a new candidate of the predicted result is displayed in the second predictive conversion field 119. The user can select the next candidate by gazing at the operation area 113 in which the upward arrow is displayed.

[0032] The input character string is added to the text box 117 first, and is moved to the text box 125 in the order of the oldest input character. The operation area 116 in which "OK" is displayed is an area for outputting the character strings input in the text boxes 117 and 125 by voice or to a text box of another software. That is, the text boxes 117 and 125 correspond to the "temporary text box" of the present invention, and the text box of the other software corresponds to the "other specified text box" of the present invention. The operation areas 114 and 115 in which "Yes" and "No" are displayed are used to indicate affirmative and negative intentions. When "Yes" and "No" are selected, the character strings displayed in the text boxes 117 and 125 are not changed, and "Yes" and "No" are output by voice or to a text box of another software, respectively. That is, the process of outputting the affirmative or negative intention is interrupted. Such an output is useful because it can be used to answer another question while inputting characters.

[0033] The operation area 120 in which "First" is displayed is an area for changing the setting to a predetermined beginner mode. In the beginner mode, the waiting time after the first screen and the second screen are switched is set to, for example, 2 seconds. The operation area 121 in which "Speak" is displayed is an operation area for turning on / off (enabling or disabling) the voice output when the character string is output. The operation areas 122 and 123 in which an upward triangle and a downward triangle are displayed are areas for increasing and decreasing the gaze time, which is the time required to gaze at the operation area in order to determine that the operation area has been selected. The operation area 124 in which "Break" is displayed is an area for temporarily stopping the acceptance of operations by gaze in this system. When the acceptance of operations is temporarily stopped, the processing of this system is resumed when the operation area 124 is gazed at again.

[0034] According to the user interface shown in Figs. 3 and 4, the user inputs the next character by moving his / her gaze mainly to each operation area arranged on the arc while checking the character input in the central text box 117. Since each operation area is approximately equidistant from the center, the degree of fatigue does not vary depending on the character to be input. In addition, the arrangement of the operation areas is relatively easy to remember, and even if the arrangement is not memorized, the user can input the next character by moving his / her gaze mainly to each operation area arranged on the arc. The desired operation region can be found by searching clockwise.

[0035] The gaze time required to determine that the operation area has been selected on the first screen may be different between the inside and outside of the double circle. For example, the operation area on the inner arc is set to 1.0 second, and the operation area on the outer arc is set to 0.4 seconds. That is, when inputting text, the inner operation area is generally used less frequently than the outer operation area. Also, when the user checks the characters input in the central text box 117 and moves his / her gaze to the outer operation area to select the next character to input, his / her gaze passes through the inner operation area, which may cause erroneous input. By setting the gaze time longer on the inner side, such erroneous input can be suppressed.

[0036] FIG. 5 is a diagram for explaining the position and size of an operation area set on a display device. In this embodiment, the operation area is set based on a human's visual field and eye movement. FIG. 5 shows the relationship between the user's viewpoint (eyeball) and the gaze point on the display unit 32. In general, in order for a human to identify the shape of an object with high resolution, central vision is required in which a retinal image is projected at the central portion of the retina with a visual angle of about 1 to 2 degrees. In this embodiment, the size of the operation area is set to at least the range that can be captured by central vision.

[0037] Furthermore, when a human continues to capture (fixate) an object with central vision, an eye movement called microsaccade (fixational eye movement) occurs. A microsaccade is an involuntary movement of the gaze with an amplitude of about 1 to 2 degrees. Although there is some variation, the average speed is about 10 degrees / second and the frequency is about 1 to 3 Hz. When the gaze detection device 4 can detect a change in the gaze caused by a microsaccade, depending on the arrangement or size of the operation area, the user may unconsciously perform an erroneous operation due to the influence of the microsaccade. Therefore, in this embodiment, the operation area is set with an arrangement and size that will not move away from the area the user is gazing at even with a microsaccade.

[0038] The radius r of the circular operation area shown in Figures 3 and 4 can be calculated by the following formula (1), where l is the distance from the eyeball to the gaze point, and θ is the angle between the line connecting the eyeball and the gaze point and the line connecting the eyeball and the edge of the centrally visible range on the image displayed on the display unit 32. That is, r is the radius of the centrally visible range. Also, θ can be said to be the angle at which the gaze moves due to microsaccades. r = ltanθ (1) For example, if l is 550 mm and θ is 2 degrees, then r is 19.2 mm.

[0039] By setting the radius of the operation area to r, when the user gazes at the center of the operation area, even if the user is affected by microsaccades, the detected gaze point can be prevented from moving out of the operation area. FIG. 6 is a diagram showing the relationship between the first screen shown in FIG. 3 and the above-mentioned radius r. The outer circle on which the centers of the operation areas 101 to 110 for inputting consonants are arranged on the first screen has a radius of 6r. On the outer circle, operation areas with a radius of r are arranged at intervals of 30 degrees from the center. On the inner circle on which the centers of the operation areas 114, 115 for inputting intentions on the first screen are arranged has a radius of 3r. On the inner circle, operation areas with a radius of r are arranged at intervals of 60 degrees from the center. Therefore, the distance between the operation areas 113 and 114, between the operation areas 114 and 115, and between the operation areas 115 and 116 arranged on the inner circle is also r. In this way, each operation area is placed at least r apart, so that if the user is looking at the edge of an operation area, even if there is an effect of microsaccades, the detected gaze point is prevented from being determined to be within another operation area.

[0040] FIG. 7 is a diagram showing the relationship between the second screen shown in FIG. 4 and the above-mentioned radius r. In the second screen, the circle on which the centers of operation areas 131 to 135 for inputting vowels are arranged has a radius of 6r. On the circle, operation areas of radius r are arranged at intervals of 60 degrees from the center. Since each operation area is also arranged at an interval of r or more on the second screen, if the user looks at the edge of an operation area, it is possible to prevent the detected gaze point from being determined to be within another operation area even if the user is affected by microsaccades. As described above, for example, with the arrangement of operation areas shown in Figs. 6 and 7, it is possible to suppress unconscious erroneous operations (erroneous detections) caused by the influence of microsaccades.

[0041] <Character input processing> FIG. 8 is a process flow diagram showing an example of the process performed by the display device. The display device 3 receives a video signal for displaying the above-mentioned first screen and second screen from the control device 2 via the communication I / F 31 (FIG. 8: S1). Thereafter, the display unit 32 of the display device 3 displays the video of the first screen and the second screen (S2). The display unit 32 also judges whether to continue the display operation (S3). For example, when a user operates to end the operation, or when no video signal is received for a predetermined time or more, the display unit 32 judges to end the operation. If the operation is to be continued (S3: YES), the process returns to S1 and is repeated. On the other hand, if the operation is not to be continued (S3: NO), the process is ended.

[0042] FIG. 9 is a process flow diagram showing an example of a process performed by the gaze detection device. In a state where the gaze detection device 4 is installed on the display device 3, the infrared transmission / reception unit 42 transmits infrared rays to the front of the display device 3 and receives the reflected light (FIG. 9: S11). After that, the gaze detection unit 43 of the gaze detection device 4 judges whether the gaze of the user has been detected from the reflected light (S12). In this step, the gaze detection unit 43 of the gaze detection device 4 identifies the position of the pupil in the eye from, for example, the image of the user, and judges that the gaze has been detected if it has been identified. If the gaze has been detected (S12: YES), the gaze detection unit 43 calculates the gaze point based on, for example, the position of the pupil (S13). In this step, an existing gaze detection technology can be used, and the position of the gaze is calculated based on the position of the pupil relative to some reference, such as the reflected light of infrared rays from the inner corner of the eye or the cornea. The gaze point is calculated as, for example, a coordinate on the display device 3 or on the image displayed on the display device 3. Furthermore, the gaze detection unit 43 transmits information indicating the position of the gaze point (gazing point data) to the control device 2 via the communication I / F 41 (S14). After S14, or if the gaze cannot be detected in S12 (S12: NO), the gaze detection device 4 determines whether to continue the gaze detection operation (S15). In this step, it determines to end the operation, for example, when an operation to end the operation is received from the user. If the operation is to be continued (S15: YES), the process returns to S11 and is repeated. On the other hand, if the operation is not to be continued (S15: NO), the process ends.

[0043] 10 to 13 are process flow diagrams showing an example of the process performed by the control device. The I / F output unit 241 of the control device 2 causes the display device 3 to output a first screen including an area related to input of a consonant (FIG. 10: S21). In this step, an image as shown in FIG. 3 is displayed on the display device 3. The gaze detection unit 242 of the control device 2 acquires information indicating the position of the user's gaze point on the display unit 32 of the display device 3 (on the image displayed on the display unit 32) from the gaze detection device 4 (S22). The gaze detection unit 242 also determines whether a consonant or the like has been selected based on the acquired information indicating the position of the gaze point (S23). In this step, it is determined whether any of the operation areas 101 to 111 of the first screen has been selected.

[0044] If it is determined that a consonant or the like has been selected (S23: YES), the I / F output unit 241 of the control device 2 causes the display device 3 to output a second screen including an area for inputting a vowel (S24). In this step, an image such as that shown in FIG. 4 is displayed on the display device 3. After switching to the second screen, it is assumed that no input by eye gaze is accepted for a predetermined waiting time (S25). In addition, the eye gaze detection unit 242 of the control device 2 detects the user's gaze on the display unit 32 of the display device 3 and Information indicating the position of the gaze point is acquired from the gaze detection device 4 (S26). Then, the gaze detection unit 242 judges whether a vowel or the like has been selected based on the acquired information indicating the position of the gaze point (S27). In this step, it is judged whether any of the operation areas 131 to 135 of the second screen has been selected.

[0045] If it is determined that a vowel or the like has been selected (S27: YES), character input unit 243 accepts input of a character based on the combination of the consonant determined to have been selected in S23 and the vowel determined to have been selected in S27 (S28). In this step, character input unit 243 adds a character specified by the selected consonant and vowel to text box 117.

[0046] In addition, in S28, the character input unit 243 extracts the result of predictive conversion from pre-registered fixed phrases based on the character string displayed in the text box 117, and displays it in the first predictive conversion field 118 and the second predictive conversion field 119. The predicted result is extracted from the character strings stored in the storage device 22 as fixed phrases in advance that match the character string being input. In addition, for example, a fixed phrase that has been used many times in input during a recent predetermined period may be preferentially extracted. In this way, the fixed phrases with higher priority based on a predetermined criterion are displayed in the first predictive conversion field 118 and the second predictive conversion field 119. On the other hand, if a vowel is not selected in S27 (S27: NO), such as when the area 136 of the second screen is selected, or after S28, the process proceeds to the process of FIG. 11. Then, the I / F output unit 241 outputs the first screen including the area related to the input of consonants to the display device 3 (FIG. 11: S29). In this step, an image as shown in FIG. 3 is displayed on the display device 3. After switching to the first screen, the system does not accept any eye-gaze input for a predetermined waiting time (S30), and then the system returns to S22 and repeats the process.

[0047] Also, if it is determined in S23 of FIG. 10 that a character other than a consonant has been selected, the process proceeds to the process of FIG. 12. The gaze detection unit 242 determines whether a change to the most recently input character has been instructed based on the acquired information indicating the position of the gaze point (FIG. 12: S31). In this step, it is determined whether the user's gaze point is located in the operation area 111 with a voiced consonant, a semi-voiced consonant, or a small character (separated kana) for a predetermined time or more. If the operation area 111 is selected (S31: YES), if the most recently input character displayed in the text box 117 is a character to be changed, a voiced consonant or a semi-voiced consonant is added, or the character is changed to a small character (S32). In this step, for example, if the most recently input character is a character in the ka row, sa row, ta row, or ha row, the character input unit 243 changes it to a voiced consonant. If the most recently input character is a character in the a row or ya row, the character is changed to a small character. Also, if the immediately preceding character is a character in the ba row or "zu", it is changed to a semi-voiced sound or "っ" (short consonant), respectively. Note that if the immediately preceding character is a character in the ga row, za row, a character in the da row or pa row other than "zu", or "っ", it may be changed back to a clear sound character. Then, the process returns to S21 in FIG. 10.

[0048] Also, if it is determined in S31 that a change in character has not been instructed (S31: NO), the gaze detection unit 242 determines whether input of a fixed phrase (i.e., a result of predictive conversion) has been selected based on the acquired information indicating the position of the gaze point (S33). In this step, it is determined whether the user's gaze point is located in the operation area 113 with an upward arrow for a predetermined period of time or more. If the operation area 113 is selected (S33: YES), input of a fixed phrase is accepted (S34). In this step, the character input unit 243 inputs the prediction result displayed in the first predictive conversion field 118 into the text box 117, displays the prediction result displayed in the second predictive conversion field 119 in the first predictive conversion field 118, and displays new prediction result candidates extracted based on a predetermined priority in the second predictive conversion field 119. Then, the process returns to S21 in FIG. 10.

[0049] Also, if it is determined in S33 that a fixed phrase has not been selected (S33: NO), the gaze detection unit 242 determines whether a predetermined intention has been selected based on the acquired information indicating the position of the gaze point (S35). In this step, it is determined whether the user's gaze point is located in the operation area 114 marked with "Yes" or in the operation area 115 marked with "No" for a predetermined period of time or more. If the operation area 114 or 115 is selected (S35: YES), the input of the intention is accepted by interrupt (S36). In this step, the character input unit 243 outputs "Yes" or "No" by voice according to the selected operation area, or outputs it to a text box of another software. Then, the process returns to S21 in FIG. 10.

[0050] Also, if it is determined in S35 that a predetermined intention expression has not been selected, the gaze detection unit 242 determines whether a setting change of the present system has been instructed based on the acquired information indicating the position of the gaze point (S37). In this step, it is determined whether the user's gaze point is located in the operation area 120 marked with "first", the operation area 121 marked with "utterance", the operation area 122 marked with an upward triangle, or the operation area 123 marked with a downward triangle for a predetermined period of time or more. If these operation areas are selected (S37: YES), a predetermined setting change is made (S38). If the operation area 120, 122, or 123 is selected, the waiting time, which is a period during which no input is accepted after the first screen and the second screen are switched, is changed. Specifically, if the operation area 120 is selected, the setting change unit 244 changes the waiting time to a preset setting value for beginners. The setting value for beginners is, for example, 2 seconds. In this embodiment, a predetermined waiting time is provided after the screen is switched, but a user who is not familiar with the operation may take time to find the desired operation area, and in the process, an unintended operation area may be determined to have been selected. Therefore, by setting a longer waiting time after the screen is switched in the beginner mode, it is expected that the user will be able to operate with ease. In addition, the setting change unit 244 increases the gaze time when the operation area 122 is selected, and decreases the gaze time when the operation area 123 is selected. The gaze time may be increased or decreased by switching the setting in a predetermined step such as 0.4 seconds, 0.7 seconds, 1.0 seconds, and 1.3 seconds. In addition, the gaze time may be changed so that the gaze time for the operation area arranged on the inner circle is longer than the gaze time for the operation area arranged on the outer circle on the first screen. When the operation area 121 is selected, the activation / inactivation of the voice output is switched when the input character string is output. After S38, the process returns to S21 in FIG. 10.

[0051] If it is determined in S37 that no change in the setting has been instructed, the gaze detection unit 242 determines whether the operation area 116 marked with "OK" has been selected based on the acquired information indicating the position of the gaze point (S39). If the operation area 116 has been selected (S39: YES), the character strings displayed in the text boxes 125 and 117 are output (S40). In this step, the character input unit 243 outputs the character strings displayed in the text boxes 125 and 117, for example, by voice, or outputs them to a text box of another software executed by the control device 2. At this time, the presence or absence of voice output may be switched depending on the ON / OFF setting that can be changed by the operation area 121 marked with "Speak". After S40, the process proceeds to the process of FIG. 10.

[0052] Furthermore, if it is determined in S39 that output of a character string has not been instructed, the gaze detection unit 242 determines whether the operation area 124 marked with "Break" has been selected based on the acquired information indicating the position of the gaze point (FIG. 13: S41). If the operation area 124 has been selected (S41: YES), the system temporarily stops accepting gaze-based operations until an instruction to cancel the break is given. Here, the gaze detection unit 242 acquires information indicating the position of the user's gaze point on the display unit 32 of the display device 3 from the gaze detection device 4 (S42). Then, the gaze detection unit 242 determines whether the operation area 124 marked with "Break" has been selected again based on the acquired information indicating the position of the gaze point (S43). If the operation area 124 has not been selected, If not (S43: NO), the process returns to S42 and repeats. On the other hand, if the operation area 124 is selected (S43: YES), the process returns to the process of S21 in FIG. 10. If it is determined in S41 that the operation area 124 is not selected, the gaze detection unit 242 determines whether the operation area 112 marked with "x" is selected based on the acquired information indicating the position of the gaze point (S44). If the operation area 112 is selected (S44: YES), the character input unit 243 cancels the input of the character input immediately before (S45). That is, the last character of the character string displayed in the text box 117 is deleted. Note that if no character is displayed in the text box 117, the last character of the character string displayed in the text box 125 may be deleted. Then, after S45, the process returns to the process of S21 in FIG. 10. Also, when it is determined based on the acquired information indicating the position of the gaze point that none of the operation areas shown in Fig. 3 has been gazed upon for a predetermined gaze time or longer (S44: NO), the process returns to S21 of S10 and repeats the process. Note that the process may be terminated if an operation to end the operation is received from the user at any timing in Figs. 10 to 13.

[0053] <Modifications of the layout of the second screen> Among the operation areas on the outer arc in the first screen shown in Fig. 3, the operation areas 102, 104, 106, 108, 110, and 112 are located at the same positions on the display unit 32 of the display device 3 as the operation areas 131 to 136 in the second screen shown in Fig. 4. Therefore, although a waiting time is provided after switching between the first screen and the second screen, if it is determined that the user's line of sight remains in the same position, there is a risk of erroneous input occurring. Therefore, the arrangement of the operation areas may be different in the second screen after the operation areas 102, 104, 106, 108, 110, and 112 are selected.

[0054] FIG. 14 is a diagram showing an example of a second screen according to a modified example. In the modified example, the same reference numerals are used for corresponding components, and the description is omitted. In the example of FIG. 14, a plurality of operation areas arranged on an arc are displayed near the center of the display unit 32. On the arc, operation areas 131 to 136 are arranged clockwise from the left (the 9 o'clock direction when viewed from the center), displaying "ka", "ki", "ku", "ke", "ko" and "x". In addition, the text boxes 117 and 125 are the same as the corresponding components in FIG. 4. In this way, the operation areas 131 to 136 of the second screen may be arranged at positions that do not overlap with the operation areas 102, 104, 106, 108, 110 and 112 of the first screen, to suppress erroneous input. In this case, the waiting time after switching between the first screen and the second screen may be shortened or eliminated.

[0055] FIG. 15 is a diagram showing an example of a second screen according to another modified example. In the example of FIG. 15, a plurality of operation areas arranged on a circle with a radius of 3r are displayed near the center of the display unit 32. On the arc, operation areas 131 to 136 are arranged clockwise from the left (the 9 o'clock direction when viewed from the center), displaying "a", "i", "u", "e", "o" and "x". Each of the operation areas 131 to 136 is also a circle with a radius of r, and is arranged at intervals of 60 degrees from the center. That is, each of the operation areas 131 to 136 is also arranged at a distance of r or more from each other. The text boxes 117 and 125 are the same as the corresponding components in FIG. 4. Even in this way, the operation areas 131 to 136 of the second screen can be arranged at positions that do not overlap with the operation areas 101 to 112 of the first screen, and erroneous input can be suppressed. In this case, the waiting time after switching between the first screen and the second screen may be shortened or eliminated. In the example shown in FIG. 14, when operation areas 101, 103, 105, 107, 109, and 111 are selected on the first screen, the layout of the second screen is different from that when operation areas 102, 104, 106, 108, 110, and 112 are selected, so the user may be confused about the operation. In the example of FIG. 15, the layout of the second screen is unified, so the user can easily remember the layout of the operation areas. On the other hand, the gaze time of the user on operation areas 113-116 on the first screen is relatively long, so that the user's line of sight is not in the same position as operation areas 131-134 on the second screen after switching from the second screen to the first screen. It is unlikely that the

[0056] <User Interface Modifications> The characters displayed in the operation areas of the first screen and the second screen are not limited to those shown in FIG. 3, FIG. 4, etc. For example, it may be possible to input characters in Roman letters. That is, "a", "k", "s", "t", "n", "h", "m", "y", "r", and "w" are displayed in the operation areas 101-110 of the first screen, and "a", "i", "u", "e", and "o" are displayed in the operation areas 131-135 of the second screen, and characters may be input by combining the operation areas selected in the same manner as in the above-mentioned embodiment. In addition, it may be applied to languages ​​other than Japanese in which characters can be determined by combining vowels and consonants.

[0057] Fig. 16 is a diagram showing an example of a first screen according to another modified example. The size of operation areas 101-111 and the distance between the operation areas and the surrounding operation areas can be determined based on the expected distance from the user to the screen and the radius r calculated using the above-mentioned formula (1). Although the first screen shown in Fig. 3 may be used, the example of Fig. 16 uses an arrangement that users are accustomed to using on a general smartphone.

[0058] Fig. 17 is a diagram showing an example of a second screen according to another modified example. On the second screen as well, the operation areas are spaced apart in consideration of microsaccades. Furthermore, the operation areas 131-135 on the second screen are arranged at the vertices of a polygon (a pentagon in Fig. 17) or on a circle so as not to overlap with the operation areas on the first screen, preventing erroneous input.

[0059] Fig. 18 is a diagram showing an example of the second screen according to another modified example. In the example of Fig. 17, the operation areas 131-135 of the second screen are arranged at the vertices of a polygon or at positions on a circle centered on a selected operation area among the operation areas 101-112 of the first screen. Such operation areas 131-135 of the second screen are also arranged so as not to overlap with the operation areas of the first screen, preventing erroneous input. Also, compared to the example of Fig. 17, the distance of movement of the line of sight required for input operation is shorter.

[0060] In the example of Fig. 16, the operation area for inputting a predetermined intention and the predictive conversion field are omitted, but for example, operation areas for inputting fixed phrases such as "Yes", "No", "Thank you", "Hello", "Goodbye", and "Nice to meet you" may be arranged in the positions of operation areas 101 to 111 shown in Fig. 16, so that character strings that are used relatively frequently can be easily input. In this way, the efficiency of character input can be improved.

[0061] <Modifications of the gaze detection device> The gaze detection device 4 is not limited to a device that detects the position of the user's pupil using infrared rays. For example, it may be a device that identifies the direction of the gaze by detecting the position of an iris using an image captured using visible light. That is, a camera provided in a smartphone or tablet may be used as the gaze detection device 4, and a touch panel may be used as the display device 3. Furthermore, the gaze detection device 4 may be a user contact type device that uses a predetermined potential difference.

[0062] FIG. 19 is a functional block diagram showing an example of a character input device. In this modification, the control device 2 shown in FIG. 2 is, for example, a smartphone, the input / output device 23, which is, for example, a touch panel, also serves as the display unit 32 of the display device 3, and the input / output device 23, which is, for example, a front camera mounted on the touch panel side, also serves as part of the gaze detection device 4. The processor 24 is connected to the input / output device 23 via a signal line 25, and causes the input / output device 23 to display a predetermined user interface. The processor 24 also analyzes a facial image of the user obtained from the front camera, and determines the touch position based on, for example, the positions of the pupils and irises in the eyes. That is, the camera and the processor 24 function as a gaze detection device.

[0063] <Other> The above-mentioned configuration is merely an example, and the present invention is not limited to the illustrated configuration. For example, the above-mentioned items can be combined as appropriate without departing from the objectives and technical ideas of the present invention. Regarding the processing flow, the order of the processes can be changed or the processes can be performed in parallel as long as the results do not change. After S32, S34, S36, S38, and S40 in FIG. 12, and S43 and S45 in FIG. 13, the screen does not change, so the process may be returned to S22 in FIG. 10. Also, the above-mentioned processing steps may be executed by a computer using software, or by hardware such as a PLD (Programmable Logic Device). The present invention may be realized as a method executed by an apparatus by using a program executed by a computer. The program may be provided by recording it on a computer-readable recording medium. A computer-readable recording medium is a recording medium that stores information by electrical, magnetic, optical, mechanical, or chemical action and can be read by a computer. Among such recording media, those that can be removed from a computer include, for example, optical disks, magneto-optical disks, flexible disks, magnetic tapes, and memory cards. In addition, a hard disk drive (HDD) is an example of a recording medium that is fixed to a computer. ), SSD (Solid State Drive), ROM (Read Only Memory), etc.

[0064] Furthermore, the system of the present invention may be connected to the Internet and used as a device for communicating with people in remote locations, such as overseas, using only the eyes. Also, a plurality of the present systems may be connected to the Internet and used as a device for communicating with a plurality of people using only the eyes. That is, the character input system 1 shown in FIG. 2 or the character input device 2 shown in FIG. 19 is connected to one or more other character input systems 1 or character input devices 2 via the communication I / F 31 and the network. Then, the character input system 1 or the character input device 2 accepts character input from a user using the above-mentioned first screen and second screen, and performs processing to output the character input to the other character input system 1 or character input device 2. The output may be to display a character string, or to read aloud by voice synthesis. Also, software for performing such processing may be provided, or the system may operate in cooperation with an existing online call service or web conference system. When cooperating with an existing service or system, a character string is output to a predetermined text box of the other software, or a voice read aloud via virtual audio input / output is output to an audio input of the other software. [Explanation of symbols]

[0065] 1: Character input system 2: Control device 21: Communication I / F 22: Storage device 23: Input / output device 24: Processor 241: Output section 242: Gaze detection unit 243: Character input section 244: Setting change section 25: Bus 3:Display device 31: Communication I / F 32: Display section 4: Gaze detection device 41: Communication I / F 42: Infrared transmitter / receiver 43: Gaze detection unit 101~116, 120~124, 131~136: Operation area 117, 125: Text box 118, 119: Predictive conversion column

Claims

1. a gaze detection unit that switches between a first screen including a plurality of regions related to consonants and a second screen including a plurality of regions related to vowels, which is displayed when it is determined that any one of the plurality of regions related to consonants has been selected on the first screen, detects the gaze of a user facing the display device, acquires information indicating the gaze point from a gaze detection device that outputs the gaze point of the user, and determines that the region has been selected when the gaze point indicated by the acquired information is included in the predetermined region for a predetermined gaze time or longer; a character input unit that accepts an input of a character according to a combination of one of a plurality of areas related to the consonant selected on the first screen and one of a plurality of areas related to the vowel selected on the second screen based on a determination of the gaze detection unit; Equipped with a plurality of areas included in the first screen and a plurality of areas included in the second screen are displayed at a distance equal to or greater than a predetermined distance from other areas included in the same screen, Each of the plurality of areas included in the second screen is displayed at a position on the display device that does not overlap with a position at which the selected area is displayed on the first screen. Character input device.

2. A gaze detection unit which switches between a first screen including a plurality of regions relating to consonants and a second screen including a plurality of regions relating to vowels which is displayed when it is determined that any one of the plurality of regions relating to consonants has been selected on the first screen, and which detects the gaze of a user facing the display device, acquires information indicating the gaze point from a gaze detection device which outputs the user's gaze point, and determines that the region has been selected when the gaze point indicated by the acquired information is included in the predetermined region for a predetermined gaze time or longer; a character input unit that accepts an input of a character according to a combination of one of a plurality of areas related to the consonant selected on the first screen and one of a plurality of areas related to the vowel selected on the second screen based on a determination of the gaze detection unit; Equipped with a plurality of areas included in the first screen and a plurality of areas included in the second screen are displayed at a distance equal to or greater than a predetermined distance from other areas included in the same screen, The plurality of areas included in the first screen and the plurality of areas included in the second screen are Each is arranged on an arc, the first screen further includes a plurality of areas for a user to indicate a predetermined intention, the plurality of areas relating to consonants being arranged on an outer arc of a double circle, and the plurality of areas for indicating the predetermined intention being arranged on an inner arc of the double circle, The gaze detection unit takes the gaze time required to determine that a plurality of areas for expressing the predetermined intention have been selected longer than the gaze time required to determine that a plurality of areas related to the consonants have been selected on the first screen. Character input device.

3. The plurality of regions included in the first screen and the plurality of regions included in the second screen are arranged on a circular arc. The character input device according to claim 1 .

4. The first screen further includes a plurality of areas for a user to express a predetermined intention, the plurality of areas relating to consonants being arranged on an outer arc of a double circle, and the plurality of areas for expressing the predetermined intention being arranged on an inner arc of the double circle. The character input device according to claim 3 .

5. The plurality of regions relating to the vowels on the second screen are arranged on the same arc as the outer arc of the double circle on the first screen.

5. The character input device according to claim 2 or 4.

6. The gaze detection unit takes the gaze time required to determine that a plurality of areas for expressing the predetermined intention have been selected longer than the gaze time required to determine that a plurality of areas related to the consonants have been selected on the first screen.

6. The character input device according to claim 4 or 5.

7. the first screen further includes a temporary text box for temporarily displaying a character string that has been input, and a decision area for outputting the character strings displayed in the temporary text box to another predetermined text box or outputting them as voice, When the gaze detection unit determines that the decision area has been selected, the character input unit outputs the character strings displayed in the temporary text box to the predetermined text box or outputs them as voice, When the gaze detection unit determines that any one of a plurality of areas for performing the predetermined intention has been selected, the character input unit performs output according to the selected area for performing the predetermined intention before outputting the character string displayed in the temporary text box.

7. The character input device according to claim 2, 4, 5 or 6.

8. the first screen further includes a rest button area for stopping a process by the gaze detection unit to determine that the area has been selected, When the gaze detection unit determines that the rest button area is selected, the gaze detection unit stops accepting operations other than an operation to release the pause. The character input device according to claim 1 .

9. The gaze detection unit provides a waiting time during which it does not determine whether the gaze point is included in a predetermined area for a predetermined gaze time or more after switching the display of the display device between the first screen and the second screen. The character input device according to claim 1 .

10. the first screen includes a change area for changing the gaze time or the waiting time, When it is determined that the change area is selected, the gaze detection unit increases or decreases the gaze time or the waiting time. The character input device according to claim 9.

11. The predetermined distance is a radius r of a centrally visible range calculated by the following formula, where l is an expected distance from the eyeball of the user to the gaze point on the display device, and θ is an angle between a straight line connecting the eyeball and the gaze point and a straight line connecting the eyeball and an edge of a centrally visible range on the display device. r = l tan θ The character input device according to claim 1 .

12. The display device; The gaze detection device; The character input device according to claim 1 , A character input system comprising:

13. a display step of switching between a first screen including a plurality of areas related to consonants and a second screen including a plurality of areas related to vowels, which is displayed when it is determined that any one of the plurality of areas related to consonants is selected on the first screen, and displaying the first screen and the second screen on the display device; a gaze detection step of detecting a gaze of a user facing the display device, acquiring information indicating the gaze point from a gaze detection device that outputs a gaze point of the user, and determining that a predetermined area has been selected when the gaze point indicated by the acquired information is included in the predetermined area for a predetermined gaze time or longer; a character input step of accepting an input of a character corresponding to a combination of any one of a plurality of areas related to the consonant selected on the first screen and any one of a plurality of areas related to the vowel selected on the second screen based on a determination made in the line of sight detection step; The computer executes a plurality of areas included in the first screen and a plurality of areas included in the second screen are displayed at a distance equal to or greater than a predetermined distance from other areas included in the same screen, Each of the plurality of areas included in the second screen is displayed at a position on the display device that does not overlap with a position at which the selected area is displayed on the first screen. How to enter text.

14. a display step of switching between a first screen including a plurality of areas related to consonants and a second screen including a plurality of areas related to vowels, which is displayed when it is determined that any one of the plurality of areas related to consonants is selected on the first screen, and displaying the first screen and the second screen on the display device; a gaze detection step of detecting a gaze of a user facing the display device, acquiring information indicating the gaze point from a gaze detection device that outputs a gaze point of the user, and determining that a predetermined area has been selected when the gaze point indicated by the acquired information is included in the predetermined area for a predetermined gaze time or longer; a character input step of accepting an input of a character corresponding to a combination of any one of a plurality of areas related to the consonant selected on the first screen and any one of a plurality of areas related to the vowel selected on the second screen based on a determination made in the line of sight detection step; on the computer, a plurality of areas included in the first screen and a plurality of areas included in the second screen are displayed at a distance equal to or greater than a predetermined distance from other areas included in the same screen, Each of the plurality of areas included in the second screen is displayed at a position on the display device that does not overlap with a position at which the selected area is displayed on the first screen. Text input program.

Images