control device

Abstract

To provide a technique for achieving further improvement in a control apparatus.SOLUTION: In a control apparatus 12, a coordinate reception unit 26 receives spatial coordinates of a detection object located in front of a display surface of a display 14. A setting unit 32 sets a first determination region associated with first character information included in a first character information group based on coordinates of a first display region in which information on the first character information group is displayed on the display surface and spatial coordinates of the detection object at a reference time that is a time when the first display region is designated by the detection object. After the first determination region is set, a determination unit 30 determines whether or not the detection object is located in the first determination region. A display output unit 36 displays the first character information in the first display region when the determination unit 30 determines that the detection object is located in the first determination region.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] This disclosure relates to a control device.

Background Art

[0002] There is known an information processing apparatus that stores virtual keys, which are keys of a virtual keyboard arranged in space, in association with a plurality of layers (see, for example, Patent Document 1). In this information processing apparatus, the position of an instruction medium in space is specified, and when the instruction medium is located in a region corresponding to a layer, the virtual keys stored in association with the layer are displayed on a display screen.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a control device that receives operation inputs from a user, further improvements are required.

Means for Solving the Problems

[0005] To solve the above problems, a control device according to an aspect of the present disclosure includes a coordinate acquisition unit that acquires spatial coordinates of a detection target located in front of a display surface of a display, coordinates of a first display area on the display surface where information regarding a first character information group is displayed, and spatial coordinates of the detection target at a reference time which is the time when the first display area is designated by the detection target, and a setting unit that sets a first determination area associated with first character information included in the first character information group based on these, a determination unit that determines whether or not the detection target is located within the first determination area after the first determination area is set, and a display output unit that causes the first character information to be displayed in the first display area when the determination unit determines that the detection target is located within the first determination area.

[0006] Another aspect of this disclosure is also a control device. This device includes: a coordinate acquisition unit that acquires the spatial coordinates of a first detection target and a second detection target located in front of the display surface of a display; a setting unit that sets a first determination area associated with first character information included in a first character information group based on the spatial coordinates of the first detection target at a reference time which is the time when a predetermined input start condition is met, and sets a second determination area associated with second character information included in a second character information group based on the spatial coordinates of the second detection target at a reference time; a determination unit that determines whether the first detection target is located within the first determination area and whether the second detection target is located within the second determination area after the first and second determination areas have been set; and a display output unit that, if the determination unit determines that the first detection target is located within the first determination area, displays the first character information in a first display area of ​​the display surface, and if the determination unit determines that the second detection target is located within the second determination area, displays the second character information in a second display area of ​​the display surface.

[0007] Another aspect of this disclosure is also a control device. This device includes: a coordinate acquisition unit that acquires the spatial coordinates of a detection target located in front of the display surface of a display; a setting unit that sets a first determination area associated with first character information based on the spatial coordinates of the detection target when the detection target enters a predetermined input space; a determination unit that determines whether or not the detection target is located within the first determination area after the first determination area has been set; and a display output unit that displays the first character information on the display surface if the determination unit determines that the detection target is located within the first determination area. [Effects of the Invention]

[0008] Further improvements can be achieved through the above-described method. [Brief explanation of the drawing]

[0009] [Figure 1] This is a block diagram of the information processing system according to the first embodiment. [Figure 2]This is a plan view of the display used to explain the character input operation in the information processing system shown in Figure 1. [Figure 3] This diagram, following Figure 2, illustrates the text input process. [Figure 4] This diagram, following Figure 3, illustrates the text input process. [Figure 5] Figure 1 is a flowchart showing the processing steps of the information processing system. [Figure 6] This is a plan view of a display to illustrate another example of the operation of specifying the display area in the information processing system shown in Figure 1. [Figure 7] This figure shows another example of the decision area in the information processing system shown in Figure 1. [Figure 8] This is a diagram illustrating the character input operation in the information processing system of the second embodiment. [Figure 9] This is a plan view of a display illustrating the character input operation in the information processing system of the third embodiment. [Figure 10] This is a plan view of the display, following Figure 9, to explain the text input operation. [Figure 11] This is a flowchart showing the processing of the information processing system in the third embodiment. [Figure 12] This is a perspective view of a display illustrating the character input operation in the information processing system of the fourth embodiment. [Figure 13] This diagram, following Figure 12, illustrates the text input process. [Figure 14] This is a flowchart showing the processing of the information processing system according to the fourth embodiment. [Figure 15] This is a plan view of a display illustrating the character input operation in the information processing system of the fifth embodiment. [Figure 16] This is a plan view of the display, following Figure 15, to explain the text input operation. [Figure 17] This is a plan view of the display, following Figure 16, to explain the text input operation. [Figure 18]It is a plan view of a display for explaining another example of a character input operation in the information processing system according to the fifth embodiment. [Figure 19] It is a plan view of a display for explaining a character input operation, following FIG. 18. [Figure 20] It is a flowchart showing the processing of the information processing system according to the fifth embodiment. [Figure 21] It is a flowchart following FIG. 20. [Figure 22] It is a diagram for explaining a first example of a character input operation in the information processing system according to the sixth embodiment. [Figure 23] It is a diagram for explaining a second example of a character input operation in the information processing system according to the sixth embodiment. [Figure 24] It is a plan view of a display for explaining a third example of a character input operation in the information processing system according to the sixth embodiment. [Figure 25] It is a plan view of a display for explaining a fourth example of a character input operation in the information processing system according to the sixth embodiment. [Figure 26] It is a diagram for explaining a fifth example of a character input operation in the information processing system according to the sixth embodiment. [Figure 27] It is a diagram showing various setting examples of a determination area in an information processing system. [Figure 28] It is a diagram showing another setting example of a determination area in an information processing system. [Figure 29] It is a diagram showing yet another setting example of a determination area in an information processing system. [Figure 30] It is a perspective view of a display for explaining a character input operation in the information processing system according to the seventh embodiment. [Figure 31] It is a diagram for explaining a character input operation, following FIG. 30. [Figure 32] It is a diagram showing another setting example of a plurality of determination areas in the seventh embodiment. [Figure 33]This figure illustrates the resizing process of the determination area in the information processing system of the seventh embodiment. [Figure 34] This figure illustrates the character input operation in an information processing system that is a modified example of the first embodiment. [Modes for carrying out the invention]

[0010] In the following, identical or equivalent components, members, and processes shown in each drawing will be denoted by the same reference numerals, and redundant explanations will be omitted where appropriate. Furthermore, the dimensions of the members in each drawing will be enlarged or reduced as appropriate for ease of understanding.

[0011] (First Embodiment) Figure 1 is a block diagram of the information processing system 1 of the first embodiment. The information processing system 1 includes a sensor 10, a control device 12, and a display 14.

[0012] The information processing system 1 may be installed indoors or mounted in a vehicle such as an automobile, and its use is not particularly limited. The user can select characters by performing a character input action by moving their fingertips in the air without touching the display 14, and input the selected characters into the control device 12. This character input action can also be called an air touch action or gesture action.

[0013] Sensor 10 is, for example, a depth camera and is positioned to capture images of the user located in front of the display surface of the display 14. Sensor 10 periodically captures images of the user and outputs the captured time-series distance images to the control device 12. The imaging frequency can be determined appropriately through experiments or simulations, and may be, for example, multiple times per second. The distance image contains spatial coordinate information for each position in the image. These spatial coordinates are coordinates in a three-dimensional Cartesian coordinate system specific to sensor 10.

[0014] The control device 12 recognizes the user's fingertip movements based on the time-series distance images supplied from the sensor 10 and accepts character input according to the fingertip movements. The control device 12 then performs processing based on the received characters.

[0015] Fingertips are just one example of an object that can be used for input, and the objects to be detected are not limited to fingertips. Other examples of objects to be detected include a clenched fist, which is a part of the user's body. The objects to be detected are not limited to a part of the user's body; they may also be objects held in the user's hand or objects worn on the user's body.

[0016] The display 14 displays an image according to the control of the control device 12. The display 14 may be, for example, a screen with a screen size of several tens of inches or more for indoor installation, or a smaller screen size for in-vehicle installation.

[0017] As shown in Figure 1, the control device 12 comprises a control unit 20 and a storage unit 22. The control unit 20 includes an image acquisition unit 24, a coordinate acquisition unit 26, a calculation unit 28, a determination unit 30, a setting unit 32, a function execution unit 34, and a display output unit 36.

[0018] The configuration of the control unit 20 can be realized in hardware terms using the CPU, memory, and other LSIs of any computer, and in software terms by programs loaded into memory, etc., but here we are describing the functional blocks realized by the cooperation of these.Therefore, it will be understood by those skilled in the art that these functional blocks can be realized in various ways by hardware alone, software alone, or a combination thereof.The control device 12 comprises, for example, a processor and memory, and the processor executes programs loaded into memory to realize the functions of the control unit 20 and each of the functional blocks of the control unit 20.

[0019] The storage unit 22 stores various types of information. The storage unit 22 is implemented by hardware for storing information (in other words, data), such as memory or storage. The storage unit 22 holds coordinate information, character information, display area information, and judgment area information. The coordinate information includes the spatial coordinates of the installation position of the sensor 10, the mounting angle of the sensor 10, and the spatial coordinates of the installation position of the display 14. The character information includes information for identifying multiple character information groups and the character information contained in each of the multiple character information groups. The display area information includes information on the center coordinates and size of each of the multiple display areas, and information on the correspondence between the display areas and the character information groups. The judgment area information includes information on the sizes of multiple judgment areas. Details of the coordinate information, character information, display area information, and judgment area information will be described later.

[0020] Figure 2 is a plan view of the display 14 used to illustrate the character input operation in the information processing system 1 shown in Figure 1. In Figure 2, only a portion of the display surface 15 of the display 14 is shown, and the sensor 10 and control device 12 are not shown.

[0021] The following describes an example of how the control device 12 accepts input of Japanese hiragana characters. The characters that can be input are not particularly limited; the control device 12 may also accept input of Japanese katakana characters, numbers as will be described later, or characters in any language such as English.

[0022] The five characters in the "a" row, "a", "i", "u", "e", and "o", are designated as the first character information group. The five characters in the "ka" row are designated as the second character information group. The five characters in the "sa" row are designated as the third character information group. The five characters in the "ta" row are designated as the fourth character information group. The five characters in the "na" row are designated as the fifth character information group. The five characters in the "ha" row are designated as the sixth character information group. The five characters in the "ma" row are designated as the seventh character information group. The three characters in the "ya" row are designated as the eighth character information group. The five characters in the "ra" row are designated as the ninth character information group. The semi-voiced consonant mark "゜" and the voiced consonant mark "゛" are designated as the tenth character information group. The characters in the "wa" row, "wa", "wo", and "n", are designated as the tenth character information group. The symbols "?", ".", "," and "!" are considered character information for the 11th character information group.

[0023] In Figure 2, the display surface 15 displays an image in its initial state. The display surface 15 has a first display area 40_1, a second display area 40_2, a third display area 40_3, a fourth display area 40_4, a fifth display area 40_5, a sixth display area 40_6, a seventh display area 40_7, an eighth display area 40_8, a ninth display area 40_9, a tenth display area 40_10, an eleventh display area 40_11, a twelfth display area 40_12, and an input display area 42. When these first display area 40_1 etc. are not distinguished, they are referred to as display area 40. Multiple display areas 40 can also be called a screen keyboard. Display areas 40 and character information groups are pre-associated on a one-to-one basis.

[0024] The display output unit 36, based on the correspondence information between the display area 40 and the character information group stored in the storage unit 22, displays information about each of the multiple character information groups in the corresponding display area 40 in the initial image. In the initial image, the information about the character information group is, for example, information about the character that represents the character information group. The information about the character that represents the character information group can also be called information for identifying the character information group.

[0025] The first display area 40_1 displays the character "あ" which represents the first character information group. The second display area 40_2 displays the character "か" which represents the second character information group. The third display area 40_3 displays the character "さ" which represents the third character information group. The same applies to the fourth display area 40_4 to the twelfth display area 40_12. As will be described later, the characters displayed in each display area 40 may change depending on the user's character input actions.

[0026] Furthermore, as will be described later, the input display area 42 displays the characters entered in response to the user's character input actions.

[0027] Figure 2 shows the X, Y, and Z axes of the three-dimensional Cartesian coordinate system of the information processing system 1. The Z axis extends in the direction normal to the display surface 15 of the display 14. The X axis extends in the horizontal direction of the display surface 15. The Y axis extends in the vertical direction of the display surface 15.

[0028] The following describes an example of a user inputting the character "i" from the "a" row. As shown in Figure 2, first, the user uses their fingertip 50 to point to the first display area 40_1 where the character "a" is displayed, without touching the display 14. Next, the user moves their fingertip 50 closer to the first display area 40_1 until the character "a" displayed in the first display area 40_1 changes to the character "i". When the character "i" is displayed in the first display area 40_1, the user performs a predetermined confirmation action with their fingertip. As a result, the character "i" is displayed in the input display area 42 and input to the control device 12. The user can input multiple characters by repeating these actions. A more detailed explanation follows.

[0029] The image acquisition unit 24 acquires time-series distance images of the user positioned in front of the display 14 from the sensor 10, and detects the user's fingertips 50 and eyes 52 by performing image recognition on each of the acquired distance images. Known image recognition techniques can be used to detect the fingertips 50 and eyes 52. The image acquisition unit 24 acquires the spatial coordinates of the detected fingertips 50 and eyes 52 in time series and outputs them to the coordinate acquisition unit 26.

[0030] The coordinate acquisition unit 26 calculates and acquires the time-series spatial coordinates of the user's fingertip 50 and eye 52 in the three-dimensional Cartesian coordinate system of the information processing system 1, based on the spatial coordinates of the user's fingertip 50 and eye 52 in the coordinate system of the sensor 10 supplied by the image acquisition unit 24, and the spatial coordinates of the installation position of the sensor 10 and the mounting angle of the sensor 10 included in the coordinate information stored in the storage unit 22. The coordinate acquisition unit 26 outputs the acquired spatial coordinates to the calculation unit 28 and the determination unit 30. The processing of the calculation unit 28 will be described later.

[0031] The determination unit 30 determines whether the fingertip 50 is specifying one of the multiple display areas 40 based on the spatial coordinates of the fingertip 50 and eye 52 supplied from the coordinate acquisition unit 26, and the information on the center coordinates and size of each of the multiple display areas 40 stored in the storage unit 22.

[0032] The determination unit 30 determines that if the display area 40 lies on the straight line L0 passing through the eye 52 and the fingertip 50, the fingertip 50 has specified the display area 40 on that straight line L0. The determination unit 30 determines that if the display area 40 does not lie on the straight line L0 passing through the eye 52 and the fingertip 50, the fingertip 50 has not specified the display area 40. The determination unit 30 outputs the determination result to the setting unit 32. If the determination unit 30 determines that the fingertip 50 has specified one of the multiple display areas 40, it also outputs the spatial coordinates of the fingertip 50 used for the determination to the setting unit 32. The spatial coordinates of the fingertip 50 used for the determination are the spatial coordinates of the fingertip 50 at the reference time, which is the time when the display area 40 was specified by the fingertip 50.

[0033] In the example shown in Figure 2, the user moves at least one of their fingertip 50 and eye 52 so that the first display area 40_1 is located on the line L0 in order to specify the first display area 40_1. The determination unit 30 determines that the first display area 40_1 intersecting the line L0 has been specified.

[0034] Figure 3 is a diagram illustrating the character input operation, following Figure 2. Figure 3(a) is a plan view of the display 14, Figure 3(b) is a side view of the display 14, and Figure 3(c) is a top view of the display 14. In Figure 3(a), the first display area 40_1, the second display area 40_2, and the third display area 40_3 are shown enlarged compared to Figure 2, while the other display areas 40 are omitted from the illustration.

[0035] Let n be the number of display areas 40, and let i be any integer from 1 to n. If the determination unit 30 determines that the fingertip 50 is specifying the i-th display area 40_i, the setting unit 32 sets the first determination area 44_1, the second determination area 44_2, the third determination area 44_3, the fourth determination area 44_4, and the fifth determination area 44_5 between the i-th display area 40_i and the fingertip 50, based on the coordinates of the i-th display area 40_i and the spatial coordinates of the fingertip 50 at the reference time supplied by the determination unit 30. If these first determination area 44_1 etc. are not distinguished, they are called determination area 44.

[0036] The setting unit 32 sets an operational line L1 connecting the spatial coordinates of the fingertip 50 at the reference time and the i-th display area 40_i, and sets multiple judgment areas 44 such that multiple judgment areas 44 intersect the line L1. The setting unit 32 may also set an operational line L1 connecting the spatial coordinates of the eye 52 at the reference time, the spatial coordinates of the fingertip 50 at the reference time, and the i-th display area 40_i. In this case, the operational line L1 is the same as line L0. The setting unit 32 may also set an operational line L1 connecting the spatial coordinates of the eye 52 at the reference time and the i-th display area 40_i.

[0037] The setting unit 32 sets multiple determination areas 44 of the same size at equal intervals parallel to the display surface 15, based on the size information of the multiple determination areas 44 stored in the storage unit 22. The determination areas 44 have any three-dimensional shape and may be plate-shaped, for example, as shown in the example in Figure 3. The cross-sectional area and shape of the determination area 44 in a cross section parallel to the display surface 15 are not particularly limited, but may be equivalent to the area and shape of the display area 40, for example, as shown in the example in Figure 3. The determination areas are arranged in the order of first determination area 44_1, second determination area 44_2, third determination area 44_3, fourth determination area 44_4, and fifth determination area 44_5, starting from the fingertip 50 side towards the display surface 15 side. The setting unit 32 associates the multiple character information included in the i-th character information group with the multiple determination areas 44 on a one-to-one basis.

[0038] In the example shown in Figure 3, since the fingertip 50 selected the first display area 40_1, the setting unit 32 sets multiple judgment areas 44 that intersect the straight line L1 connecting the spatial coordinates of the fingertip 50 at the reference time and the center coordinates of the first display area 40_1. The first judgment area 44_1 corresponds to the character "あ", the second judgment area 44_2 corresponds to the character "い", the third judgment area 44_3 corresponds to the character "う", the fourth judgment area 44_4 corresponds to the character "え", and the fifth judgment area 44_5 corresponds to the character "お".

[0039] Furthermore, when the determination area 44 is set, the setting unit 32 sets the selection area R1 shown in Figures 3(b) and 3(c). The selection area R1 includes multiple determination areas 44 and has a three-dimensional shape. The cross-sectional area and shape of the selection area R1 in a cross section parallel to the display surface 15 are not particularly limited, but may be equivalent to the area and shape of the determination area 44, for example, as shown in the example in Figure 3. The determination unit 30 determines whether the fingertip 50 is located within the selection area R1. If the determination unit 30 determines that the fingertip 50 is located within the selection area R1, the setting unit 32 maintains the multiple determination areas 44.

[0040] If the determination unit 30 determines that the fingertip 50 is not located within the selection area R1, the setting unit 32 cancels the settings for the multiple determination areas 44. If the determination unit 30 determines that the fingertip 50 is not located within the selection area R1, it determines again whether the fingertip 50 is specifying one of the multiple display areas 40. In this case, the user can specify a new display area 40 with the fingertip 50.

[0041] Figure 4 is a diagram illustrating the character input operation, following Figure 3. Figure 4(a) is a plan view of the display 14, and Figure 4(b) is a side view of the display 14. As shown in Figure 4, the user brings their fingertip 50 close to the first display area 40_1, and the fingertip 50 is in the second judgment area 44_2 for the character "い".

[0042] After multiple determination areas 44 have been set, the determination unit 30 determines whether or not the fingertip 50 is located within any of the determination areas 44, and outputs the determination result to the function execution unit 34. The determination result includes information for identifying the determination area 44 in which the fingertip 50 is located.

[0043] If the determination unit 30 determines that the fingertip 50 is located within any of the determination areas 44, the function execution unit 34 performs a process on the image displayed on the display surface 15 to change the character information of the i-th display area 40_i based on the character information held in the storage unit 22. The function execution unit 34 generates image data and supplies it to the display output unit 36.

[0044] The display output unit 36 ​​outputs image data supplied from the function execution unit 34 to the display 14, causing the image to be displayed on the display 14.

[0045] In other words, with m being the number of determination areas and j being any integer from 1 to m, the display output unit 36 ​​displays the character information associated with the j-th determination area 44_j in the i-th display area 40_i when the determination unit 30 determines that the fingertip is located within the j-th determination area 44_j.

[0046] The function execution unit 34 may highlight the characters in the i-th display area 40_i while the fingertip is located in the j-th determination area 44_j. Examples of highlighting include changing the color of the characters or changing the size of the characters. Even when multiple determination areas 44 are set with intervals between them, the highlighting allows the user to recognize whether the fingertip 50 is located in a determination area 44, that is, whether or not it is possible to input characters at the position of the fingertip 50.

[0047] In the example shown in Figure 4, since the fingertip 50 is located in the second determination area 44_2, the display output unit 36 ​​displays the character "い" in the first display area 40_1 instead of the character "あ".

[0048] The calculation unit 28 calculates the trajectory of the fingertip 50 in a predetermined unit of time based on the spatial coordinates supplied from the coordinate acquisition unit 26, and supplies the calculated trajectory information to the determination unit 30. The trajectory of the fingertip 50 is used to determine whether a decision action has been performed. If the decision action is an action that stops the fingertip 50 for a certain period of time, the calculation unit 28 does not need to be provided.

[0049] The determination unit 30 determines whether a decision action was performed by the fingertip 50 in the j-th determination area 44_j, based on the spatial coordinates supplied from the coordinate acquisition unit 26 and the trajectory information of the fingertip 50 supplied from the calculation unit 28.

[0050] When the determination unit 30 determines that the fingertip 50 is located in the j-th determination area 44_j and that a determination operation has been performed, the function execution unit 34 determines the character information associated with the j-th determination area 44_j and accepts the input of said character information. The function execution unit 34 may use the received character information to execute processing for a predetermined application, or it may supply the received character information to another processing device (not shown).

[0051] The function execution unit 34 performs the process of adding the received character information to the input display area 42 on the image displayed on the display surface 15.

[0052] In other words, if the determination unit 30 determines that the fingertip 50 is located in the j-th determination area 44_j and that a determination operation has been performed, the display output unit 36 ​​displays the character information associated with the j-th determination area 44_j in the input display area 42.

[0053] In the example shown in Figure 4, the user makes a decision with their fingertip 50 when the character "い" they want to input is displayed in the first display area 40_1, that is, when their fingertip 50 is located in the first determination area 44_1. In response, the function execution unit 34 accepts the input of the character "い", which is the character information associated with the first determination area 44_1, and the display output unit 36 ​​displays the character "い" in the input display area 42.

[0054] Thus, according to this embodiment, a determination area 44 corresponding to the display area 40 specified by the user's fingertip 50 can be set in the space between the fingertip 50 and the specified display area 40. Therefore, it is easy to select text information regardless of the size of the display surface 15 of the display 14.

[0055] Furthermore, the user can select and confirm character information included in the character information group associated with the display area 40 by moving their fingertip 50 closer to the specified display area 40 from the position of their fingertip 50 when the display area 40 was specified. In other words, character information can be entered by moving the fingertip 50 diagonally or vertically relative to the specified display area 40. Therefore, it is intuitive to operate and has good usability.

[0056] Next, the overall operation of the information processing system 1 with the above configuration will be explained. Figure 5 is a flowchart showing the processing of the information processing system 1 in Figure 1. The determination unit 30 determines whether the display area 40 lies on the straight line L0 passing through the eye 52 and the fingertip 50 (S10). If the display area 40 does not lie on the straight line L0 (N in S10), the determination unit 30 returns to the processing of S10.

[0057] If the display area 40 lies on the line L0 (Y in S10), the determination unit 30 determines the i-th character information group corresponding to the identified i-th display area 40_i (S12). The setting unit 32 sets the operation line L1 (S14) and sets the determination area 44 and the selection area R1 on the operation line L1 (S16).

[0058] The determination unit 30 determines whether the fingertip 50 is located within any of the determination areas 44 (S18). If the fingertip 50 is located within any of the determination areas 44 (Y in S18), the display output unit 36 ​​displays the character information associated with the identified j-th determination area 44_j in the i-th display area 40_i (S20).

[0059] The determination unit 30 determines whether a decision operation has occurred in the j-th determination area 44_j (S22). If a decision operation has occurred (Y in S22), the function execution unit 34 determines the character information associated with the j-th determination area 44_j (S24). The determination unit 30 determines whether the fingertip 50 has moved out of the selection area R1 (S26). If the fingertip 50 has not moved out of the selection area R1 (N in S26), the determination unit 30 returns to the process in S18.

[0060] If the fingertip 50 moves out of the selection area R1 (Y in S26), the display output unit 36 ​​displays the character information representing the i-th character information group in the i-th display area 40_i (S28), and the processing of the information processing system 1 returns to the processing of S10 by the determination unit 30.

[0061] In S18, if the fingertip 50 is not located within any of the determination areas 44 (N in S18), the determination unit 30 proceeds to the process in S26. In S22, if there is no decision operation in the j-th determination area 44_j (N in S22), the determination unit 30 proceeds to the process in S26.

[0062] The determination unit 30 may determine whether the fingertip 50 is pointing to one of the multiple display areas 40, without using the position of the user's eye 52.

[0063] Figure 6 is a plan view of the display 14 to illustrate another example of the operation of specifying the display area 40 in the information processing system 1 of Figure 1. In Figure 6, the illustration of the input display area 42 is omitted. Figure 6(a) is a diagram illustrating the operation of specifying the display area 40 based on the direction of the fingertip 50. The coordinate acquisition unit 26 acquires the time-series spatial coordinates of the user's fingertip 50 and a predetermined position 54 on the finger. The predetermined position 54 on the finger is, for example, the first joint, the second joint, or the base of the finger. The determination unit 30 determines that if the display area 40 lies on a straight line L1 passing through the fingertip 50 and the predetermined position 54 on the finger, the fingertip 50 is specifying the display area 40 on the said straight line L1. In the example of Figure 6(a), it is determined that the fingertip 50 is specifying the first display area 40_1. The straight line L1 is also used as a straight line for operation.

[0064] Figure 6(b) is a diagram illustrating the operation of specifying the display area 40 based on the direction of movement of the fingertip 50. The determination unit 30 determines that the fingertip 50 is specifying the display area 40 if the display area 40 lies on a straight line L1 that passes through the spatial coordinates of the fingertip 50 at time t1 and the spatial coordinates of the fingertip 50 at time t2, which is after time t1. In the example in Figure 6(b), it is determined that the fingertip 50 is specifying the first display area 40_1. The straight line L1 is also used as a straight line for operation. In these modified examples, the user can specify the display area 40 without considering the position of the eye 52.

[0065] Furthermore, the sizes of the multiple determination areas 44 may differ. Figure 7 shows another example of the determination area 44 in the information processing system 1 of Figure 1. Figure 7(a) is a plan view of the display 14, Figure 7(b) is a side view of the display 14, and Figure 7(c) is a top view of the display 14.

[0066] The setting unit 32 sets the cross-sectional area of ​​the judgment area 44 in a section parallel to the display surface 15 to be smaller the closer it is to the position of the fingertip 50 at the reference time. The cross-sectional area of ​​the second judgment area 44_2 in a section parallel to the display surface 15 is larger than the cross-sectional area of ​​the first judgment area 44_1 in a section parallel to the display surface 15. The cross-sectional area of ​​the third judgment area 44_3 in a section parallel to the display surface 15 is larger than the cross-sectional area of ​​the second judgment area 44_2 in a section parallel to the display surface 15. The cross-sectional area of ​​the fourth judgment area 44_4 in a section parallel to the display surface 15 is larger than the cross-sectional area of ​​the third judgment area 44_3 in a section parallel to the display surface 15. The cross-sectional area of ​​the fifth judgment area 44_5 in a section parallel to the display surface 15 is larger than the cross-sectional area of ​​the fourth judgment area 44_4 in a section parallel to the display surface 15.

[0067] In each judgment region 44, the cross-sectional area of ​​the judgment region 44 in a section parallel to the display surface 15 is equal regardless of the position of the section parallel to the display surface 15.

[0068] Furthermore, the volume of the second determination region 44_2 is greater than the volume of the first determination region 44_1. The volume of the third determination region 44_3 is greater than the volume of the second determination region 44_2. The volume of the fourth determination region 44_4 is greater than the volume of the third determination region 44_3. The volume of the fifth determination region 44_5 is greater than the volume of the fourth determination region 44_4.

[0069] This modified version allows for greater flexibility in setting the judgment area 44.

[0070] (Second Embodiment) In the second embodiment, the setting direction of the determination area 44 differs from that of the first embodiment. The differences from the first embodiment will be explained below.

[0071] Figure 8 is a diagram illustrating the character input operation in the information processing system 1 of the second embodiment. Figure 8(a) is a plan view of the display 14, Figure 8(b) is a side view of the display 14, and Figure 8(c) is a top view of the display 14.

[0072] The setting unit 32 sets multiple determination regions 44 such that multiple determination regions 44 intersect with line L1, which is a straight line L1 connecting the spatial coordinates of the fingertip 50 at the reference time and the display surface 15, and is the normal to the display surface 15. In other words, line L1 is different from line L0.

[0073] In the example shown in Figure 8, since the fingertip 50 selected the second display area 40_2, the setting unit 32 sets multiple determination areas 44 corresponding to the characters "ka", "ki", "ku", "ke", and "ko" in the space between the fingertip 50 and the display surface 15.

[0074] According to this embodiment, the user can select character information by moving their fingertip 50 along the normal direction of the display surface 15 from the position of their fingertip 50 when they specified the display area 40.

[0075] (Third embodiment) In the third embodiment, the method for selecting and determining character information differs from that of the first embodiment. The differences from the first embodiment will be explained below.

[0076] Figure 9 is a plan view of the display 14 for illustrating the character input operation in the information processing system 1 of the third embodiment. The first character information group includes the character information of five characters: "a", "ka", "sa", "ta", and "na". The second character information group includes the character information of five characters: "ha", "ma", "ya", "ra", and "wa". Each character included in the first character information group is associated with multiple characters in the row of that character. That is, the character "a" is associated with the other characters in the "a" row: "i", "u", "e", and "o". The character "ka" is associated with the other characters in the "ka" row: "ki", "ku", "ke", and "ko". The same applies to the character "sa", etc.

[0077] The first display area 40_1 is associated with the first character information group, and the character "あ" which represents the first character information group is displayed. As shown in Figure 9, when the first display area 40_1 is selected by the fingertip 50, the setting unit 32 sets the five judgment areas 44 to intersect with the straight line L1, and associates the five characters of the first character information group with each of the five judgment areas 44.

[0078] In Figure 9, the first determination area 44_1 corresponds to the character "あ", the second determination area 44_2 corresponds to the character "か", the third determination area 44_3 corresponds to the character "さ", the fourth determination area 44_4 corresponds to the character "た", and the fifth determination area 44_5 corresponds to the character "な".

[0079] Although not shown in the diagram, the sixth display area within the display surface 15 is associated with the second character information group, and the character "は" which represents the second character information group is displayed. When the sixth display area is specified by the fingertip 50, the setting unit 32 sets five determination areas 44 and associates the five characters of the second character information group with each of the five determination areas 44.

[0080] After multiple determination areas 44 have been set, the determination unit 30 determines whether or not the fingertip 50 is located within any of the determination areas 44.

[0081] When the determination unit 30 determines that the fingertip 50 is located within the j-th determination area 44_j, the display output unit 36 ​​displays the character information associated with the j-th determination area 44_j in the first display area 40_1, and also displays four character pieces associated with the character information displayed in the first display area 40_1, one character each, in the second display area 40_2, third display area 40_3, fourth display area 40_4, and fifth display area 40_5.

[0082] The second display area 40_2, the third display area 40_3, the fourth display area 40_4, and the fifth display area 40_5 are all adjacent to the first display area 40_1. The second display area 40_2 is located to the left of the first display area 40_1. The third display area 40_3 is located above the first display area 40_1. The fourth display area 40_4 is located to the right of the first display area 40_1. The fifth display area 40_5 is located below the first display area 40_1.

[0083] In Figure 9, since the fingertip 50 is located within the first judgment area 44_1, the character "あ" is displayed in the first display area 40_1, the character "い" is displayed in the second display area 40_2, the character "う" is displayed in the third display area 40_3, the character "え" is displayed in the fourth display area 40_4, and the character "お" is displayed in the fifth display area 40_5.

[0084] Figure 10 is a plan view of the display 14, following Figure 9, to explain the character input operation. In Figure 10, judgment areas 44 other than the second judgment area 44_2 are omitted. In Figure 10, since the fingertip 50 is located within the second judgment area 44_2, the character "ka" is displayed in the first display area 40_1, the character "ki" is displayed in the second display area 40_2, the character "ku" is displayed in the third display area 40_3, the character "ke" is displayed in the fourth display area 40_4, and the character "ko" is displayed in the fifth display area 40_5.

[0085] In this way, the user can display relevant, inputtable character information around the character information selected by the fingertip movement 50 times.

[0086] The determination unit 30 determines whether a first decision action or a second decision action has been performed by the fingertip 50. If the determination unit 30 determines that the fingertip 50 is located in the j-th determination area 44_j and that a first decision action has been performed, the display output unit 36 ​​displays the character information associated with the j-th determination area 44_j in the input display area 42. In other words, the character information of the first character information group can be input by the first decision action. The first decision action may be the same as the decision action in the first embodiment.

[0087] The display output unit 36 ​​displays the character information targeted by the second decision operation in the input display area 42 when the determination unit 30 determines that the fingertip 50 has selected one of the display areas 40 surrounding the first display area 40_1 and that a second decision operation has been performed. The second decision operation is a decision operation on character information in the display areas 40 surrounding the first display area 40_1, and is, for example, an operation in which the fingertip 50 is moved from the first display area 40_1 toward one of the surrounding display areas 40 at a predetermined speed or faster. This operation can also be called a flick operation in the air.

[0088] The second decision action may be the same as the first decision action. In other words, the first decision action may be performed while the surrounding display area 40 is specified with the fingertip 50.

[0089] In Figure 10, a flicking motion is performed in the air by the fingertip 50, and since the fingertip 50 has selected the third display area 40_3, the character "く" displayed in the third display area 40_3 is input.

[0090] Thus, character information can be entered using a character input operation different from that of the first embodiment.

[0091] Alternatively, the first character information group and the second character information group may be combined into a single character information group. In this case, when the first display area 40_1 is specified by the fingertip 50, the setting unit 32 sets up 10 judgment areas 44 and associates the 10 characters of the character information group with each of the 10 judgment areas 44.

[0092] Figure 11 is a flowchart showing the processing of the information processing system 1 in the third embodiment. The processing from S10 to S18 is the same as in the first embodiment in Figure 5. In S18, if the fingertip 50 is located within any of the determination areas 44 (Y in S18), the display output unit 36 ​​displays the character information associated with the identified j-th determination area 44_j in the i-th display area 40_i, and displays the character information associated with the said character information around the i-th display area 40_i (S40).

[0093] The determination unit 30 determines whether a first decision operation has occurred in the j-th determination area 44_j (S42). If a first decision operation has occurred (Y in S42), the function execution unit 34 determines the character information associated with the j-th determination area 44_j (S44). The determination unit 30 determines whether the fingertip 50 has moved out of the selection area R1 (S50). The selection area R1 is set in the same way as in the first embodiment. If the fingertip 50 has not moved out of the selection area R1 (N in S50), the determination unit 30 returns to the process in S18.

[0094] If the fingertip 50 moves out of the selection area R1 (Y in S50), the display output unit 36 ​​displays the character information representing the i-th character information group in the i-th display area 40_i and hides the character information surrounding the i-th display area 40_i (S52), and the processing of the information processing system 1 returns to the processing of S10 by the determination unit 30.

[0095] In S18, if the fingertip 50 is not located within any of the determination areas 44 (S18, N), the determination unit 30 proceeds to the process of S50. In S42, if there is no decision operation in the j-th determination area 44_j (S42, N), the determination unit 30 determines whether there is a second decision operation for the character information surrounding the i-th display area 40_i (S46). If there is a second decision operation (S46, Y), the function execution unit 34 determines the character information that is the target of the second decision operation (S48), and the processing of the information processing system 1 returns to the process of S50 by the determination unit 30. If there is no second decision operation in S46 (S46, N), the determination unit 30 proceeds to the process of S50.

[0096] (Fourth embodiment) The fourth embodiment differs from the first embodiment in that characters can be entered using the character input actions of each of the five fingers. The following will focus on explaining the differences from the first embodiment.

[0097] Figure 12 is a perspective view of a display 14 illustrating the character input operation in the information processing system 1 of the fourth embodiment. A predetermined input space 48 is set on the display surface 15 of the display 14. When the user starts character input, they hold the five fingertips of one hand in front of the display surface 15 and place them in the input space 48.

[0098] The coordinate acquisition unit 26 acquires the spatial coordinates of the first fingertip 50_1, second fingertip 50_2, third fingertip 50_3, fourth fingertip 50_4, and fifth fingertip 50_5 of the hand located in front of the display surface 15. For example, the first fingertip 50_1 is the tip of the thumb, the second fingertip 50_2 is the tip of the index finger, and the third fingertip 50_3 is the tip of the middle finger. The fourth fingertip 50_4 is the tip of the ring finger, and the fifth fingertip 50_5 is the tip of the little finger. Hereafter, when the first fingertip 50_1 to the fifth fingertip 50_5 are not distinguished, they will be referred to as fingertip 50. The five fingertips 50 are just one example of five detection targets used for operation input, and the five detection targets are not limited to the five fingertips 50.

[0099] The determination unit 30 determines whether the five fingertips 50 have entered the input space 48 based on the spatial coordinates of each of the five fingertips 50. The entry of the five fingertips 50 into the input space 48 corresponds to the fulfillment of a predetermined input start condition. The determination unit 30 outputs the determination result to the display output unit 36. The determination unit 30 outputs the determination result and the spatial coordinates of each of the five fingertips 50 at the reference time, which is the time when the input start condition was fulfilled, to the setting unit 32.

[0100] Figure 13 is a diagram illustrating the text input operation, following Figure 12. Figure 13(a) is a plan view of the display 14, and Figure 13(b) is a top view of the display 14.

[0101] When the determination unit 30 determines that the input start condition has been met, the display output unit 36 ​​displays an image on the display surface 15 that includes a first display area 40_1, a second display area 40_2, a third display area 40_3, a fourth display area 40_4, and a fifth display area 40_5, based on the spatial coordinates of each of the five fingertips 50 at the reference time. Let k be an integer from 1 to 5. For example, the kth display area 40_k intersects the straight line that is the normal to the display surface 15, which connects the spatial coordinates of the kth fingertips 50_k at the reference time to the display surface 15.

[0102] The display output unit 36 ​​displays the character information representing the kth character information group, which is information related to the kth character information group, in the kth display area 40_k in the initial image.

[0103] As shown in Figure 13(a), from the user's perspective, the character "あ" is superimposed on the first fingertip 50_1, the character "か" is superimposed on the second fingertip 50_2, the character "さ" is superimposed on the third fingertip 50_3, the character "た" is superimposed on the fourth fingertip 50_4, and the character "な" is superimposed on the fifth fingertip 50_5. Therefore, the user can easily understand which fingertip 50 to move to input characters from which row.

[0104] The setting unit 32 sets up multiple determination areas 44 between the kth display area 40_k and the kth fingertip 50_k based on the coordinates of the kth display area 40_k and the spatial coordinates of the kth fingertip 50_k at the reference time.

[0105] The setting unit 32 sets the kth line Lk for operation, which connects the spatial coordinates of the kth fingertip 50_k at the reference time with the kth display area 40_k, and sets five judgment areas 44 such that the five judgment areas 44 intersect the kth line Lk.

[0106] The setting unit 32 associates the 25 character information items included in the first to fifth character information groups with the 25 determination areas 44 on a one-to-one basis.

[0107] As shown in Figure 13(b), the first line L1 intersects with the first determination region 44_1 to the fifth determination region 44_5. The second line L2 intersects with the sixth determination region 44_6 to the tenth determination region 44_10. The third line L3 intersects with the eleventh determination region 44_11 to the fifteenth determination region 44_15. The fourth line L4 intersects with the sixteenth determination region 44_16 to the twentieth determination region 44_20. The fifth line L5 intersects with the twenty-first determination region 44_21 to the twenty-fifth determination region 44_25.

[0108] After multiple determination areas 44 are set, the determination unit 30 determines whether the first fingertip 50_1 is located in any of the first determination area 44_1 to the fifth determination area 44_5, and whether the second fingertip 50_2 is located in any of the sixth determination area 44_6 to the tenth determination area 44_10. After multiple determination areas 44 are set, the determination unit 30 determines whether the third fingertip 50_3 is located in any of the eleventh determination area 44_11 to the fifteenth determination area 44_15, whether the fourth fingertip 50_4 is located in any of the sixteenth determination area 44_16 to the twentieth determination area 44_20, and whether the fifth fingertip 50_5 is located in any of the twenty-first determination area 44_21 to the twenty-fifth determination area 44_25.

[0109] Let m be an integer between 1 and 5. If the determination unit 30 determines that the m-th fingertip 50_m is located within the j-th determination area 44_j, the display output unit 36 ​​displays the character information associated with the j-th determination area 44_j in the m-th display area 40_m.

[0110] The determination unit 30 determines whether a decision action was performed by the m-th fingertip 50_m in the j-th determination area 44_j.

[0111] When the determination unit 30 determines that the m-th fingertip 50_m is located in the j-th determination area 44_j and that a determination operation has been performed, the function execution unit 34 determines the character information associated with the j-th determination area 44_j and accepts input of the character information.

[0112] If the determination unit 30 determines that the m-th fingertip 50_m is located in the j-th determination area 44_j and that a determination operation has been performed, the display output unit 36 ​​displays the character information associated with the j-th determination area 44_j in the input display area 42.

[0113] The user can input the selected character by bringing their fingertip 50 close to the display area 40 corresponding to the row containing the character they want to input, selecting the character, and then confirming the selection with their fingertip 50.

[0114] Furthermore, the determination unit 30 determines whether a predetermined character switching operation has been performed in the input space 48 using at least one fingertip 50. If the determination unit 30 determines that a character switching operation has been performed, the display output unit 36 ​​displays information about the (k+5)th character information group in the kth display area 40_k in the initial image. The characters "ha", "ma", "ya", "ra", and "wa" are displayed on the display surface 15.

[0115] If the determination unit 30 determines that a character switching operation has been performed, the setting unit 32 associates one-to-one with multiple character information included in the 6th character information group to the 10th character information group and multiple determination areas 44.

[0116] Figure 14 is a flowchart showing the processing of the information processing system 1 according to the fourth embodiment. The determination unit 30 determines whether or not the five fingertips 50 are located in the input space 48 (S60). If the five fingertips 50 are not located in the input space 48 (N in S60), the determination unit 30 returns to the processing of S60.

[0117] When the five fingertips 50 are located in the input space 48 (Y in S60), the display output unit 36 ​​displays character information representing each character information group (S62). The setting unit 32 assigns each fingertip 50 to a character information group (S64) and sets the judgment area 44 for each fingertip 50 (S66).

[0118] The determination unit 30 determines whether any fingertip 50 is located within any of the determination areas 44 (S68). If this condition is met (Y in S68), the display output unit 36 ​​displays the character information associated with the identified j-th determination area 44_j in the m-th display area 40_m (S70).

[0119] The determination unit 30 determines whether a decision operation has occurred in the j-th determination area 44_j (S72). If a decision operation has occurred (Y in S72), the function execution unit 34 determines the character information associated with the j-th determination area 44_j (S74). The determination unit 30 determines whether the five fingertips 50 have left the input space 48 (S76). If the five fingertips 50 have not left the input space 48 (N in S76), the determination unit 30 returns to the process in S68.

[0120] If the five fingertips 50 move outside the input space 48 (Y in S76), the display output unit 36 ​​hides the character information representing each character information group (S78), and the processing of the information processing system 1 returns to the processing in S60 by the determination unit 30.

[0121] If, in S68, any fingertip 50 is not located within any of the determination areas 44 (N in S68), the determination unit 30 proceeds to the process in S76. If, in S72, there is no determination operation in the j-th determination area 44_j (N in S72), the determination unit 30 proceeds to the process in S76.

[0122] According to this embodiment, character information can be entered using five fingers.

[0123] The multiple display areas 40 may be displayed at predetermined positions on the display surface 15, regardless of the spatial coordinates of each of the five fingertips 50 at the reference time.

[0124] Furthermore, similar to the second embodiment, multiple determination areas 44 may be set for each fingertip 50 along the normal of the display surface 15. In other words, the setting unit 32 may set five determination areas 44 such that they intersect the k-th line Lk, which is the normal of the display surface 15 and is an operation line Lk connecting the k-th fingertip 50_k at the reference time and the display surface 15. The user can select character information by moving each fingertip 50 along the direction of the normal of the display surface 15.

[0125] (Fifth embodiment) The fifth embodiment differs from the first embodiment in that it allows the selection and input of words that begin with the entered characters. The differences from the first embodiment will be explained below.

[0126] Figure 15 is a plan view of the display 14 for illustrating the character input operation in the information processing system 1 of the fifth embodiment. First, the user performs a character input operation with their fingertip 50, as in the first embodiment. Here, we assume that the character "あ" has been entered.

[0127] Figure 16 is a plan view of the display 14, following Figure 15, to illustrate the character input operation. The memory unit 22 stores multiple word information. When the function execution unit 34 receives character input, it selects a word that starts with the received character from the word information held in the memory unit 22 and generates image data to display the selected word in the word display area 60. For example, as shown in Figure 15, the word "asa" is displayed in the word display area 60.

[0128] When a user wants to input a word, they use their fingertip 50 to specify the word display area 60. If the determination unit 30 determines that the fingertip 50 is specifying the word display area 60, the setting unit 32 sets multiple word determination areas 44 between the word display area 60 and the fingertip 50 based on the coordinates of the word display area 60 and the spatial coordinates of the fingertip 50 at a reference time supplied by the determination unit 30.

[0129] The setting unit 32 selects multiple words from the word information stored in the memory unit 22 that begin with the received character, and associates the selected multiple words with multiple determination areas 44 on a one-to-one basis. The selected multiple words constitute a character information group.

[0130] Figure 17 is a plan view of the display 14, following Figure 16, to illustrate the character input operation. The user brings their fingertip 50 close to the word display area 60, and the fingertip 50 is in the second judgment area 44_2. The word "aki" is associated with the second judgment area 44_2.

[0131] If the determination unit 30 determines that the fingertip 50 is located within the j-th determination area 44_j, the display output unit 36 ​​displays the word associated with the j-th determination area 44_j in the word display area 60. In the second determination area 44_2, the word "aki" is displayed instead of the word "asa".

[0132] When the user sees the word they want to input, "aki," displayed in the word display area 60, that is, when their fingertip 50 is positioned in the second determination area 44_2, they perform a decision action with their fingertip 50. In response, the function execution unit 34 accepts the input of the word "aki," which is the character information associated with the second determination area 44_2, and the display output unit 36 ​​displays the word "aki" in the input display area 42. According to this embodiment, a word can be input.

[0133] Multiple words may be displayed in the word display area 60. Figure 18 is a plan view of the display 14 to illustrate another example of character input operation in the information processing system 1 of the fifth embodiment.

[0134] When the function execution unit 34 receives input character information, it selects multiple words starting with the received character from the word information stored in the storage unit 22, and generates image data to display the selected multiple words in the word display area 60. For example, as shown in Figure 18, the three words "asa," "aki," and "ame" are displayed in the word display area 60.

[0135] If the determination unit 30 determines that the fingertip 50 is pointing to a word display area 60, the setting unit 32 sets multiple word determination areas 44 between the word display area 60 and the fingertip 50.

[0136] The setting unit 32 selects multiple words from the word information stored in the memory unit 22 that begin with the received character, and associates the selected multiple words with multiple determination areas 44 on a multiple-to-one basis. For example, the setting unit 32 associates three words with one determination area 44. The selected multiple words constitute a character information group. Each of the multiple character information items included in the character information group contains multiple words that begin with a predetermined character.

[0137] Figure 19 is a plan view of the display 14, following Figure 18, to illustrate the character input operation. The user brings their fingertip 50 close to the word display area 60, and the fingertip 50 is in the second judgment area 44_2. The second judgment area 44_2 is associated with three words as character information: "ari", "aji", and "asu".

[0138] If the determination unit 30 determines that the fingertip 50 is located within the j-th determination area 44_j, the display output unit 36 ​​displays multiple words associated with the j-th determination area 44_j in the word display area 60. In the second determination area 44_2, the three words "ari", "aji", and "asu" are displayed instead of the three words "asa", "aki", and "ame".

[0139] When the word "tomorrow" that the user wants to input is displayed in the word display area 60, that is, when the fingertip 50 is located in the second determination area 44_2, the user makes a decision action with their fingertip 50 for the word they want to input. The decision action for the word to input is, for example, to point the fingertip 50 at the word displayed in the word display area 60 and perform the decision action of the first embodiment. In response, the function execution unit 34 accepts the input of the word "tomorrow," which is the character information that is the target of the decision action, and the display output unit 36 ​​displays the word "tomorrow" in the input display area 42. In this example, multiple words are displayed, making it easy to select a word.

[0140] Figure 20 is a flowchart showing the processing of the information processing system 1 in the fifth embodiment. Figure 21 is a flowchart following Figure 20. The determination unit 30 determines whether the character display area 40 lies on a straight line passing through the eye 52 and the fingertip 50 (S80). If the character display area 40 lies on the straight line (Y in S80), the determination unit 30 proceeds to the process in S12. The processes from S12 to S24 are the same as in the first embodiment in Figure 5. After S24, the display output unit 36 ​​displays multiple words starting with the determined character (S82), and the determination unit 30 proceeds to the process in S26. The processes in S26 and S28 are the same as in the first embodiment in Figure 5.

[0141] In S80, if the character display area 40 is not on the straight line passing through the eye 52 and the fingertip 50 (N in S80), the determination unit 30 determines whether the word display area 60 is on the straight line passing through the eye 52 and the fingertip 50 (S84). If the word display area 60 is not on the straight line (N in S84), the determination unit 30 returns to the process in S10. If the word display area 60 is on the straight line (Y in S84), the setting unit 32 sets the determination area 44 and the selection area R1 on the operation line L1 (S86).

[0142] The determination unit 30 determines whether the fingertip 50 is located within any of the determination areas 44 (S88). If the fingertip 50 is located within any of the determination areas 44 (Y in S88), the display output unit 36 ​​displays multiple words associated with the identified j-th determination area 44_j in the word display area 60 (S90).

[0143] The determination unit 30 determines whether there is a decision action for any of the words (S92). If there is a decision action (Y in S92), the function execution unit 34 determines the word that is the target of the decision action (S94). The display output unit 36 ​​hides multiple words (S98), and the processing of the information processing system 1 returns to the processing of S80 by the determination unit 30.

[0144] If the fingertip 50 is not located within any of the determination areas 44 in S88 (N in S88), or if there is no decision operation in S92 (N in S92), the determination unit 30 determines whether the fingertip 50 has moved out of the selection area R1 (S96). If the fingertip 50 has not moved out of the selection area R1 (N in S96), the determination unit 30 proceeds to the processing of S88. If the fingertip 50 has moved out of the selection area R1 (Y in S96), the display output unit 36 ​​executes the processing of S98.

[0145] (Sixth embodiment) The sixth embodiment differs from the first embodiment in that it accepts input of letters or numbers. The following description will focus on the differences from the first embodiment.

[0146] Figure 22 is a diagram illustrating a first example of character input operation in the information processing system 1 of the sixth embodiment. Figure 22(a) is a plan view of the display 14, and Figure 22(b) is a side view of the display 14.

[0147] The control device 12 accepts alphabetical input. Multiple character information groups contain different alphabetical characters. Each character information group contains both uppercase and lowercase letters of the same alphabetical character.

[0148] The first character information group associated with the first display area 40_1 includes the characters "a" and "A". The second character information group associated with the second display area 40_2 includes the characters "b" and "B". The third character information group associated with the third display area 40_3 includes the characters "c" and "C". The same applies to the other character information groups associated with each of the other multiple display areas 40 that are not shown.

[0149] Let's explain an example of a user inputting the character "A". As shown in Figure 22(a), first, the user uses their fingertip 50 to specify the first display area 40_1 where the character "a" is displayed. The setting unit 32 sets the first determination area 44_1 associated with the character "a" and the second determination area 44_2 associated with the character "A".

[0150] Next, the user moves their fingertip 50 closer to the first display area 40_1 until the character "a" displayed in the first display area 40_1 changes to the character "A". When the fingertip 50 enters the second judgment area 44_2 and the character "A" is displayed in the first display area 40_1, the user makes a decision with their fingertip 50. As a result, the character "A" is displayed in an input display area (not shown) and input to the control device 12. The user can input multiple characters by repeating these actions.

[0151] Figure 23 is a diagram illustrating a second example of character input operation in the information processing system 1 of the sixth embodiment. Figure 23(a) is a plan view of the display 14, and Figure 23(b) is a side view of the display 14.

[0152] Each of the multiple character information groups contains multiple uppercase letters of the alphabet. The first character information group associated with the first display area 40_1 contains the letters "A", "B", and "C". The second character information group associated with the second display area 40_2 contains the letters "D", "E", and "F". The third character information group associated with the third display area 40_3 contains the letters "G", "H", and "I". The characters in the other character information groups associated with each of the other multiple display areas 40 are similar, although this explanation is omitted here, as shown in Figure 23(a).

[0153] In the initial image, the first display area 40_1 displays multiple characters "ABC" that belong to the first character information group. The second display area 40_2 displays multiple characters "DEF" that belong to the second character information group. The third display area 40_3 displays multiple characters "GHI" that belong to the third character information group. As shown in Figure 23(a), although the explanation is omitted here, multiple characters that belong to the corresponding character information group are also displayed in each of the other display areas 40. This makes it easy for the user to understand which characters are selectable in which display area 40.

[0154] Let's explain an example of a user inputting the character "A". As shown in Figure 23(a), first, the user uses their fingertip 50 to specify the first display area 40_1 where the characters "ABC" are displayed. The setting unit 32 sets up the first judgment area 44_1 associated with multiple characters "ABC", the second judgment area 44_2 associated with the character "A", the third judgment area 44_3 associated with the character "B", and the fourth judgment area 44_4 associated with the character "C".

[0155] Next, the user moves their fingertip 50 closer to the first display area 40_1 until the character "ABC" displayed in the first display area 40_1 changes to the character "A". When the fingertip 50 enters the second judgment area 44_2 and the display in the first display area 40_1 changes to the character "A", the user makes a decision with their fingertip 50. As a result, the character "A" is displayed in an input display area (not shown) and input to the control device 12. The user can input multiple characters by repeating these actions.

[0156] Furthermore, if the fingertip 50 is positioned in the first determination area 44_1 which is associated with multiple characters "ABC", the characters "ABC" will be displayed in the first display area 40_1. If a decision operation is performed while the fingertip 50 is positioned in the first determination area 44_1, the characters may or may not be input to the control device 12.

[0157] Furthermore, in the first example of character input operation in Figure 22 and the second example of character input operation in Figure 23, the system may be configured to allow selection and input of words starting with the entered character, similar to the fifth embodiment. The basic character input operation and processing are the same as in the fifth embodiment. For example, if the character "C" is entered first, one or more words starting with the character "C" will be displayed in the word display area, and the displayed words can be selected and input according to the movement of the fingertip 50.

[0158] Figure 24 is a plan view of the display 14 to illustrate a third example of character input operation in the information processing system 1 of the sixth embodiment. The basic character input operation and processing are the same as in the third embodiment.

[0159] The character information group contains character information for the six characters "A", "F", "K", "P", "U", and "Z". Each character in the character information group is associated with the four characters that follow it in alphabetical order. That is, the character "A" is associated with the four characters that follow "A", "B", "C", "D", and "E". The character "F" is associated with the four characters that follow "F", "G", "H", "I", and "J". The same applies to the characters "K", etc.

[0160] The first display area 40_1 is associated with the above character information group, and in the initial state, the character "A" representing the character information group is displayed. When the first display area 40_1 is selected by the fingertip 50, the setting unit 32 sets six judgment areas 44 and associates the six characters of the character information group with the six judgment areas 44. As shown in Figure 24, for example, when the second display area 40_2 is selected by the fingertip 50, the character "F" is displayed in the first display area 40_1, and the characters "G", "H", "I", and "J" are displayed around it. Alphabet characters can be entered using the same character input operation as in the third embodiment.

[0161] Figure 25 is a plan view of the display 14 to illustrate a fourth example of character input operation in the information processing system 1 of the sixth embodiment. The basic character input operation and processing are the same as in the fourth embodiment. In this example, a QWERTY keyboard is simulated. From the user's perspective, the character "Q" is superimposed on the first fingertip 50_1, the character "W" is superimposed on the second fingertip 50_2, the character "E" is superimposed on the third fingertip 50_3, the character "R" is superimposed on the fourth fingertip 50_4, and the character "T" is superimposed on the fifth fingertip 50_5.

[0162] The setting unit 32 sets, for example, three determination areas 44 between the kth display area 40_k and the kth fingertip 50_k. Although not shown in the figure, between the first display area 40_1 and the first fingertip 50_1, for example, a first determination area 44_1 corresponding to the character "Q", a second determination area 44_2 corresponding to the character "A", and a third determination area 44_3 corresponding to the character "Z" are set. Between the second display area 40_2 and the second fingertip 50_2, for example, a fourth determination area 44_4 corresponding to the character "W", a fifth determination area 44_5 corresponding to the character "S", and a sixth determination area 44_6 corresponding to the character "X" are set.

[0163] The user can input the selected character by bringing their fingertip 50 close to the display area 40 corresponding to the character they want to input, selecting the character, and then confirming the selection with their fingertip 50.

[0164] Figure 26 is a diagram illustrating a fifth example of character input operation in the information processing system 1 of the sixth embodiment. Figure 26(a) is a plan view of the display 14, and Figure 26(b) is a side view of the display 14.

[0165] The control device 12 accepts numerical input. In Figure 26(a), the display surface 15 displays, for example, a four-character password input screen. The four display areas 40 each function as input display areas that display the entered numbers.

[0166] Each of the four display areas 40 is associated with the same character information group. The character information group contains information about multiple numbers. For example, the character information group contains information about integers from 0 to 9.

[0167] The user, without touching the display 14, uses their fingertip 50 to specify, for example, the first display area 40_1. The setting unit 32 sets the first determination area 44_1 associated with the number "0", the second determination area 44_2 associated with the number "1", the third determination area 44_3 associated with the number "2", and so on.

[0168] Next, the user moves their fingertip 50 closer to the first display area 40_1 until the number displayed in the first display area 40_1 changes to the number they want to input. When the number they want to input is displayed in the first display area 40_1, the user makes a confirmation action with their fingertip 50. This inputs the number they want to input into the control device 12. The user can input four numbers by performing these actions in each of the four display areas 40.

[0169] (Various examples of the determination area 44 in the first to sixth embodiments) Figure 27 shows various setting examples for the judgment area 44 in the information processing system 1. Figure 27(a) shows a first setting example for the judgment area 44. Regardless of the angle between the line L1 and the display surface 15, the multiple judgment areas 44 are parallel to the display surface 15. Regardless of the angle between the line L1 and the display surface 15, the spacing s1 between two adjacent judgment areas 44 along the normal of the display surface 15 is equal. For example, the spacing s1 between the first judgment area 44_1 and the second judgment area 44_2 along the normal of the display surface 15 is equal to the spacing s1 between the second judgment area 44_2 and the third judgment area 44_3 along the normal of the display surface 15.

[0170] Figure 27(b) shows a second example of setting the judgment area 44. Regardless of the angle between the line L1 and the display surface 15, the multiple judgment areas 44 are perpendicular to the line L1. Regardless of the angle between the line L1 and the display surface 15, the distance s2 between two adjacent judgment areas 44 along the line L1 is equal. For example, the distance s2 between the first judgment area 44_1 and the second judgment area 44_2 along the line L1 is equal to the distance s2 between the second judgment area 44_2 and the third judgment area 44_3 along the line L1. In this example, regardless of the direction in which the fingertip 50 is moved, the distance the fingertip 50 moves between two adjacent judgment areas 44 can be made equal.

[0171] Figure 28 shows another example of setting the determination area 44 in the information processing system 1. Figure 28(a) shows a third example of setting the determination area 44. The setting unit 32 arranges multiple determination areas 44 adjacent to each other along the straight line L1.

[0172] The setting unit 32 sets the length of the judgment area 44 along the line L1 to be longer the further the judgment area 44 is from the position of the fingertip 50 at the reference time. For example, the length a2 of the second judgment area 44_2 along the line L1 is longer than the length a1 of the first judgment area 44_1 along the line L1. Also, the volume of the second judgment area 44_2 is larger than the volume of the first judgment area 44_1.

[0173] As the arm is extended and the fingertip 50 moves further away from the user's body, the position of the fingertip 50 tends to fluctuate more, but even in that situation, it is still easy to select the judgment area 44. Therefore, it is easy to select the judgment area 44 that is close to the display surface 15.

[0174] Figure 28(b) shows a fourth example of setting the judgment area 44. In the examples shown in Figures 28(b) to 28(d), the letters are numbers. The setting unit 32 sets the distance between two adjacent judgment areas 44 along the straight line L1 to be wider the further away the fingertip 50 is from the position at the reference time.

[0175] The interval s2 along the line L1 between the second determination region 44_2 and the third determination region 44_3 is wider than the interval s1 along the line L1 between the first determination region 44_1 and the second determination region 44_2. The interval s3 along the line L1 between the third determination region 44_3 and the fourth determination region 44_4 is wider than the interval s2. The interval s4 along the line L1 between the fourth determination region 44_4 and the fifth determination region 44_5 is wider than the interval s3.

[0176] The first character information corresponding to the first determination area 44_1 is a number obtained by raising a predetermined number to the power of the first exponent. For example, the predetermined number is "10" and the first exponent is "0". The second character information corresponding to the second determination area 44_2 is a number obtained by raising a predetermined number to the power of the second exponent, which is greater than the first exponent. For example, the second exponent is "1".

[0177] The third character information corresponding to the third judgment area 44_3 is a number obtained by raising a predetermined number to the power of a third exponent greater than the second exponent. For example, the third exponent is "2". The fourth character information corresponding to the fourth judgment area 44_4 is a number obtained by raising a predetermined number to the power of a fourth exponent greater than the third exponent. For example, the fourth exponent is "3". The fifth character information corresponding to the fifth judgment area 44_5 is a number obtained by raising a predetermined number to the power of a fifth exponent greater than the fourth exponent. For example, the fifth exponent is "4".

[0178] In this example, you can select numbers that change with exponents. Selecting larger numbers requires moving your fingertip 50 a longer distance, making the selection process intuitive.

[0179] Figure 28(c) shows an example of setting the judgment area when the movement speed of the fingertip 50 is relatively fast. Figure 28(d) shows an example of setting the judgment area when the movement speed of the fingertip 50 is relatively slow. The calculation unit 28 periodically calculates the movement speed of the fingertip 50 and outputs the calculated movement speed to the judgment unit 30. The judgment unit 30 compares the movement speed of the fingertip 50 supplied from the calculation unit 28 with a speed threshold and outputs the comparison result to the setting unit 32. One or more speed thresholds are set in advance.

[0180] Based on the comparison results supplied by the determination unit 30, the setting unit 32 sets the spacing between multiple determination areas 44 along the straight line L1 wider and reduces the number of determination areas 44, as the faster the fingertip 50 moves after a predetermined time from the reference time. The predetermined time can be appropriately determined by experiment or simulation. This makes it easier to select a determination area 44 when the fingertip 50 moves at a high speed.

[0181] The setting unit 32 associates a larger number with the judgment area 44 that is further away from the position of the fingertip 50 at the reference time. The setting unit 32 sets a larger difference between the number associated with one of two adjacent judgment areas 44 and the number associated with the other judgment area 44, as the faster the fingertip 50 moves after a predetermined time from the reference time. As a result, when the fingertip 50 moves at a fast speed, a larger number can be selected than when the movement speed is slow.

[0182] Figure 29 shows yet another example of setting the determination area 44 in the information processing system 1. In Figure 29, the determination areas 44 other than the first determination area 44_1 and the second determination area 44_2 are not shown. Figure 29(a) shows the determination area 44 with a standard size. Figure 29(b) shows the determination area 44 with a changed size.

[0183] The setting unit 32 arranges multiple judgment areas 44 adjacent to each other along a straight line L1. When the fingertip 50 is located in the second judgment area 44_2, the setting unit 32 expands the second judgment area 44_2 toward the first judgment area 44_1 and narrows the first judgment area 44_1.

[0184] If the fingertip 50 then moves from the second determination area 44_2 to the first determination area 44_1, the setting unit 32 restores the sizes of the first determination area 44_1 and the second determination area 44_2 to their original sizes. In other words, the setting unit 32 restores the determination area 44 to the standard size shown in Figure 29(a).

[0185] In this example, when the fingertip 50 is located at the boundary between the standard-sized first judgment area 44_1 and the second judgment area 44_2, it is possible to suppress the frequent switching and instability of the characters displayed in the display area 40.

[0186] (Seventh Embodiment) The seventh embodiment differs from the first embodiment in that it allows for character selection without pointing the fingertip 50 at the display area 40. The following will focus on the differences from the first embodiment.

[0187] Figure 30 is a perspective view of the display 14 for illustrating the character input operation in the information processing system 1 of the seventh embodiment. A predetermined input space 48 is set at a position that does not overlap with the display surface 15 of the display 14. When the user starts character input, they place their fingertip 50 into the input space 48.

[0188] The determination unit 30 determines whether the fingertip 50 has entered the input space 48. The fact that the fingertip 50 has entered the input space 48 corresponds to the input start condition being met. The determination unit 30 outputs the determination result and the spatial coordinates of the fingertip 50 at the reference time, which is the time when the input start condition was met, to the setting unit 32.

[0189] Figure 31 is a diagram illustrating the character input operation, following Figure 30. The setting unit 32 sets multiple uppermost judgment areas 44 within a plane 60 parallel to the display surface 15, based on the spatial coordinates of the fingertip 50 at the reference time. The number of judgment areas 44 within the plane 60 is equal to the number of display areas 40, which in this example is "12". The setting unit 32 sets a first judgment area 44_1 that overlaps with the spatial coordinates of the fingertip 50 at the reference time, and sets multiple other judgment areas 44 in a matrix within the plane 60. The setting unit 32 sets the lower multiple judgment areas 44 in a direction parallel to the Z axis. For example, the first judgment area 44_1, second judgment area 44_2, third judgment area 44_3, fourth judgment area 44_4, and fifth judgment area 44_5 are set spaced apart in a direction parallel to the Z axis. Figure 31 shows a part of the lower judgment areas 44.

[0190] Multiple character information contained within multiple character information groups are pre-assigned one-to-one to multiple determination areas 44.

[0191] The user selects character information associated with one of the multiple determination areas 44 by moving their fingertip 50 within the surface 60 and in a direction parallel to the Z-axis.

[0192] After multiple determination areas 44 have been set, the determination unit 30 determines whether or not the fingertip 50 is located in any of the multiple determination areas 44.

[0193] When the determination unit 30 determines that the fingertip 50 is located within the j-th determination area 44_j, the display output unit 36 ​​displays the character information associated with the j-th determination area 44_j in the display area 40 associated with the character information group containing that character information. For example, if the fingertip 50 is located within the third determination area 44_3, the character "う" is replaced with the character "あ" and displayed in the first display area 40_1.

[0194] When the determination unit 30 determines that the fingertip 50 is located in the j-th determination area 44_j and that a determination operation has been performed, the function execution unit 34 determines the character information associated with the j-th determination area 44_j and accepts input of the character information.

[0195] If the determination unit 30 determines that the fingertip 50 is located in the j-th determination area 44_j and that a determination operation has been performed, the display output unit 36 ​​displays the character information associated with the j-th determination area 44_j in the input display area 42.

[0196] In this way, the first determination area 44_1 can be set at the user's desired position in the input space 48. Therefore, character information can be easily selected by aerial movements.

[0197] Figure 32 shows another example of setting up the multiple determination regions 44 in the seventh embodiment. Figure 32(a) is a diagram illustrating an example in which the multiple determination regions 44 are set up parallel to the XZ plane.

[0198] The setting unit 32 sets multiple uppermost judgment regions 44 within a plane 62 that intersects the display surface 15 and is parallel to the XZ plane. The setting unit 32 sets multiple lower judgment regions 44 in a direction parallel to the Y axis. For example, the first judgment region 44_1, the second judgment region 44_2, the third judgment region 44_3, the fourth judgment region 44_4, and the fifth judgment region 44_5 are set spaced apart in a direction parallel to the Y axis. Figure 32 shows a portion of the uppermost and lower judgment regions 44.

[0199] Figure 32(b) illustrates an example in which multiple determination regions 44 are set non-parallel to the XY plane, XZ plane, and ZY plane. The display surface 15 is not shown in Figure 32(b).

[0200] The setting unit 32 sets multiple uppermost judgment regions 44 within the surface 64 that intersects the display surface 15, the XZ plane, and the ZY plane. The setting unit 32 sets multiple lower judgment regions 44 in the direction of the normal to the surface 64. For example, the first judgment region 44_1, the second judgment region 44_2, the third judgment region 44_3, the fourth judgment region 44_4, and the fifth judgment region 44_5 are set spaced apart in the direction of the normal to the surface 64.

[0201] In the example shown in Figure 32, character information can be selected by moving the fingertip 50 within the surface 62 that intersects the display surface 15 and in the direction of the normal to surface 62.

[0202] Furthermore, when the fingertip 50 enters the input space 48, the setting unit 32 may set multiple determination areas 44 to predetermined positions without using the position of the fingertip 50 as a reference.

[0203] Figure 33 is a diagram illustrating the resizing process of the determination region 44 in the information processing system 1 of the seventh embodiment. In Figure 33, the first determination region 44_1, the sixth determination region 44_6, and the eleventh determination region 44_11, which are adjacent in the X-axis direction, are shown among the multiple determination regions 44 in the plane 60 of Figure 31.

[0204] Figure 33(a) shows the standard size determination area 44. Figure 33(b) shows the determination area 44 with a changed size, following Figure 33(a). Figure 33(c) shows the determination area 44 with a changed size, following Figure 33(b). Figure 33(d) shows the determination area 44 with a changed size, following Figure 33(c).

[0205] As shown in Figures 33(a) and 33(b), when the fingertip 50 is located in the first determination area 44_1, the setting unit 32 widens the first determination area 44_1 toward the sixth determination area 44_6 and the opposite side of the sixth determination area 44_6, and narrows the sixth determination area 44_6.

[0206] If the fingertip 50 then moves from the expanded first determination area 44_1 to the narrowed sixth determination area 44_6, the setting unit 32 expands the sixth determination area 44_6 towards the first determination area 44_1 and the eleventh determination area 44_11, and narrows the first determination area 44_1.

[0207] The setting unit 32 executes the above process if the fingertip 50 is located in any of the multiple determination areas 44.

[0208] In this example, incorrect input can be suppressed when the fingertip 50 is located at the boundary between the first judgment area 44_1 and the sixth judgment area 44_6 of the standard size.

[0209] Furthermore, when the fingertip 50 is located in the first determination area 44_1, the setting unit 32 may expand the first determination area 44_1 only toward the sixth determination area 44_6, expand the first determination area 44_1 vertically, or expand the first determination area 44_1 both vertically and horizontally.

[0210] The present disclosure has been described above based on embodiments. These embodiments are illustrative, and it will be understood by those skilled in the art that various modifications are possible for each component or combination of processing processes, and that such modifications are also within the scope of the present disclosure.

[0211] For example, the fifth embodiment may be combined with the third embodiment. The new embodiment resulting from this combination will possess the combined effects of each of the embodiments that are combined.

[0212] Furthermore, in each of the first, second, third, fourth, fifth, and sixth embodiments, the cross-sectional areas of the determination area 44 and the selection area R1 in a cross section parallel to the display surface 15 may be larger than the area of ​​the display area 40. Figure 34 is a diagram illustrating the character input operation in the information processing system 1 of a modified example of the first embodiment. Figure 34(a) is a plan view of the display 14, Figure 34(b) is a side view of the display 14, and Figure 34(c) is a top view of the display 14.

[0213] As shown in Figure 34, in a cross-section parallel to the display surface 15, both the judgment area 44 and the selection area R1 are longer than the display area 40 in the X-axis direction. Therefore, compared to the example in Figure 3, the fingertip 50 is less likely to deviate from the judgment area 44 and the selection area R1 in the left-right direction. Thus, even if the fingertip 50 unintentionally moves towards the second display area 40_2 due to, for example, indecision about which character to input, it is less likely to select the unintended second display area 40_2 compared to the example in Figure 3. Furthermore, even if the position of the fingertip 50 shifts left-right during the decision operation, input errors can be suppressed compared to the example in Figure 3.

[0214] Furthermore, in a cross-section parallel to the display surface 15, the judgment area 44 and the selection area R1 may each be longer than the display area 40 in the Y-axis direction. In this case, compared to the example in Figure 3, the fingertip 50 is less likely to deviate vertically from the judgment area 44 and the selection area R1. Also, in a cross-section parallel to the display surface 15, the judgment area 44 and the selection area R1 may each be longer than the display area 40 in the X-axis and Y-axis directions, respectively. In this case, compared to the example in Figure 3, the fingertip 50 is less likely to deviate vertically and horizontally from the judgment area 44 and the selection area R1.

[0215] One aspect of this disclosure is as follows:

[0216] [Item 1] A coordinate acquisition unit that acquires the spatial coordinates of the object to be detected located in front of the display surface of the display, A setting unit sets a first determination area associated with the first character information included in the first character information group, based on the coordinates of a first display area on the display surface where information about the first character information group is displayed, and the spatial coordinates of the object to be detected at a reference time which is the time when the first display area is designated by the object to be detected. After the first determination area is set, a determination unit determines whether or not the object to be detected is located within the first determination area, When the determination unit determines that the object to be detected is located within the first determination area, the display output unit displays the first character information in the first display area. A control device equipped with the following features. According to this embodiment, the judgment area related to the display area specified by the user can be set to a desired position. Therefore, regardless of the size of the display surface of the display, it is easy to select text information by moving the device in mid-air.

[0217] [Item 2] The setting unit sets the first determination area such that the first determination area intersects the straight line connecting the spatial coordinates of the object to be detected at the reference time and the first display area. The control device described in item 1. In this case, the first character information can be selected by bringing the object to be detected close to the first display area from any direction relative to the display surface.

[0218] [Item 3] The setting unit sets the first determination area such that it intersects with a straight line that connects the spatial coordinates of the object to be detected at the reference time and the display surface, and which is the normal to the display surface. The control device described in item 1. In this case, the first character of the information can be selected by bringing the object to be detected closer to the display surface along the direction of the normal to the display surface.

[0219] [Item 4] The setting unit sets the second determination region, which is associated with the second character information included in the first character information group, to intersect the straight line. After the second determination area is set, the determination unit determines whether or not the object to be detected is located within the second determination area. The display output unit, when it determines that the object to be detected is located within the second determination area, displays the second character information in the first display area. The control device described in item 2 or 3. In this case, by bringing the object to be detected closer to the display surface, the user can select either the first or second character of information.

[0220] [Item 5] The display output unit displays information related to the second character information group in the second display area of ​​the display surface. The determination unit determines whether the object to be detected is specifying either the first display area or the second display area, based on the spatial coordinates of the object to be detected, the coordinates of the first display area, and the coordinates of the second display area. The aforementioned setting unit is, If the determination unit determines that the object to be detected is specifying the first display area, it associates the first character information with the first determination area and sets the first determination area. If the determination unit determines that the object to be detected has designated the second display area, it associates the second character information included in the second character information group with the first determination area, and sets the first determination area based on the spatial coordinates of the object to be detected at the reference time, which is the time when the object to be detected designated the second display area. A control device as described in any of items 1 to 3. In this case, you can specify the desired group of character information.

[0221] [Item 6] The display output unit, when it determines that the object to be detected is located within the first determination area, displays the first character information in the first display area and displays the second character information associated with the first character information in the second display area adjacent to the first display area. The control device described in item 2. In this case, the related second character information can be displayed next to the selected first character information.

[0222] [Item 7] The setting unit sets the second determination region, which is associated with the third character information included in the first character information group, to intersect the straight line. After the second determination area is set, the determination unit determines whether or not the object to be detected is located within the second determination area. The display output unit, when it determines that the object to be detected is located within the second determination area, displays the third character information in the first display area and the fourth character information associated with the third character information in the second display area. The control device described in item 6. In this case, you can select either the first or third character information, and the related character information can be displayed next to the selected character information.

[0223] [Item 8] The display surface has an input display area where the entered characters are displayed. The determination unit determines whether a first decision operation or a second decision operation has been performed by the detected object, and whether the detected object has designated the second display area. The aforementioned display output unit is If the detection target object is located in the first determination area and the determination unit determines that the first determination operation has been performed, the first character information is displayed in the input display area. If the detection target object designates the second display area and the determination unit determines that the second determination operation has been performed, the second character information is displayed in the input display area. The control device described in item 6 or 7. In this case, you can enter either the first or second character of the information.

[0224] [Item 9] The first character information and the second character information each contain information about one or more words that begin with a predetermined character. The control device described in item 4. In this case, you can select a word.

[0225] [Item 10] The setting unit sets the third determination region, which is associated with the third character information included in the first character information group, to intersect the straight line. The first determination area, the second determination area, and the third determination area are arranged in order from the object to be detected to the display surface side, The distance between the first determination area and the second determination area along the straight line is equal to the distance between the second determination area and the third determination area along the straight line. The control device described in item 4. In this case, regardless of the position of the object being detected at the reference time, the distance traveled by the object between two adjacent determination areas can be made equal.

[0226] [Item 11] The setting unit sets the distance between the first determination area and the second determination area along the straight line to be wider the faster the movement speed of the object to be detected after a predetermined time from the reference time. The control device described in item 4. In this case, it is easier to select the detection area when the object being detected is moving at a high speed.

[0227] [Item 12] The first character information and the second character information are, respectively, numerical information. The setting unit sets the difference between the digits of the first character information and the digits of the second character information to be larger the faster the movement speed of the object to be detected after a predetermined time from the reference time. The control device described in item 11. In this case, the selectable numbers can be changed according to the movement speed of the object being detected.

[0228] [Item 13] The second determination area is located closer to the display surface than the first determination area. The volume of the second determination region is greater than the volume of the first determination region. The control device described in item 4. In this case, it is easier to select a judgment area that is close to the display surface.

[0229] [Item 14] The second determination area is located closer to the display surface than the first determination area. The cross-sectional area of ​​the second determination region in a section parallel to the display surface is greater than the cross-sectional area of ​​the first determination region in a section parallel to the display surface. The control device described in item 4. In this case, it is easier to select a judgment area that is close to the display surface.

[0230] [Item 15] The second determination area is located closer to the display surface than the first determination area. The length of the second determination region along the aforementioned straight line is longer than the length of the first determination region along the aforementioned straight line. The control device described in item 4. In this case, it is easier to select a judgment area that is close to the display surface.

[0231] [Item 16] The setting unit sets the third determination region, which is associated with the third character information included in the first character information group, to intersect the straight line. The first determination area, the second determination area, and the third determination area are arranged in order from the object to be detected to the display surface side, The distance between the second determination area and the third determination area along the straight line is wider than the distance between the first determination area and the second determination area along the straight line. The aforementioned first character information is a number obtained by raising a predetermined number to the power of the first exponent, The second character information is a number obtained by raising the predetermined number to the power of a second exponent that is greater than the first exponent. The third character information is a number obtained by raising the predetermined number to the power of a third exponent that is greater than the second exponent. The control device described in item 4. In this case, you can choose a number that changes with powers.

[0232] [Item 17] The first determination area and the second determination area are adjacent to each other along the straight line, The setting unit, when the object to be detected is located in the second determination area, expands the second determination area toward the first determination area and narrows the first determination area. The control device described in item 4. In this case, it is possible to suppress the instability of the display of text information when the object to be detected is located at the boundary between the first and second judgment areas.

[0233] [Item 18] The display surface has an input display area where the entered characters are displayed. The determination unit determines whether or not a decision action was performed based on the detected object. The display output unit, when it determines that the object to be detected is located in the first determination area and that a determination operation has been performed, displays the first character information in the input display area. A control device as described in any one of items 1 through 5 or 9 through 17. In this case, you can enter text information.

[0234] [Item 19] A coordinate acquisition unit that acquires the spatial coordinates of a first detection target and a second detection target located in front of the display surface of the display, A setting unit sets a first determination region associated with the first character information included in the first character information group based on the spatial coordinates of the first detection target at a reference time which is the time when a predetermined input start condition is met, and sets a second determination region associated with the second character information included in the second character information group based on the spatial coordinates of the second detection target at the reference time. After the first determination area and the second determination area are set, a determination unit determines whether the first object to be detected is located within the first determination area and whether the second object to be detected is located within the second determination area. When the determination unit determines that the first detection object is located in the first determination area, the display output unit displays the first character information in the first display area on the display surface, and when the determination unit determines that the second detection object is located in the second determination area, the display output unit displays the second character information in the second display area on the display surface. A control device comprising. According to this aspect, character information can be selected by a plurality of detection objects.

[0235] [Item 20] The setting unit sets the first determination area so that the first determination area intersects a first straight line connecting the spatial coordinates of the first detection object at the reference time and the first display area. sets the second determination area so that the second determination area intersects a second straight line connecting the spatial coordinates of the second detection object at the reference time and the second display area. The control device according to item 19. In this case, the first character information can be selected by approaching the first detection object to the first display area from an arbitrary direction with respect to the display surface. The second character information can be selected by approaching the second detection object to the second display area from an arbitrary direction with respect to the display surface.

[0236] [Item 21] The setting unit sets the first determination area so that the first determination area intersects a first straight line connecting the spatial coordinates of the first detection object at the reference time and the display surface, and the first straight line is a normal line of the display surface. sets the second determination area so that the second determination area intersects a second straight line connecting the spatial coordinates of the second detection object at the reference time and the display surface, and the second straight line is a normal line of the display surface. The control device according to item 19. In this case, the first character information can be selected by approaching the first detection object to the display surface along the normal direction of the display surface. The second character information can be selected by approaching the second detection object to the display surface along the normal direction of the display surface.

[0237] [Item 22] The setting unit sets a third determination area associated with third character information included in the first character information group so as to intersect the first straight line, sets a fourth determination area associated with fourth character information included in the second character information group so as to intersect the second straight line, The determination unit determines whether or not the first detection object is located within the third determination area after the third determination area is set, determines whether or not the second detection object is located within the fourth determination area after the fourth determination area is set, The display output unit causes the third character information to be displayed in the first display area when the determination unit determines that the first detection object is located within the third determination area, causes the fourth character information to be displayed in the second display area when the determination unit determines that the second detection object is located within the fourth determination area. The control device according to item 20 or 21. In this case, by bringing the first detection object closer to the display surface, the first character information or the third character information can be selected. By bringing the second detection object closer to the display surface, the second character information or the fourth character information can be selected.

[0238] [Item 23] a coordinate acquisition unit that acquires spatial coordinates of a detection object located in front of a display surface of a display; a setting unit that sets a first determination area associated with first character information based on the spatial coordinates of the detection object when the detection object enters a predetermined input space; a determination unit that determines whether or not the detection object is located within the first determination area after the first determination area is set; a display output unit that causes the first character information to be displayed on the display surface when the determination unit determines that the detection object is located within the first determination area; A control device comprising: According to this embodiment, a first determination area can be set at a position desired by the user in the input space. Therefore, character information can be easily selected by aerial movements.

[0239] [Item 24] The setting unit sets the first determination area and the second determination area associated with the second character information in a plane parallel to the display surface or in a plane intersecting the display surface. After the second determination area is set, the determination unit determines whether or not the object to be detected is located within the second determination area. The display output unit, when the determination unit determines that the object to be detected is located within the second determination area, displays the second character information on the display surface. The control device described in item 23. In this case, text information can be selected within a plane parallel to the display surface, or within a plane intersecting the display surface.

[0240] [Item 25] The first determination area and the second determination area are adjacent, The setting unit, when the object to be detected is located in the first determination area, expands the first determination area toward the second determination area and narrows the second determination area. The control device described in item 24. In this case, it is possible to suppress the instability of the display of text information when the object to be detected is located at the boundary between the first and second judgment areas.

[0241] [Item 26] The steps include obtaining the spatial coordinates of the object to be detected located in front of the display surface of the display, A step of setting a first determination area associated with the first character information included in the first character information group, based on the coordinates of a first display area on the display surface where information about the first character information group is displayed, and the spatial coordinates of the object to be detected at a reference time which is the time when the object to be detected designated the first display area. After the first determination area is set, the step of determining whether or not the object to be detected is located within the first determination area, If it is determined that the object to be detected is located within the first determination area, the first step is to display the first character information in the first display area. A control method comprising the following features. According to this configuration, text information can be easily selected by airborne movements, regardless of the size of the display surface.

[0242] [Item 27] The steps include obtaining the spatial coordinates of a first detection target and a second detection target located in front of the display surface of the display, The steps include setting a first determination region associated with the first character information included in the first character information group based on the spatial coordinates of the first detection target at a reference time which is the time when a predetermined input start condition is met, and setting a second determination region associated with the second character information included in the second character information group based on the spatial coordinates of the second detection target at the reference time, After the first determination area and the second determination area are set, the steps include determining whether the first object to be detected is located within the first determination area and whether the second object to be detected is located within the second determination area, If it is determined that the first object to be detected is located in the first determination area, the first character information is displayed in the first display area of ​​the display surface; if it is determined that the second object to be detected is located in the second determination area, the second character information is displayed in the second display area of ​​the display surface; A control method comprising the following features. According to this embodiment, text information can be selected from multiple objects to be detected.

[0243] [Item 28] The steps include obtaining the spatial coordinates of the object to be detected located in front of the display surface of the display, The steps include setting a first determination area associated with first character information, based on the spatial coordinates of the object to be detected when the object enters a predetermined input space, After the first determination area is set, a step of determining whether the detection target is located within the first determination area; When the determination unit determines that the detection target is located within the first determination area, a step of causing the first character information to be displayed on the display surface; A control method comprising the above. According to this aspect, it is easy to select character information by an aerial operation.

Description of Signs

[0244] 1... Information processing system, 10... Sensor, 12... Control device, 14... Display, 15... Display surface, 20... Control unit, 22... Storage unit, 24... Image acquisition unit, 26... Coordinate acquisition unit, 28... Calculation unit, 30... Determination unit, 32... Setting unit, 34... Function execution unit, 36... Display output unit, 40... Display area, 42... Input display area, 44... Determination area, 48... Input space, 60... Display area.

Claims

1. A coordinate acquisition unit that acquires the spatial coordinates of the object to be detected located in front of the display surface of the display, A setting unit sets a first determination area associated with the first character information included in the first character information group, based on the coordinates of a first display area on the display surface where information related to the first character information group is displayed, and the spatial coordinates of the object to be detected at a reference time which is the time when the first display area is designated by the object to be detected. After the first determination area is set, a determination unit determines whether or not the object to be detected is located within the first determination area, When the determination unit determines that the object to be detected is located within the first determination area, the display output unit displays the first character information in the first display area. Equipped with, The setting unit sets the first determination area such that the first determination area intersects the straight line connecting the spatial coordinates of the object to be detected at the reference time and the first display area. The setting unit sets the second determination region, which is associated with the second character information included in the first character information group, to intersect the straight line. After the second determination area is set, the determination unit determines whether or not the object to be detected is located within the second determination area. The display output unit, when it determines that the object to be detected is located within the second determination area, displays the second character information in the first display area. The setting unit is a control device that sets the distance between the first determination area and the second determination area along the straight line to be wider the faster the movement speed of the object to be detected after a predetermined time from the reference time.

2. A coordinate acquisition unit that acquires the spatial coordinates of a detection target located in front of the display surface of the display, A setting unit sets a first determination area associated with the first character information included in the first character information group, based on the coordinates of a first display area on the display surface where information related to the first character information group is displayed, and the spatial coordinates of the object to be detected at a reference time which is the time when the first display area is designated by the object to be detected. After the first determination area is set, a determination unit determines whether or not the object to be detected is located within the first determination area, When the determination unit determines that the object to be detected is located within the first determination area, the display output unit displays the first character information in the first display area. Equipped with, The setting unit sets the first determination area such that the first determination area intersects the straight line connecting the spatial coordinates of the object to be detected at the reference time and the first display area. The setting unit sets the second determination region, which is associated with the second character information included in the first character information group, to intersect the straight line. After the second determination area is set, the determination unit determines whether or not the object to be detected is located within the second determination area. The display output unit, when it determines that the object to be detected is located within the second determination area, displays the second character information in the first display area. The second determination area is located closer to the display surface than the first determination area. A control device wherein the volume of the second determination region is greater than the volume of the first determination region.

3. A coordinate acquisition unit that acquires the spatial coordinates of a detection target located in front of the display surface of the display, A setting unit sets a first determination area associated with the first character information included in the first character information group, based on the coordinates of a first display area on the display surface where information related to the first character information group is displayed, and the spatial coordinates of the object to be detected at a reference time which is the time when the first display area is designated by the object to be detected. After the first determination area is set, a determination unit determines whether or not the object to be detected is located within the first determination area, When the determination unit determines that the object to be detected is located within the first determination area, the display output unit displays the first character information in the first display area. Equipped with, The setting unit sets the first determination area such that the first determination area intersects the straight line connecting the spatial coordinates of the object to be detected at the reference time and the first display area. The setting unit sets the second determination region, which is associated with the second character information included in the first character information group, to intersect the straight line. After the second determination area is set, the determination unit determines whether or not the object to be detected is located within the second determination area. The display output unit, when it determines that the object to be detected is located within the second determination area, displays the second character information in the first display area. The first determination area and the second determination area are adjacent to each other along the straight line, The setting unit is a control device that, when the object to be detected is located in the second determination area, expands the second determination area toward the first determination area and narrows the first determination area.

4. The display output unit displays information related to the second character information group in the second display area of ​​the display surface. The determination unit determines whether the object to be detected is specifying either the first display area or the second display area, based on the spatial coordinates of the object to be detected, the coordinates of the first display area, and the coordinates of the second display area. The aforementioned setting unit is, If the determination unit determines that the object to be detected is specifying the first display area, it associates the first character information with the first determination area and sets the first determination area. If the determination unit determines that the object to be detected has designated the second display area, it associates the second character information included in the second character information group with the first determination area, and sets the first determination area based on the spatial coordinates of the object to be detected at the reference time, which is the time when the object to be detected designated the second display area. The control device according to any one of claims 1 to 3.

5. When the determination unit determines that the object to be detected is located within the first determination area, the display output unit displays the first character information in the first display area and displays the third character information associated with the first character information in a second display area adjacent to the first display area. The control device according to any one of claims 1 to 3.

6. When the determination unit determines that the object to be detected is located within the second determination area, the display output unit displays the second character information in the first display area and displays the fourth character information associated with the second character information in the second display area. The control device according to claim 5.

7. The display surface has an input display area where the entered characters are displayed. The determination unit determines whether a first decision operation or a second decision operation has been performed by the detected object, and whether the detected object has designated the second display area. The aforementioned display output unit is If the detection target object is located in the first determination area and the determination unit determines that the first determination operation has been performed, the first character information is displayed in the input display area. If the detection target object specifies the second display area and the determination unit determines that the second determination operation has been performed, the third character information is displayed in the input display area. The control device according to claim 5.

8. The first character information and the second character information are each numerical information. The setting unit sets the difference between the digits of the first character information and the digits of the second character information to be larger the faster the movement speed of the object to be detected after a predetermined time from the reference time. The control device according to claim 1.

9. The display surface has an input display area where the entered characters are displayed. The determination unit determines whether or not a decision action was performed based on the detected object. The display output unit, when it determines that the object to be detected is located in the first determination area and that a determination operation has been performed, displays the first character information in the input display area. The control device according to any one of claims 1 to 3.

10. A coordinate acquisition unit that acquires the spatial coordinates of a first detection target and a second detection target located in front of the display surface of the display, A setting unit sets a first determination region associated with the first character information included in the first character information group based on the spatial coordinates of the first detection target at a reference time which is the time when a predetermined input start condition is met, and sets a second determination region associated with the second character information included in the second character information group based on the spatial coordinates of the second detection target at the reference time, After the first determination area and the second determination area are set, a determination unit determines whether the first object to be detected is located within the first determination area and whether the second object to be detected is located within the second determination area. A display output unit which, when the determination unit determines that the first object to be detected is located in the first determination area, displays the first character information in the first display area of ​​the display surface, and when the determination unit determines that the second object to be detected is located in the second determination area, displays the second character information in the second display area of ​​the display surface, Equipped with, The aforementioned setting unit is, The first determination area is set such that it intersects with a first straight line connecting the spatial coordinates of the first detection target at the reference time and the first display area. The second determination area is set such that it intersects with the second straight line connecting the spatial coordinates of the second detection target at the aforementioned reference time and the second display area. The aforementioned setting unit is, The third determination region, which is associated with the third character information included in the first character information group, is set to intersect the first straight line. The fourth determination region, which is associated with the fourth character information included in the second character information group, is set to intersect with the second line, The determination unit, After the third determination area is set, it is determined whether or not the first object to be detected is located within the third determination area. After the fourth determination area is set, it is determined whether or not the second object to be detected is located within the fourth determination area. The aforementioned display output unit is If the determination unit determines that the first object to be detected is located within the third determination area, the third character information is displayed in the first display area. If the determination unit determines that the second object to be detected is located within the fourth determination area, the fourth character information is displayed in the second display area. The aforementioned setting unit is, The faster the movement speed of the first object to be detected after a predetermined time from the reference time, the wider the distance between the first determination area and the third determination area along the first straight line is set. A control device that sets a wider distance between the second determination area and the fourth determination area along the second straight line as the speed of movement of the second detection target object is faster from the reference time to the predetermined time.

11. A coordinate acquisition unit that acquires the spatial coordinates of a first detection target and a second detection target located in front of the display surface of the display, A setting unit sets a first determination region associated with the first character information included in the first character information group based on the spatial coordinates of the first detection target at a reference time which is the time when a predetermined input start condition is met, and sets a second determination region associated with the second character information included in the second character information group based on the spatial coordinates of the second detection target at the reference time, After the first determination area and the second determination area are set, a determination unit determines whether the first object to be detected is located within the first determination area and whether the second object to be detected is located within the second determination area. A display output unit which, when the determination unit determines that the first object to be detected is located in the first determination area, displays the first character information in the first display area of ​​the display surface, and when the determination unit determines that the second object to be detected is located in the second determination area, displays the second character information in the second display area of ​​the display surface, Equipped with, The aforementioned setting unit is, The first determination area is set such that it intersects with a first straight line connecting the spatial coordinates of the first detection target at the reference time and the first display area. The second determination area is set such that it intersects with the second straight line connecting the spatial coordinates of the second detection target at the aforementioned reference time and the second display area. The aforementioned setting unit is, The third determination region, which is associated with the third character information included in the first character information group, is set to intersect the first straight line. The fourth determination region, which is associated with the fourth character information included in the second character information group, is set to intersect with the second line, The determination unit, After the third determination area is set, it is determined whether or not the first object to be detected is located within the third determination area. After the fourth determination area is set, it is determined whether or not the second object to be detected is located within the fourth determination area. The aforementioned display output unit is If the determination unit determines that the first object to be detected is located within the third determination area, the third character information is displayed in the first display area. If the determination unit determines that the second object to be detected is located within the fourth determination area, the fourth character information is displayed in the second display area. The third determination area is located closer to the display surface than the first determination area. The fourth determination area is located closer to the display surface than the second determination area. The volume of the third determination region is greater than the volume of the first determination region. A control device wherein the volume of the fourth determination region is greater than the volume of the second determination region.

12. A coordinate acquisition unit that acquires the spatial coordinates of a first detection target and a second detection target located in front of the display surface of the display, A setting unit sets a first determination region associated with the first character information included in the first character information group based on the spatial coordinates of the first detection target at a reference time which is the time when a predetermined input start condition is met, and sets a second determination region associated with the second character information included in the second character information group based on the spatial coordinates of the second detection target at the reference time, After the first determination area and the second determination area are set, a determination unit determines whether the first object to be detected is located within the first determination area and whether the second object to be detected is located within the second determination area. A display output unit which, when the determination unit determines that the first object to be detected is located in the first determination area, displays the first character information in the first display area of ​​the display surface, and when the determination unit determines that the second object to be detected is located in the second determination area, displays the second character information in the second display area of ​​the display surface, Equipped with, The aforementioned setting unit is, The first determination area is set such that it intersects with a first straight line connecting the spatial coordinates of the first detection target at the reference time and the first display area. The second determination area is set such that it intersects with the second straight line connecting the spatial coordinates of the second detection target at the aforementioned reference time and the second display area. The aforementioned setting unit is, The third determination region, which is associated with the third character information included in the first character information group, is set to intersect the first straight line. The fourth determination region, which is associated with the fourth character information included in the second character information group, is set to intersect with the second line, The determination unit, After the third determination area is set, it is determined whether or not the first object to be detected is located within the third determination area. After the fourth determination area is set, it is determined whether or not the second object to be detected is located within the fourth determination area. The aforementioned display output unit is If the determination unit determines that the first object to be detected is located within the third determination area, the third character information is displayed in the first display area. If the determination unit determines that the second object to be detected is located within the fourth determination area, the fourth character information is displayed in the second display area. The first determination area and the third determination area are adjacent to each other along the first straight line, The second determination area and the fourth determination area are adjacent to each other along the second straight line, The aforementioned setting unit is, If the first object to be detected is located in the third determination area, the third determination area is expanded toward the first determination area, and the first determination area is narrowed. A control device that, when the second object to be detected is located in the fourth determination area, expands the fourth determination area toward the second determination area and narrows the second determination area.

13. A coordinate acquisition unit that acquires the spatial coordinates of the object to be detected located in front of the display surface of the display, A setting unit sets a first determination area associated with first character information, based on the spatial coordinates of the object to be detected when the object enters a predetermined input space, After the first determination area is set, a determination unit determines whether or not the object to be detected is located within the first determination area, When the determination unit determines that the object to be detected is located within the first determination area, the display output unit displays the first character information on the display surface. Equipped with, The setting unit sets the first determination area and the second determination area associated with the second character information in a plane parallel to the display surface or in a plane intersecting the display surface. After the second determination area is set, the determination unit determines whether or not the object to be detected is located within the second determination area. The display output unit, when the determination unit determines that the object to be detected is located within the second determination area, displays the second character information on the display surface. The first determination area and the second determination area are adjacent, The setting unit is a control device that, when the object to be detected is located in the first determination area, expands the first determination area toward the second determination area and narrows the second determination area.

Images