Non-contact input support device, non-contact input support method, and non-contact input support program

Abstract

To provide an input supporting device that allows a user to operate an operation screen by intuition without detailed explanations.SOLUTION: A contactless input supporting device includes: a projection unit for displaying an operation screen, in the air, for receiving an operation of a user; a measurement unit for measuring a position of a hand of the user on the operation screen; a shadow generation unit for generating a shadow of the hand from a result obtained by the measuring unit; and a display control unit for displaying the shadow in the position where the hand and the operation screen are overlapped with each other.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a technology for supporting the operation of an aerial display. [Background technology]

[0002] Various methods for supporting the operation of an operation screen have been proposed. For example, Patent Document 1 describes a method of using a marker during non-contact operation of an operation screen (air display).

[0003] The shape of this marker changes when the user (operator) operates the operation screen with his / her hand. Specifically, when the user's hand is away from the operation screen, the marker is initially displayed in a divided circular shape. When the user's hand approaches the operation screen, the marker forms a ring. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2014-219938 A Summary of the Invention [Problem to be solved by the invention]

[0005] In the configuration of Patent Document 1, the shape of the marker changes depending on the positional relationship between the operation screen and the user's hand, but for users who cannot grasp the state that the marker represents, they may not understand the intention of the marker's change, which may cause them to operate the operation screen erroneously. Also, when used by a wide range of users, from the elderly to children, it is necessary to provide instructions on how to operate the device, which is not very convenient.

[0006] Therefore, an object of the present invention is to provide a non-contact input support device that enables a user to intuitively operate an operation screen without needing detailed explanations. [Means for solving the problem]

[0007] According to an example of the present disclosure, the non-contact input support device includes a projection unit that displays in the air an operation screen that accepts user operations, a measurement unit that measures the distance between the operation screen and the user's hand that overlaps the operation screen, a shadow generation unit that generates a shadow of the hand from the result of measurement by the measurement unit, and a display control unit that displays the shadow at a position on the operation screen that overlaps with the hand.

[0008] In this configuration, the shadow of the user's hand is displayed on the operation screen according to the relative positions of the hand and the operation screen, so that the user feels as if they are operating an actual operation panel or buttons, even though they are operating the operation screen, improving convenience.

[0009] According to an example of the present disclosure, the measurement unit of the non-contact input support device may measure the hand in three dimensions.

[0010] With this configuration, the user's hand can be measured in three dimensions, and the distance between the operation screen and the user's hand can be grasped stereoscopically.

[0011] According to an example of the present disclosure, the shadow generating unit of the non-contact input support device includes a positional relationship calculation unit, a shadow image generating unit, and a position control unit. The positional relationship calculation unit may calculate a three-dimensional position between the hand and the operation screen using a result of measurement by the measurement unit. The shadow image generating unit may generate a shadow similar to the shape of the hand using the result of measurement and the three-dimensional position. The position control unit may control a position at which the shadow is displayed on the operation screen.

[0012] This configuration allows a shadow to be generated according to the shape and position of the user's hand, allowing the user to feel as if they are operating an actual operation panel or buttons, improving convenience.

[0013] According to an example of the present disclosure, the shadow generating unit of the non-contact input support device may determine the shading of the shadow based on the ratio between the distance between the operation screen and the user's hand and the distance between the projection unit and the operation screen.

[0014] With this configuration, the shadow can be configured to become darker as the user's hand approaches the operation screen, and become lighter as the user moves away. This makes the shadow more similar to the actual shadow movement, making it easier for the user to see.

[0015] According to an example of the present disclosure, the shadow generating unit of the non-contact input support device may determine the size of the shadow based on the ratio of the distance between the operation screen and the user's hand to the distance between the projection unit and the operation screen.

[0016] With this configuration, the shadow can be configured to become smaller when the user's hand approaches the operation screen and become larger when the user moves away. This makes the shadow more similar to the movement of an actual shadow, making it easier for the user to see. Effect of the Invention

[0017] According to the present invention, it is possible to provide an input support device that enables a user to intuitively operate an operation screen without the need for detailed explanations. [Brief description of the drawings]

[0018] [Figure 1] FIG. 1 is a schematic block diagram of a non-contact input support device according to a first embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram of the non-contact input support device according to the first embodiment of the present invention. [Diagram 3] 3(A), 3(B), and 3(C) are diagrams showing the movement of a shadow of the non-contact input support device according to the first embodiment of the present invention. [Figure 4] FIG. 4 is a flowchart showing a process flow of the measurement unit of the non-contact input support device according to the first embodiment of the present invention. [Diagram 5] FIG. 5 is a flowchart showing a flow of processing by the shadow generating unit and the display control unit of the non-contact input support device according to the first embodiment of the present invention. [Figure 6] 6(A), 6(B), and 6(C) are diagrams showing the movement of a shadow of a non-contact input support device according to a second embodiment of the present invention. [Figure 7]FIG. 7 is a diagram showing the movement of a shadow of a non-contact input support device according to the third embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Hereinafter, embodiments of the present invention will be described with reference to several drawings.

[0020] Application Examples First, an example to which the present invention is applied will be described with reference to Fig. 1. Fig. 1 is a schematic diagram of a non-contact input support device according to a first embodiment of the present invention. The non-contact input support device 10 includes a display unit 110, a measurement unit 120, a shadow generation unit 130, and a display control unit 140. The display unit 110 includes a projection unit 111 and an operation screen 112. The operation screen 112 is, for example, an aerial display or an aerial monitor. Note that the distance between the measurement unit 120 and the operation screen 112 is previously defined as a distance d0.

[0021] Projection unit 111 displays information on a liquid crystal display of a ticket vending machine or the like (hereinafter, the ticket vending machine screen) on operation screen 112. More specifically, using a reflective material or the like, the ticket vending machine screen is reflected and displayed on operation screen 112. Operation screen 112 displays the ticket vending machine screen.

[0022] The user brings his / her hand closer to the operation screen 112. More specifically, the user brings his / her hand closer to a button in order to press the button. The measurement unit 120 measures the shape of the user's hand, the orientation of the hand, and the distance between the operation screen 112 and the hand. The measurement unit 120 outputs the measurement result (hereinafter, the measurement result) to the shadow generation unit 130.

[0023] The shadow generating unit 130 generates a shadow similar to the shape of the user's hand (for example, a two-dimensional image of the hand) based on the measurement result. The shadow generating unit 130 outputs the shadow, the position of the shadow, the size of the shadow, etc. to the display control unit 140.

[0024] The display control unit 140 outputs the shadow of the user's hand to the projection unit 111. The projection unit 111 displays the shadow of the hand on the operation screen 112 so that the shadow of the hand, the measurement unit 120, the user's hand, and the shadow of the user's hand overlap (are aligned in one direction).

[0025] With this configuration, the user can see the shadow of his or her hand on the operation screen 112, and therefore can intuitively operate how to move his or her hand to the button.

[0026] Configuration example 1 FIG. 1 is a schematic block diagram of a non-contact input support device according to a first embodiment of the present invention. FIG. 2 is a block diagram of a non-contact input support device according to a first embodiment of the present invention. FIGS. 3(A), 3(B), and 3(C) are diagrams showing the movement of a shadow of the non-contact input support device according to the first embodiment of the present invention. FIG. 4 is a flowchart showing the processing flow of a measurement unit of the non-contact input support device according to the first embodiment of the present invention. FIG. 5 is a flowchart showing the processing flow of a shadow generation unit and a display control unit of the non-contact input support device according to the first embodiment of the present invention.

[0027] A more specific configuration example will be described using the configuration of the non-contact input support device 10 in Fig. 1 and the block diagram of the non-contact input support device in Fig. 2. First, the shadow will be defined as follows. The shadow generated by the shadow generating unit 130 is, for example, a two-dimensional image generated to resemble the shadow (dark part) of a hand formed by blocking light with the hand. In other words, it is different from the shadow that is actually visible from the user's hand and light.

[0028] The non-contact input support device 10 includes a display unit 110, a measurement unit 120, a shadow generation unit 130, and a display control unit 140. The display unit 110, the measurement unit 120, the shadow generation unit 130, and the display control unit 140 are connected so as to be able to transmit and receive data signals. These may be connected by a wired method such as a cable, or by a wireless method such as Bluetooth (registered trademark). The non-contact input support device 10 and the ticket vending machine 20 are connected so that the projection unit 111 can display the ticket vending machine screen on the operation screen 112, and further so that the result of the user operating the operation screen 112 can be output to the ticket vending machine 20.

[0029] First, the configuration of the non-contact input support device 10 will be described.

[0030] (Configuration of display unit 110) The display unit 110 includes a projection unit 111 and an operation screen 112. The operation screen 112 is, for example, an aerial display or an aerial monitor.

[0031] Projection unit 111 reflects light onto the ticket vending machine screen of ticket vending machine 20 using a reflecting material or the like. Operation screen 112 is formed by forming an image of this reflected light in the air. The ticket vending machine screen of ticket vending machine 20 is displayed on this operation screen 112. When a user operates operation screen 112, a motion sensor or area sensor (not shown) detects this operation. By detecting this operation, the user can operate ticket vending machine 20 by operating operation screen 112.

[0032] (Configuration of measuring unit 120) The measuring unit 120 is, for example, a 3D camera. The 3D camera may be any of a stereo camera, a TOF (Time of Flight) camera, and a LiDAR (Light Detection and Ranging). The 3D camera to be used may be determined based on the location where the measuring unit 120 is installed and the processing performance of a computing device that calculates the results of the measuring unit 120.

[0033] The measurement unit 120 measures the shape and orientation of the user's hand, and the distance between the operation screen 112 and the user's hand (hereinafter, hand distance). Note that since the measurement unit 120 is a 3D camera, it is possible to obtain three-dimensional information with depth, rather than a flat image, of the user's hand shape and orientation.

[0034] (Configuration of the shadow generation unit 130) The shadow generation unit 130 includes an information acquisition unit 131, a positional relationship calculation unit 132, a shadow image generation unit 133, and a position control unit 134.

[0035] Information acquisition unit 131 acquires the shape, hand orientation, and hand distance of the user (hereinafter, measurement result) using measurement unit 120. Positional relationship calculation unit 132 uses the measurement result to calculate the three-dimensional positional relationship (hereinafter, three-dimensional position) between the user's hand and operation screen 112. When determining the three-dimensional position, positional relationship calculation unit 132 may use the coordinates of an arbitrary point or may use the coordinates forming the outline of the user's hand.

[0036] The three-dimensional position is a coordinate system indicating the position of the user's hand in a three-dimensional space formed by the measurement unit 120 and the operation screen 112. In other words, the three-dimensional position indicates the position of the user's hand in this three-dimensional space.

[0037] The shadow image generating unit 133 generates a shadow similar to the shape of the user's hand and fingers measured by the measuring unit 120. The position control unit 134 determines a position (hereinafter, shadow output position) at which a shadow is output on the operation screen 112. Note that the shadow output position is the coordinate at which the shadow is output when the operation screen 112 is represented on the XY plane.

[0038] (Configuration of display control unit 140) The display control unit 140 outputs the shadow formed by the shadow image generation unit 133 and the shadow output position to the projection unit 111. The projection unit 111 outputs the shadow and the shadow output position to the operation screen 112.

[0039] (Processing flow of the non-contact input support device 10) Based on the above-mentioned configuration, a specific process flow of the non-contact input support device 10 will be described. A user purchases a ticket. At this time, the user holds his / her hand over the operation screen 112. The measurement unit 120 measures three-dimensional information. The measurement unit 120 continues to perform measurements until the user removes his / her hand from the operation screen 112 (ends screen operation) or completes the purchase of the ticket at the ticket machine 20.

[0040] The measuring unit 120 transmits the three-dimensional measurement results measured by the measuring unit 120 to the information acquiring unit 131. The information acquiring unit 131 acquires the measurement results. The information acquiring unit 131 outputs the measurement results to the positional relationship calculating unit 132.

[0041] The positional relationship calculation unit 132 calculates a three-dimensional position from the measurement result. The positional relationship calculation unit 132 outputs the measurement result and the three-dimensional position to the shadow image generation unit 133.

[0042] The shadow image generating unit 133 generates a shadow (two-dimensional image) of the user's hand using the measurement result and the three-dimensional position. The shadow image generating unit 133 also determines the size, shape, and shade of the shadow. A specific method for the shadow image generating unit 133 to determine the size and shade of the shadow will be described later. The shadow image generating unit 133 outputs the size, shape, shade, and three-dimensional position of the shadow to the position control unit 134.

[0043] The position control unit 134 determines the shadow output position by using the three-dimensional position. In other words, the position control unit 134 determines the coordinates at which the shadow is output on the operation screen 112, which is an XY plane. The position control unit 134 outputs the shadow and the shadow output position to the display control unit 140.

[0044] The display control unit 140 determines at what position on the operation screen 112 the shadow is to be displayed. The display control unit 140 outputs the size, shape, and shade of the shadow, as well as the shadow output position, to the projection unit 111. The projection unit 111 outputs the shadow onto the operation screen 112.

[0045] The user moves their hand closer to a button displayed on operation screen 112 while watching the movement of the shadow of their own hand displayed on operation screen 112. The user operates (presses) the button displayed on operation screen 112 to purchase a ticket.

[0046] With this configuration, the user can see the shadow of his / her hand on the operation screen 112. That is, even when operating the operation screen 112, which has no substance, the user can intuitively understand how to move his / her hand to the button while referring to the shadow of the hand.

[0047] Next, a specific process performed by the shadow image generating unit 133 to generate a shadow will be described with reference to FIGS. 3(A), 3(B), and 3(C).

[0048] Fig. 3(A) shows a state where the user's hand 201 is separated from the operation screen 112. Fig. 3(B) shows a state where the user's hand is approaching the operation screen 112. Fig. 3(C) shows a state immediately before the user touches a button on the operation screen 112.

[0049] First, the structure of the shadow 202A will be described with reference to FIG. 3(A). As shown in FIG. 3(A), the user's hand 201 is spaced a distance d1 from the operation screen 112. The shadow image generating unit 133 generates the shadow 202A of the user's hand 201. In order to determine the size and shade of the shadow, the shadow image generating unit 133 compares the shadow 202A with the user's hand 201. Operation screen 112 A ratio of a distance d1 between the measuring unit 120 and the operation screen 112 and a distance d0 between the measuring unit 120 and the operation screen 112 is calculated.

[0050] The structure of the shadow 202B will be described with reference to FIG. 3(B). As shown in FIG. 3(B), the user's hand 201 is at a distance d2 from the operation screen 112. The shadow image generating unit 133 generates a shadow 202B of the user's hand 201. As in FIG. 3(A), the shadow image generating unit 133 generates a shadow 202B of the user's hand 201. Operation screen 112 A ratio of a distance d2 between the measuring unit 120 and the operation screen 112 and a distance d0 between the measuring unit 120 and the operation screen 112 is calculated.

[0051] Next, the structure of the shadow 202C will be described with reference to FIG. 3(C). As shown in FIG. 3(C), the user's hand 201 is in a state of almost contact with the operation screen 112 (distance d3 ≈ 0). The shadow image generating unit 133 generates a shadow 202C of the user's hand 201. As in FIG. 3(A), the shadow image generating unit 133 generates a shadow 202C of the user's hand 201. Operation screen 112 and a ratio of the distance d3 between the measuring unit 120 and the operation screen 112 to the distance d0 between the measuring unit 120 and the operation screen 112.

[0052] First, the states of Fig. 3(A) and Fig. 3(B) are compared. Comparing distance d1 and distance d2, there is a relationship of d1>d2. At this time, the shadow image generating unit 133 generates the shadow 202B so that it is smaller than the shadow 202A according to the above-mentioned ratio. Furthermore, the shadow image generating unit 133 generates the shadow 202B so that the color density is darker than the shadow 202A according to the ratio.

[0053] Next, the states of FIG. 3(B) and FIG. 3(C) are compared. Comparing distance d2 and distance d3, the relationship is d2>d3. At this time, the shadow image generating unit 133 generates the shadow 202C so that it is smaller than the shadow 202B according to the above-mentioned ratio. Also, the shadow image generating unit 133 generates the shadow 202C so that it is approximately the same size as the user's hand 201 according to the ratio. Furthermore, the shadow image generating unit 133 generates the color density of the shadow 202C so that it is darker than the shadow 202B according to the ratio.

[0054] When comparing the sizes of the shadow 202A, the shadow 202B, and the shadow 202C, the order is shadow 202A>shadow 202B>shadow 202C. Also, when comparing the color density (shade) of the shadow 202A, the shadow 202B, and the shadow 202C, the shadow 202A is the lightest, and the shadows 202B, 202C become gradually darker. In this way, the shadow image generating unit 133 generates a smaller shadow with a darker color when the user's hand approaches the operation screen 112. Also, the shadow image generating unit 133 generates a larger shadow with a lighter color when the user's hand moves away from the projection unit 111. That is, the shadow image generating unit 133 generates a shadow so that it has a shape similar to that of a shadow in the natural world.

[0055] This allows the user to operate the non-contact input support device 10 with a feeling almost similar to that of actually using the ticket vending machine 20. This reduces the sense of discomfort felt by the user when operating the non-contact input support device 10.

[0056] The process flow of the non-contact input support device 10 will be described with reference to the flowcharts of Figures 4 and 5. First, the process flow of the measurement unit 120 will be described with reference to Figure 4.

[0057] The measurement unit 120 detects whether the user's hand is held over the operation screen 112 (S101). If the measurement unit 120 detects the user's hand (S102: Yes), it continues measurement (S103).

[0058] The measurement unit 120 outputs the measurement result to the information acquisition unit 131 (S104). The measurement unit 120 returns to step S101 to continue measuring the operation screen 112. The measurement unit 120 repeats steps S101 to S104.

[0059] If the measurement unit 120 cannot detect the user's hand (S102: No), the measurement unit 120 stops the measurement (S105). In this case, the measurement unit 120 outputs the measurement result in which the shadow is not displayed to the information acquisition unit 131 (S106).

[0060] Next, the flow of processing by the shadow generation unit 130 and the display control unit 140 will be described with reference to the flowchart of FIG.

[0061] The information acquiring unit 131 acquires the measurement result from the measuring unit 120 (S111). If the measurement result includes three-dimensional information (S112: Yes), the information acquiring unit 131 outputs the measurement result to the positional relationship calculating unit 132 (S113).

[0062] The positional relationship calculation unit 132 calculates the three-dimensional position from the measurement result. The positional relationship calculation unit 132 outputs the measurement result and the three-dimensional position to the shadow image generation unit 133 (S114).

[0063] The shadow image generating unit 133 generates a shadow of the user's hand using the measurement result and the three-dimensional position (S115). The shadow image generating unit 133 determines the size and shade of the shadow according to the three-dimensional position and ratio. The shadow image generating unit 133 outputs the shadow and the three-dimensional position to the position control unit 134.

[0064] The position control unit 134 uses the three-dimensional position to determine the shadow output position (S116). The position control unit 134 outputs the shadow and the shadow output position to the display control unit 140.

[0065] The display control unit 140 determines, from the shadow and the shadow output position, at which position on the operation screen 112 the shadow is to be displayed (S117). The display control unit 140 outputs the shadow and the shadow output position to the projection unit 111.

[0066] The projection unit 111 outputs a shadow onto the operation screen 112 using the shadow and the shadow output position acquired from the display control unit 140 (S118).

[0067] If the measurement result does not include three-dimensional information (S112: No), in other words, if the measurement unit 120 cannot detect the user's hand, the information acquisition unit 131 outputs to the display control unit 140 to hide the shadow (S118). The display control unit 140 hides the shadow of the projection unit 111.

[0068] By configuring it in this way, it is possible to provide a non-contact input support device that allows the user to intuitively operate the operation screen without needing detailed explanation. Operation screen 112 The size and shade of the shadow are changed according to the ratio of the distance between the object and the measuring unit 120 and the operation screen 112, allowing the user to perform operations that are more in line with the actual environment.

[0069] In the above configuration, an example in which the non-contact input support device is applied to a ticket vending machine has been described. However, the present invention can be applied to devices that use a display displayed in space, such as an aerial display (operation screen), and is not limited to ticket vending machines.

[0070] In the above configuration, an example has been described in which a shade is set for the shadow. However, a configuration may be adopted in which the color of the shadow is changed instead of the shade. For example, a configuration may be adopted in which the color of the shadow is changed to red when the user touches a button on the operation screen 112.

[0071] In the above-described configuration, an example has been shown in which the measurement unit 120 measures the shape and orientation of the user's hand, and the distance between the operation screen 112 and the user's hand. However, the measurement unit 120 may be configured to detect the movement (motion) of the user's hand.

[0072] 3C, a configuration has been shown in which the user operates a button displayed on the operation screen 112 in a state in which the user's hand 201 is almost in contact with the operation screen 112 (distance d3 ≈ 0). However, the measurement unit 120 may be configured to detect an action of bending a finger or pressing a button after bringing the hand 201 close to the button. In other words, the measurement unit 120 may be configured to determine that the user has operated a button even if the user's finger (hand 201) has not reached the operation screen 112 (there is a certain distance between the finger and the operation screen).

[0073] Configuration example 2 Next, a specific configuration of the character input device will be described with reference to Fig. 6(A), Fig. 6(B), and Fig. 6(C). Fig. 6(A), Fig. 6(B), and Fig. 6(C) are diagrams showing the movement of the shadow of the non-contact input support device according to the second embodiment of the present invention.

[0074] The second embodiment differs from the first embodiment in the output position of the shadow by the shadow image generating unit 133 and the position control unit 134. Other points are the same as those in the first embodiment, and the description of the same points will be omitted.

[0075] 6(A), 6(B), and 6(C) are in a state where the user's hand 201 is almost in contact with the operation screen 112, similar to FIG. 3(C) in the first embodiment. At this time, the closer the user's hand 201 is to the operation screen 112, the more likely it is that the user's line of sight, the user's hand 201, and the shadow will be aligned in a straight line (aligned in one direction). At this time, the shadow image generating unit 133 and the position control unit 134 adjust the size of the shadow and the shadow output position.

[0076] A more specific method will be described with reference to Fig. 6(A), Fig. 6(B), and Fig. 6(C). The shadow image generating unit 133 outputs the shadow, which is two-dimensional image information, and the three-dimensional position to the position control unit 134. The position control unit 134 determines the shadow output position on the operation screen 112.

[0077] 6A, when the user's hand 201 is almost in contact with the operation screen 112, the position control unit 134 moves the shadow 203A along the X-axis. The distance of the movement along the X-axis should be within the range on the X-axis where the button to be operated is displayed, and should be any position where the user does not feel uncomfortable. This allows the shadow image generating unit 133 to form a portion where the user's hand 201 and the shadow 203A do not overlap at the user's line of sight.

[0078] 6B, when the user's hand 201 is almost in contact with the operation screen 112, the position control unit 134 moves the shadow 203B along the Y axis. The distance of the movement along the Y axis should be within the range on the Y axis where the button to be operated is displayed, and should be any position where the user does not feel uncomfortable. This allows the shadow image generating unit 133 to form a portion where the user's hand 201 and the shadow 203B do not overlap at the user's line of sight.

[0079] 6(C), when the user's hand 201 is almost in contact with the operation screen 112, the shadow image generating unit 133 forms a shadow 203C larger than the hand 201. The size of the shadow 203C is enlarged by an arbitrary magnification from the user's hand 201. This magnification may be an arbitrary value that does not cause the user to feel uncomfortable. This allows the shadow image generating unit 133 to form a portion where the user's hand 201 and the shadow 203C do not overlap at the user's line of sight.

[0080] With this configuration, the user can operate the non-contact input support device without detailed instructions. Also, the shadow is prevented from being completely hidden by the user's hand due to the user's line of sight. In other words, the user can easily recognize the position of the hand by using the shadow.

[0081] Configuration example 3 Next, a specific configuration of the character input device will be described with reference to Fig. 7. Fig. 7 is a diagram showing the shadow of a non-contact input support device according to a third embodiment of the present invention.

[0082] The third embodiment is different from the first embodiment in the method of outputting a shadow by the shadow image generating unit 133 and the position control unit 134. The other points are the same as those in the first embodiment, and the description of the same points will be omitted.

[0083] The user operates a button on the operation screen 112, for example, using the index finger of the hand 201. The positional relationship calculation unit 132 calculates a three-dimensional position from the measurement result. The positional relationship calculation unit 132 outputs the measurement result and the positional relationship to the shadow image generation unit 133.

[0084] The shadow image generating unit 133 generates a shadow 204 using the measurement result and the three-dimensional position. At this time, the shadow image generating unit 133 displays the part corresponding to the user's finger (the finger 205 in the shadow 204) in a darker color. That is, the part of the shadow 204 where the user's hand is closest to the operation screen 112 (the index finger in this case) is displayed in a darker color. This allows the user to recognize which finger is about to perform the operation.

[0085] With this configuration, it is possible to provide a non-contact input support device that allows the user to intuitively operate the operation screen without needing detailed instructions. Also, because the finger operating the button is displayed in a darker color, the user can recognize which part is closest to the button on the operation screen 112, and erroneous operation is suppressed.

[0086] It should be noted that the present invention is not limited to the above-described embodiment as it is, and the components can be modified in the implementation stage without departing from the gist of the invention. In particular, the order of the operation processing procedures in each device is not limited to the above-described order, and the order may be changed as necessary. In addition, various inventions can be formed by appropriately combining multiple components disclosed in the above-described embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, components across different embodiments may be appropriately combined.

[0087] Furthermore, the correspondence between the configuration according to the present invention and the configuration according to the above-mentioned embodiment can be described as follows. <Additional Notes> A projection unit (111) that displays an operation screen (112) in mid-air to receive operations from a user; a measuring unit (120) that measures a distance between the operation screen (112) and the user's hand that overlaps the operation screen (112); a shadow generating unit (130) that generates a shadow of the hand from the result of measurement by the measuring unit (120); a display control unit (140) that displays the shadow at a position overlapping with the hand on the operation screen (112); A non-contact input support device (10) comprising: [Explanation of symbols]

[0088] d0, d1, d2, d3...distance 10. Non-contact input support device 20…Ticket machine 110...Display section 111...Projection section 112…Operation screen 120…Measuring section 130...Shadow generation section 131…Information acquisition department 132...Positional relationship calculation unit 133...Shadow image generating unit 134... Position control unit 140...Display control unit 201...Hand 202A, 202B, 202C, 203A, 203B, 203C, 204...Shadow 205...Finger

Claims

1. a projection unit that displays in mid-air an operation screen that accepts operations by a user; a measuring unit that measures a distance between the operation screen and the user's hand overlapping the operation screen; a shadow generating unit that generates a shadow of the hand from the result of the measurement by the measuring unit and determines a position at which the generated shadow of the hand is to be displayed on the operation screen; a display control unit that controls display of the shadow of the hand on the operation screen; Equipped with the shadow generating unit changes the color of the shadow of the hand depending on whether the distance between the user's hand and the operation screen measured by the measuring unit is longer or shorter than a set distance, and moves a position of the shadow of the hand displayed on the operation screen in a set direction when the user's hand is in contact with the operation screen. Non-contact input support device.

2. The non-contact input support device according to claim 1 , wherein the measurement unit measures the hand in three dimensions.

3. The shadow generating unit includes: a positional relationship calculation unit that calculates a three-dimensional position between the hand and the operation screen using a result of the measurement by the measurement unit; a shadow image generating unit that generates a shadow of the hand similar to the shape of the hand by using the measurement result and the three-dimensional position; a position control unit that determines a position at which the shadow of the hand is displayed on the operation screen, The non-contact input support device according to claim 1 or 2.

4. The non-contact input support device according to claim 3 , wherein the shadow generating unit determines a shade of the shadow of the hand based on a ratio between a distance between the operation screen and the user's hand and a distance between the measuring unit and the operation screen.

5. The non-contact input support device according to claim 3 , wherein the shadow generating unit determines a size of the shadow of the hand based on a ratio of a distance between the operation screen and the user's hand and a distance between the measuring unit and the operation screen.

6. An information processing device, a reception process of receiving operations from a user on an operation screen displayed in the air; a measuring step of measuring a position of the user's hand overlapping the operation screen; a generating step of generating a shadow of the hand from the measurement result and determining a position at which the generated shadow of the hand is to be displayed on the operation screen; a display step of controlling display of the shadow of the hand on the operation screen; Run the generating step is a step of changing a color of the shadow depending on whether the distance between the user's hand and the operation screen measured in the measuring step is longer or shorter than a set distance, and moving a position of the shadow of the hand displayed on the operation screen in a set direction when the user's hand is in contact with the operation screen; A method for supporting non-contact input.

7. In the information processing device, a reception process of receiving operations from a user on an operation screen displayed in the air; a measuring step of measuring a position of the user's hand overlapping the operation screen; a generating step of generating a shadow of the hand from the measurement result and determining a position at which the generated shadow of the hand is to be displayed on the operation screen; a display step of controlling display of the shadow of the hand on the operation screen; Run the command, the generating step is a step of changing a color of the shadow depending on whether the distance between the user's hand and the operation screen measured in the measuring step is longer or shorter than a set distance, and moving a position of the shadow of the hand displayed on the operation screen in a set direction when the user's hand is in contact with the operation screen; Non-contact input support program.

Images