Image display device, non-contact operation device, and application software for non-contact operation

Abstract

To provide an image display device which is excellent in operability and can be operated in a non-contact state.SOLUTION: An image display device includes a first display unit and a second display unit, in which a cursor is displayed on the first display unit or the second display unit, a touch operation can be performed on the second display unit, the second display unit displays a first image in which the cursor can be operated, and the cursor is operated on the first display unit by the touch operation on the first image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image display device that can be operated without contact and application software that enables the operation without contact. [Background technology]

[0002] In recent years, from the viewpoints of infection prevention and hygiene, there has been a demand for image display devices that can be operated without contact. Such image display devices are expected to be introduced in, for example, medical settings, restaurants, etc., and are being actively developed.

[0003] For example, various types of aerial displays have been proposed that display images in the air using retroreflective materials, etc. Patent Document 1 discloses a system that can display three-dimensional objects using an aerial display. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2023-183847 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the operability of non-contact operation using the above-mentioned aerial display is insufficient, and there are problems such as the tendency for incorrect input and operational errors to occur when precision is required for the application being operated, especially when the accuracy of the action input is required. To solve this problem, a method could be considered to reconstruct the display screen of the application being operated, but this is not easy due to the development costs and time required.

[0006] Therefore, an object of the present invention is to provide an image display device that is easy to operate and can be operated in a non-contact manner. Another object of the present invention is to provide an image display device that can be operated in a non-contact manner without changing existing applications. [Means for solving the problem]

[0007] In order to achieve the above-mentioned object, one embodiment of the present invention is an image display device having a first display unit and a second display unit, wherein a cursor is displayed on the first display unit or the second display unit, the second display unit is touch operable, the second display unit displays a first image on which the cursor can be operated, and the cursor is operated on the first display unit by touching the first image.

[0008] Furthermore, in the above configuration, it is preferable that the first image is capable of operating a first application, the second image on the first display unit is capable of operating a second application, and the touch operation moves the cursor onto the second image or performs an operation on the second image.

[0009] In the above configuration, the second display unit is preferably an aerial display.

[0010] In the above configuration, it is preferable that the operation of the second image is to click, rotate, enlarge or reduce the second image.

[0011] In the above configuration, it is preferable that the first image displays an icon, button, or slider that can be operated by a wheel of the cursor.

[0012] In the above configuration, it is preferable that the clicking, rotating, enlarging or reducing is performed by operating the icon, button or slider.

[0013] In the above configuration, it is preferable that the manipulation of the second image is scanning of the second image.

[0014] In the above configuration, the second image is preferably a cross-sectional view of a three-dimensional object in one axial direction.

[0015] In the above configuration, it is preferable that the first image has an icon, button or slider that can be operated by a wheel of the cursor.

[0016] In the above configuration, it is preferable that the scanning is executed by operating the icon, the button, or the slider. [Effects of the Invention]

[0017] According to the present invention, it is possible to provide an image display device that is easy to operate and can be operated in a non-contact manner, and also to provide an image display device that can operate existing applications in a non-contact manner. [Brief explanation of the drawings]

[0018] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of an image display device according to an embodiment of the present invention. [Figure 2] FIG. 2 is a schematic cross-sectional view showing an example of the configuration of an aerial display that can be used in an image display device according to one embodiment of the present invention. [Figure 3] FIG. 3 is a diagram showing an example of an image displayed on the image display unit in the present invention. [Figure 4] FIG. 4 is a diagram showing an example of an image displayed on the image display unit in the present invention. [Figure 5] FIG. 5 is a diagram showing an example of an image displayed on the image display unit in the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0019] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. However, the present invention is not limited to the following description, and various changes in form and details are possible without departing from the spirit and scope of the present invention. Therefore, the present invention should not be interpreted as being limited to the description of the embodiments shown below.

[0020] For ease of understanding, the position, size, range, etc. of each component shown in the drawings may not represent the actual position, size, range, etc. Therefore, the disclosed invention is not necessarily limited to the position, size, range, etc. disclosed in the drawings, etc.

[0021] In addition, in this specification and the like, when describing the configuration of the invention using drawings, the same reference numerals are used in common between different drawings.

[0022] The image display device of the present invention will be described below using examples. However, the present invention is not limited to the following examples. In the following Figs. 1 to 5, the same parts are given the same reference numerals, and their description may be omitted. Furthermore, in the drawings, for the sake of convenience, the structure of each part may be shown in an appropriately simplified manner, and the dimensional ratios of each part may be shown schematically and different from the actual ones.

[0023] 1 shows one embodiment of the configuration of an electronic device of the present invention. Electronic device 100 of this example has information processing unit 110, application execution unit 120, application execution unit 130, cursor control unit 140, image generation unit 150, image display unit 160, and image display unit 170. Note that while electronic device 100 is shown as a single device in FIG. 1, the configuration of the electronic device is not limited to this. Electronic device 100 may also be configured from multiple servers or devices, or a combination thereof, connected via a network.

[0024] The information processing unit 110 is composed of hardware including a CPU, RAM, ROM, etc. (not shown), and software consisting of various programs stored in the ROM. The information processing unit 110 can control (e.g., display control) the image display unit 160 and the image display unit 170 by executing application software. The application software may be stored in the ROM of the information processing unit 110, or may be stored in a storage medium (external HDD, etc.). The application can also be used via a web browser.

[0025] The application execution unit 120 has a function of executing an application 121, and performs various processes based on user processing and input information. Similarly, the application execution unit 130 has a function of executing an application 131, and performs various processes based on user processing and input information.

[0026] The application software 121 and the application 131 may be stored in a storage medium (such as an external HDD) in addition to being stored in the ROM of the information processing unit 110. The application 121 and the application 131 can also be used via a web browser.

[0027] Image generation unit 150 has a function of generating images 161 and 171 based on instructions from application execution unit 120 and application execution unit 130. These images are visually represented on an image display unit or screen, and include photographs, figures, graphs, illustrations, paintings, digital graphics, text, etc. Image display unit 160 and image display unit 170 display images 161 and 171 generated by image generation unit 150, respectively. FIG. 1 shows how information processed by application execution unit 120 is displayed by image display unit 160, and how information processed by application execution unit 130 is displayed by image display unit 170.

[0028] Image 161 has icons, buttons, etc. that can be used to operate application 121. Similarly, image 171 has icons, buttons, etc. that can be used to operate application 131. In other words, application 121 and application 131 can be operated by images 161 and 171 that are generated via image generation unit 150.

[0029] It is preferable that the image display unit 170 can receive touch operations. For example, the image display unit 170 can be a smartphone, a tablet, a personal computer with a touch panel, a touch screen, or the like.

[0030] An aerial display can also be used as the image display unit 170. When an aerial image display device is used as the image display unit 170, the image 171 is projected into the air.

[0031] FIG. 2 is an example of an aerial display that can be used in one embodiment of the present invention. It is a schematic cross-sectional view showing an example of the configuration of an aerial display of the present invention. As shown in FIG. 2, the aerial display 10 of this example includes an image display unit 11, a 3D plate 12, a sensor 14, and a housing 15. The housing 15 is a hollow rectangular parallelepiped with a rectangular opening formed at the top. The 3D plate 12 can form a real image on an aerial imaging surface 13.

[0032] The image display unit 11 has a display surface 11a for displaying an image. As the image display unit 11, for example, a conventionally known image display device such as a liquid crystal display can be used.

[0033] When the image display device 10 is used as the image display unit 170, the electronic device 100 can recognize a touch operation performed on the image 171 by the sensor 14. A general optical sensor, for example, an infrared sensor, can be used as the sensor 14.

[0034] Cursor control unit 140 has a function to control the movement of a cursor displayed on image display unit 160 or image display unit 170. The cursor is a graphic that visually indicates at which position on a screen displayed by an information processing device such as a personal computer an operation is to be performed, and is usually displayed as an arrow, a point, or other shape. Cursor control unit 140 controls the movement and shape of the cursor according to user input, as well as input operations such as user clicks, drags, and drops. Cursor control unit 140 can also recognize the operation of input devices such as a mouse, touchpad, and trackball. This allows the user to operate the cursor using the input device.

[0035] The cursor is displayed on the image display unit 160 or the image display unit 170, and may be displayed on the image 161 or the image 171. Furthermore, the application 121 and the application 131 can be operated by the cursor.

[0036] Usually, one cursor is displayed for one information processing device, and therefore the cursor is displayed on either image display unit 160 or image display unit 170. Therefore, application 121 and application 131 cannot be operated simultaneously using the cursor.

[0037] In one embodiment of the present invention, an application 131 is used to display an image 171, which is an operation screen of the application 131, on the image display unit 170. The image display unit 170 has a function of recognizing a touch operation. Therefore, the application 131 can recognize a touch operation on the image 171 with a finger, a touch pen, or the like. Furthermore, the application 131 has a function of controlling a cursor operation by sending an instruction to the cursor control unit 140. Furthermore, the application 121 can be operated by operating the image 161 on the image display unit 160 with a cursor. In other words, the application 121 can be operated by touching the image 171.

[0038] In other words, application 121 and application 131 can be operated simultaneously.

[0039] Here, it is preferable to use an air display as the image display unit 170. With this configuration, operations that normally require contact with a mouse, touchpad, etc. can be performed contactlessly, and the application 121 can also be operated contactlessly. In other words, by using an air display for the image display unit 170, it becomes possible to operate existing applications contactlessly by touch operation.

[0040] Applications used on personal computers and the like are generally operated by cursor manipulation using a device that requires user contact, such as a mouse. When operating the application without contact, for example, the application's operation screen is directly projected onto a display device with a touch panel or an aerial display, and the application is operated. In this case, there is a risk of malfunction due to the operability of the touch operation. This risk is particularly high when the application requires delicate cursor manipulation. Here, in one aspect of the present invention, the cursor is moved by application 131, and application 121 is operated by the cursor movement. This allows for precise operation. Furthermore, operation without contact and with fewer malfunctions is possible.

[0041] Furthermore, because what is operated by application 131 is a cursor, it is applicable without major design changes and is highly versatile when a different application is executed as application 121. For example, when attempting to operate application 122 (not shown) different from application 121 in a contactless manner, by using one aspect of the present invention, it is not necessary to improve applications 121 and 122, and contactless operation becomes possible while leaving the design of the application as is.

[0042] Cursor operations performed by the cursor control unit 140 include cursor movement, clicking, dragging, dropping, and wheel operation of a mouse. When these operations are performed by touch operation on the screen 171, which is the operation screen of the application 131, for example, cursor movement can be realized by recognizing the finger movement during the touch operation and moving the cursor according to the amount of movement of the finger position, or by displaying an icon on the screen 171, which is the operation screen of the application 131, and implementing a function in the application 131 that moves the icon to specific coordinates or displaces the icon by a specific amount by touching the icon. Similarly, click and drag-and-drop operations can be realized by implementing a function in the application 131 that recognizes the finger movement during the touch operation and converts it into the operation based on the touch operation and contact time on an icon displayed on the image 171. Note that the method of converting a touch operation on the screen 171 into cursor operation is not limited to these.

[0043] 1 shows a state in which a cursor is operated by operating screen 171, which is an operation screen for application 131, with a finger or the like, and application 121 is operated by the operated cursor, but a plurality of applications may be operated. In this case, an image capable of operating a plurality of applications is displayed in image 171. Note that the image capable of operating a plurality of applications is generated based on information processed by application execution unit 130 via image generation unit 150.

[0044] For example, electronic device 100 may further include application execution unit 180 (not shown) having a function of executing application 181 (not shown), and image generation unit 150 may further generate image 162 (not shown) based on an instruction from application execution unit 180. In this case, application 181 can be operated by operating image 171.

[0045] 1 shows the image 161 and the image 171 displayed on the image display units 160 and 170, respectively, but one aspect of the present invention is not limited to this. For example, the image 161 and the image 171 may be displayed simultaneously on the image display unit 170. That is, a plurality of images may be displayed on one image display unit, and an image of an application for operating a cursor and an image of an application operated by a cursor operated by operating the application may be displayed on one image display unit.

[0046] Furthermore, to improve visibility, when image 171 is touched with a finger or the like, a display indicating that the touch operation has been recognized may be displayed on the part of image 171 where the touch is detected. For example, the part where the touch is detected may be displayed surrounded by a symbol such as a circle or a square.

[0047] To improve visibility, when an icon or button displayed on image 171 is touched, the frame of the icon or button may be highlighted. Alternatively, the frame of the entire screen 171 may be highlighted. For example, when it is recognized that an icon has been touched, the frame of the icon may flash or change color.

[0048] To improve recognition, electronic device 100 may be configured to emit a sound when an icon or button displayed on image 171 is touched.

[0049] The above-mentioned function of increasing visibility and recognizability is possessed by the application 131 and executed by the application execution unit 130.

[0050] Note that the image 171 can also be operated by a normal cursor operation. That is, the image 171 can be operated by both a normal cursor operation and a touch operation. [Example]

[0051] Next, with reference to FIGS. 3 to 5, an example will be described in which a cursor moves and application 121 is operated by performing a touch operation on the operation screen of application 131 displayed on image display unit 170.

[0052] The operation screens 71 displayed in FIGS. 3a1, 4a1, and 5a1 each have icons 77 to 79, and the icons 77, 78, and 79 are selectable to distinguish the operation of the cursor 41 in FIGS.

[0053] 3a1, b1, c1, and d1 are examples of the operation screen of application 131 displayed as image 171, and FIGS. 3a2, b2, c2, and d2 are examples of image 161 in which the functions of application 121 are displayed via image generation unit 150. In FIGS. 3a1, b1, c1, d1, a2, b2, c2, and d2, similar figures are denoted by similar reference numerals and are not shown.

[0054] 3a1 shows an operation screen 71 of the application 131, and FIG. 3a2 shows an operation screen 61 of the application 121. The operation screen 61 has icons 62 to 66 and an image 67 and an image 68.

[0055] 3a1, a2, b1, and b2 show how operation screen 71 of application 131 switches to operation screen 80 shown in FIGS. 3b1, c1, and d1 when icon 77 in FIG. 3a1 is clicked by touch operation with finger 200. At this time, the position of cursor 41 shown in FIGS. 3a2 and 3b2 does not change. In FIGS. 3a1, a2, b1, and b2, the cursor is shown displayed in FIGS. 3b1 and b2, i.e., on image 161 in FIG. 1, but it may also be on FIGS. 3a1 and a2, i.e., on image 171 in FIG. 1.

[0056] Operation screen 80 has icons 72 to 76. Operation screen 80 is a screen for operating operation screen 61 via application 131, and the icons displayed on operation screen 61 and the icons on operation screen 80 show the same figures. That is, icons 62 and 72 have the same design, icons 63 and 73 have the same design, icons 64 and 74 have the same design, icons 65 and 75 have the same design, and icons 66 and 76 have the same design. Here, it is preferable that the display size of icons 72 to 76 is larger than that of icons 62 to 66. This configuration facilitates touch operation of icons 72 to 76, which will be described later, and enables easy operation of operation screen 61 without contact.

[0057] 3c1 and 3c2 show how clicking icon 72 on operation screen 80 by touch operation with finger 200 moves cursor 41 to the position of icon 62, and then clicks icon 62. This is possible by setting application 131 so that when icon 72 is clicked, cursor 41 moves to the coordinates where icon 62 is displayed, and then a click operation is performed. Setting application 131 to operate so that cursor 41 moves and clicks can enable application 121 to operate without contact and without making any modifications to application 121.

[0058] 3d1 and 3d2 also show how clicking icon 75 on operation screen 80 by touch operation moves the cursor to the position of icon 65, and icon 65 is then clicked. When multiple buttons or icons are displayed on operation screen 61 as in FIGS. 3c1, c2, d1 and 3d2, creating corresponding buttons or icons on operation screen 80 allows each button or icon to be operated precisely and without contact.

[0059] Although operation examples of the icons 72 and 75 are shown in FIGS. 3c1, 3c2, 3d1 and 3d2, similar functions can also be assigned to the icons 73, 74 and 76.

[0060] In this embodiment, the icons 72 to 76 have the same designs as the corresponding icons 62 to 66, but this is not a limitation of the present invention. The icons or buttons displayed on the operation screen 61 and the icons or buttons displayed on the operation screen 80 may have different display formats.

[0061] Furthermore, in this embodiment, the icons 62 to 66 are shown to be operated by operating the operation screen 80, but if URLs or links, etc. are arranged on the operation screen 61, icons or buttons corresponding to these URLs or links, etc. can be similarly arranged on the operation screen 80, thereby enabling the URLs, links, etc. on the operation screen 61 to be operated non-contact and precisely. [Example]

[0062] Next, a detailed description will be given of how the image 67 displayed on the operation screen 61 is operated by operating the application 131 using FIGS. 4a1, b1, c1, d1, a2, b2, c2, and d2.

[0063] 4a1, a2, b1, and b2 show how clicking on icon 78 in operation screen 71 by touch operation with finger 200 moves cursor 41 to the position of image 67 in operation screen 61, and furthermore, image 71 switches to image 90. In other words, clicking on icon 78 simultaneously performs the operation of switching image 71 to image 90 and the operation of moving cursor 41 onto image 67.

[0064] 4c1 and 4c2 show the image 67 being rotated by operating icon 91a or 91b on the operation screen 90 with finger 200. Clicking icon 91a or 91b rotates image 67 a certain amount. The direction and amount of rotation may be set appropriately taking into consideration operability and visibility. Operability can be improved by allowing rotation operations in opposite directions to be operated with icon 91a and icon 91b. For example, operating icon 91a is set to rotate left, and operating icon 91b is set to rotate right. It is preferable that the absolute value of the rotation amount be the same for icons 91a and 91b. The function settings of icons 91a and 91b may be reversed.

[0065] 4d1 and d2 show how the image 67 is rotated by operating the slider 92 on the image 90 with the finger 200. The rotation of the image 67 can be controlled by operating the slider 92 left and right. The direction and amount of rotation of the image 67 by operating the slider 92 may be set appropriately taking into consideration operability and visibility. The slider 92 can be operated continuously by touch operation, and can also be operated by dragging.

[0066] In order to rotate the image 67, the cursor 41 must be positioned on the image 67. In this embodiment, when the icon 78 is operated, the cursor 41 moves onto the image 67. Therefore, the image 67 can be rotated by operating the icons 91a and 91b or the slider 92, without operating the cursor 41 separately.

[0067] Typically, the rotation of the image 67 is performed by operating the wheel or drag of a mouse attached to the outside of the electronic device 100. However, in one aspect of the present invention, a similar function is realized by operating icons 91a and 91b or a slider 92 displayed by an application. This configuration allows the rotation to be performed without contact. Furthermore, by displaying operation icons, i.e., icons 91a, 91b and slider 92, on the operation screen 90, the icons can be displayed larger, thereby improving operability, which is a concern when operating a touch panel or an aerial display.

[0068] In this embodiment, the image 67 is shown rotating horizontally, but the image 67 can also be rotated vertically. In this case, although the icons 91a and 91b are displayed horizontally in FIGS. 4b1, b2, c1, c2, d1, and d2, they can also be displayed vertically, similar to the rotation direction of the image 67. Similarly, the slider 92 is displayed with a horizontal bar, but a vertical bar can also be displayed. By unifying the rotation direction and the icon display direction, malfunctions can be suppressed and operability can be improved. [Example]

[0069] Next, FIGS. 5a1, b1, c1, d1, a2, b2, c2, and d2 will explain in detail how the image 68 displayed on the operation screen 61 is operated by operating the application 131.

[0070] 5 illustrates the scanning of image 68 in a uniaxial direction of a three-dimensional object, in other words, the sequential display of uniaxial cross-sectional views of the three-dimensional object, when image 68 is part of the continuous cross-sectional view of the three-dimensional object. In particular, this embodiment illustrates scanning in the z-axis direction as an example of a uniaxial direction. An example of a continuous cross-sectional view of a three-dimensional object is a CT (computed tomography) image. Therefore, one aspect of the present invention can be applied to the analysis of CT images.

[0071] 5a1, a2, b1 and b2, clicking on an icon 79 in the operation screen 71 of the application 131 simultaneously switches the image 71 to an image 90 and moves the cursor 41 onto the image 68.

[0072] Figures 5c1 and 5c2 show how image 68 is scanned in the depth direction, i.e., the z-axis direction, by operating icon 91a or 91b on image 90. Clicking icon 91a or 91b can display a cross-sectional view at a position where the z-coordinate of the three-dimensional object has been displaced by a certain amount. The direction and amount of displacement may be set appropriately, taking into consideration operability and visibility. Operability can be improved by setting icon 91a and icon 91b to displace in opposite directions. For example, setting icon 91a to displace forward, and setting icon 91b to displace backward. It is preferable that the absolute values ​​of the displacement amounts are the same for icons 91a and 91b. The displacement direction settings for icons 91a and 91b may be reversed from those described above.

[0073] 5d1 and 5d2 show how the image 68 is scanned in the z-axis direction by operating the slider 92 on the image 90 with the finger 200. By operating the slider 92 left and right, a cross-sectional view at a position where the z coordinate of the three-dimensional object has been displaced by a certain amount can be displayed. The direction and amount of displacement may be set appropriately taking into consideration operability and visibility. The slider 92 can be operated continuously by touch operation, and can also be operated by dragging.

[0074] In this embodiment, the image 68 is scanned in the z-axis direction, but the scan direction is not limited to this. Scanning is also possible in the x-axis or y-axis direction.

[0075] Typically, scanning of image 68 is performed by wheel operation or drag operation of a mouse attached to the outside of electronic device 100. On the other hand, in one aspect of the present invention, a similar function is realized by operating icons 91a and 91b or slider 92 displayed by application 131. With this configuration, the scanning can be performed without contact. Furthermore, by displaying operation icons, i.e., icons 91a, 91b and slider 92, on operation screen 90, the icons can be displayed larger, thereby improving operability, which is a concern when operating a touch panel or an aerial display.

[0076] 4 and 5 show examples of rotating or scanning the image 67 or 68 on the operation screen 61 by the application 131, but the operable actions are not limited to these.

[0077] Furthermore, the detailed configurations, detailed operations, display screens, and operation screens of the components constituting the display devices in the above-described embodiments and examples can be modified as appropriate without departing from the spirit of the present invention. [Explanation of symbols]

[0078] 10. Aerial Display 11 Image display section 11a Display surface 12 3D plates 13 Image plane 14 sensors 15 Case 41 Cursor 61 Operation screen 62 icons 63 Icons 64 icons 65 icons 66 icons 67 images 68 images 71 Operation screen 72 icons 73 Icons 74 icons 75 icons 76 icons 77 Icons 78 Icons 79 Icons 80 Operation screen 90 Operation screen 91a Icon 91b Icon 92 Slider 100 Image display device 110 Information Processing Department 120 Application Execution Unit 121 Applications 130 Application Execution Unit 131 Applications 132 Applications 140 Cursor control section 150 Image generation unit 160 Image display unit 161 images 170 Image display unit 171 images 180 Application Execution Unit 181 Applications 200 fingers

Claims

1. A first display unit and a second display unit are included, a cursor is displayed on the first display unit or the second display unit, the second display unit is touch operable, the second display unit displays a first image on which the cursor can be operated; The image display device, wherein the cursor is operated on the first display unit by a touch operation on the first image.

2. In claim 1, the first image is capable of operating a first application, and the second image on the first display is capable of operating a second application; The image display device, wherein the touch operation moves the cursor onto the second image or performs an operation on the second image.

3. In claim 2, The image display device, wherein the second display unit is an aerial display.

4. In claim 3, An image display device, wherein the manipulation of the second image is clicking, rotating, enlarging or reducing the second image.

5. In claim 4, An image display device, wherein the first image has an icon, a button, or a slider that can be operated by a wheel of the cursor.

6. In claim 5, An image display device in which the clicking, rotating, enlarging or reducing is performed by operating the icon, the button or the slider.

7. In claim 2, The image display device, wherein the manipulation of the second image is scanning of the second image.

8. In claim 7, An image display device, wherein the second image is a cross-sectional view of a three-dimensional object in one axial direction.

9. In claim 8, An image display device, wherein the first image has an icon, a button, or a slider that can be operated by a wheel of the cursor.

10. In claim 9, An image display device in which the scanning is performed by operating the icon, the button, or the slider.

Images