Input display device

Abstract

To provide an input display device for improving the visibility of an image to be displayed within a range in which an operation part of a three-dimensional shape exists.SOLUTION: An input display device 100 according to the present invention includes a display 110 that can display an image, a touch panel 120 including at least one transparent three-dimensional UI part 130 on the surface, a touch detection unit 150 for detecting touch operation to the touch panel on the basis of a change in electrostatic capacity of the touch panel 120, and a display control unit 170 that can display an image related to the operation in an area in which the three-dimensional UI parts 130 of the display 110 exist. The three-dimensional UI parts 130 include a top face and a side face connected to the top face, and a function for reflecting the image related to the operation is given to the side face.SELECTED DRAWING: Figure 3

Description

Technical Field

[0006] , ,

[0005] ,

[0001] The present invention relates to an input display device having an interface function between a human and a machine, and particularly to an input display device including a transparent three-dimensional operation unit.

Background Art

[0002] An input display device is disclosed in which convex portions are provided on a touch panel disposed so as to overlap a display, and an image such as an operation icon is displayed at a position overlapping the convex portions (for example, Patent Document 1). A user inputs by performing a touch operation on the convex portions.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a display device that performs a capacitive touch operation, a user interface (hereinafter referred to as a three-dimensional UI) has been proposed in which the touch position can be tactilely recognized by providing uneven shapes on a cover glass, and the touch position can be understood without looking.

[0005] FIG. 1(A) is an operation example of a conventional flat touch panel. A user U visually recognizes an operation icon 12 displayed on a display 10 and inputs by performing a touch operation on the position of the operation icon 12 (the example in the figure is a musical note).

[0006] Figure 1(B) shows an example of operation of a touch panel with a three-dimensional UI, and Figure 1(C) is a schematic cross-sectional view of the three-dimensional UI. A transparent cover lens 26 with an uneven surface is attached to the touch panel 24, and the display 20 displays operation icons 22 in a position that overlaps with the cover lens 26. User U performs input by touching the cover glass 26 with their finger. A highly sensitive electrostatic sensor is used for touch detection, which can detect the capacitance (distance) of the finger even when it is far from the sensor, making it possible to determine whether or not a touch has been made even through the thick cover lens 26. This makes it possible to reduce touch errors (operation errors) in situations where it is difficult to pay attention to the in-vehicle display while driving.

[0007] When displaying GUI (Graphical User Interface) elements such as icons within a 3D UI, the visual representation is limited to a specific area depending on the part design. Figure 2(A) shows an example where a speaker icon 32 representing volume adjustment and a gauge (scale) 34 indicating the amount of rotation of the knob 30 are displayed within the area where a cylindrical transparent knob (button) 30 exists. The user rotates the knob 30 by sliding their fingertip along its side and confirms the amount of rotation with the gauge 34. The knob 30 does not actually rotate.

[0008] Because the display area is limited to the range of the knob 30, there are limitations to designing a larger speaker icon 32. Also, if the knob diameter is small, the gauge 34 becomes small, making it impossible to ensure sufficient visibility.

[0009] As shown in Figure 2(A), when the knob 30 is viewed from the front, the speaker icon 32 and gauge 34 are clearly visible. However, as shown in Figure 2(B), when the knob 30 is viewed from the diagonal right, the speaker icon 32 and gauge 34 become smaller, reducing their visibility. For example, when the display is positioned in the center of the vehicle, the driver's or other user's line of sight will be diagonally to the right.

[0010] Such challenges can arise not only with knob shapes and gauge representations, but also when representing GUIs on three-dimensional parts with height. Therefore, in the visual representation of three-dimensional UIs, methods are needed to ensure visibility within the three-dimensional parts.

[0011] The present invention aims to solve these conventional problems and to provide an input display device that improves the visibility of images displayed within the area where a three-dimensional operating part exists. [Means for solving the problem]

[0012] The input display device according to the present invention includes a display for displaying an image, a capacitive touch panel mounted on the display and having at least one transparent three-dimensional operating section on its surface, detection means for detecting touch operations including proximity to the touch panel, and display control means capable of displaying an image related to the operation in the area of ​​the display where the operating section is located, wherein the operating section includes a top surface and a side surface connected to the top surface, and the side surface is provided with a function for reflecting the image related to the operation.

[0013] In one embodiment, the side surface includes a mirror-finished surface. In another embodiment, the side surface includes a reflective film or a mirror coating. In another embodiment, the side surface includes a film with a reflective surface on the back side and a texture on the front side. In another embodiment, the input display device further includes detection means for detecting the user's viewpoint or line of sight, and the display control means changes the image relating to the operation according to the viewpoint or line of sight detected by the detection means. In another embodiment, the display control means expands the image reflected on the side surface by expanding the image relating to the operation. In another embodiment, the three-dimensional operating part is a cylindrical knob, the detection means detects the amount of rotation of the knob, and the display control means displays a gauge corresponding to the detected amount of rotation along the outer circumference of the knob, and the side surface of the knob reflects the gauge. In another embodiment, the display is mounted in a vehicle, and the detection means detects the line of sight of the driver and / or passengers. [Effects of the Invention]

[0014] According to the present invention, the side surface of the three-dimensional operating part is provided with a function to reflect images related to the operation, thereby improving the visibility of images related to the operation and enabling greater diversity in the representation of images related to the operation. [Brief explanation of the drawing]

[0015] [Figure 1] Figure 1(A) shows an example of operation of a flat touch panel, Figure 1(B) shows an example of operation of a touch panel with a 3D UI, and Figure 1(C) is a schematic cross-sectional view of the 3D UI. [Figure 2] This diagram illustrates the challenges of conventional 3D UI designs. [Figure 3] This is a block diagram showing the configuration of an input display device according to an embodiment of the present invention. [Figure 4] Figures 4(A) and 4(B) illustrate the incidence of light when viewing a GUI image in a conventional 3D UI unit, while Figures 5(A) and 5(B) illustrate the incidence of light when viewing a GUI image in the 3D UI unit of this embodiment. [Figure 5] This figure shows an example of hiding / showing a GUI image in a 3D UI section according to an embodiment of the present invention. [Figure 6] This figure shows another example of the configuration of the three-dimensional UI section according to an embodiment of the present invention. [Figure 7] This is a block diagram showing the configuration of an input display device according to a second embodiment of the present invention. [Figure 8] This is an example of the layout when the input display device of the second embodiment is mounted on a vehicle. [Figure 9] Figure 9(A) shows an example where a gap appears in the GUI image on the side when the user's viewpoint or line of sight changes, Figure 9(B) shows an example of a GUI image expanded based on the viewpoint or line of sight according to the second embodiment, and Figure 9(C) shows how the GUI image expanded according to the viewpoint or line of sight according to the second embodiment is reflected on the side. [Figure 10]A diagram for explaining an example of a line-of-sight detection unit according to a second embodiment of the present invention.

Embodiments for Carrying Out the Invention

[0016] Next, embodiments of the present invention will be described. The input display device of the present invention provides an interface between a person and a machine. The input display device of the present invention is not particularly limited, but for example, it is applied to an electronic device equipped with a display with a touch panel on which a three-dimensional operation unit is formed. An electronic device equipped with a display with a touch panel is, for example, an in-vehicle device having a navigation function, an audio-visual function, a television function, and the like.

Example

[0017] Next, the embodiments of the present invention will be described in detail with reference to the drawings. FIG. 3 is a block diagram showing the configuration of an input display device according to an embodiment of the present invention. The input display device 100 of the present embodiment includes a display 110 for displaying images and videos, a touch panel 120 including a capacitance-type sensor mounted on the display 110, one or more transparent three-dimensional UI parts (or operation parts) 130 having a three-dimensional shape attached to the surface of the touch panel 120, and a controller 140 for controlling image display on the display 110, touch detection on the touch panel 120, and the like.

[0018] The display 110 is not particularly limited, but for example, it includes a liquid crystal panel or an organic EL panel and displays image data provided from the controller 140. For example, in the area where the three-dimensional UI part 130 exists, GUI images (for example,a variety of images such as icons and gauges) related to the operation of the three-dimensional UI part 130 are displayed.

[0019] The touch panel 120 includes, for example, multiple sensors (detection units) formed at the intersection of multiple X-side and Y-side electrode lines, and these sensors detect capacitance when a user's finger or hand approaches or touches the touch panel 120 or the 3D UI unit 130. The touch panel 120 is mounted on the display 110 and provides an input interface for the user to input icons and the like displayed on the display 110.

[0020] The touch panel 120 further includes one or more three-dimensional UI sections 130 on a transparent panel. The three-dimensional UI section 130 is made of a transparent material having an uneven shape, and its shape, size, height, etc., are not particularly limited, but the three-dimensional UI section 130 is configured such that the capacitance at the position changes when the user's hand or fingers approach or touch the three-dimensional UI section 130.

[0021] Figure 3 shows an example in which four 3D UI units 130 are mounted on the lower part of the touch panel 120. GUI images such as icons and gauges representing operations are displayed in the area of ​​the display 110 where the 3D UI units 130 are located, and the user views the GUI images and makes inputs via the 3D UI units 130.

[0022] The three-dimensional UI section 130 is, for example, a protruding knob or button in the shape of a rectangle or cylinder, and is made of a clear part (transparent material) such as glass, acrylic, or polycarbonate. However, characters, figures, etc., may be drawn on a part of the surface of the three-dimensional UI section 130, to the extent that it does not affect the visibility of the GUI image displayed on the display 110. The three-dimensional UI section 130 may be attached to the surface of the touch panel 120 at a predetermined position using, for example, double-sided adhesive, or it may be integrally molded with a cover glass attached to the surface of the touch panel 120.

[0023] Information regarding the position and shape of the 3D UI unit 130 (for example, coordinates on the touch panel, shape and size of the base and surface, height, etc.) is registered in the memory of the controller 140. For example, if the 3D UI unit 130 is a cylindrical knob, the coordinates of the center of the base of the knob are registered as position information. In addition, the radius or diameter of the base of the knob, the radius or diameter of the surface, and the height of the sides are registered as shape information. If the 3D UI unit 130 is a rectangular prism-shaped knob, the coordinates of the position where the diagonals of the rectangular base intersect are registered as position information, and the length and width of the base, the length and width of the surface, and the height of the sides are registered as shape information. When a finger approaching the touch panel 120 is detected, the controller 140 refers to the registered information regarding the position and shape of the 3D UI unit 130 and determines whether the finger is performing a touch operation on the 3D UI unit 130.

[0024] The controller 140 includes hardware and / or software resources, and is responsible for all processing of the input display device, for example, by using a microcontroller including an arithmetic processing unit and ROM / RAM. For example, it performs display processing for the display 110, touch detection and touch operation determination (detection of touch position and finger distance, determination of whether a touch operation has been performed) from the output values ​​of the touch panel 120, and processes such as video display / video switching accordingly.

[0025] As shown in Figure 3, the controller 140 is composed of a touch detection unit 150, an operation determination unit 160, and a display control unit 170. The touch detection unit 150 drives multiple electrode lines on the X and / or Y sides of the touch panel 120, measures the capacitance of the detection unit of the driven electrode lines, detects the touch position based on the measurement result, and provides the detection result to the operation determination unit 160.

[0026] The operation determination unit 160 determines whether a touch operation has occurred on the touch panel 120 or the 3D UI unit 130 based on the measurement results of the touch detection unit 150. Here, "touch" includes not only contact between the user's finger and the touch panel 120 or the 3D UI unit 130, but also approaching the touch panel 120 or the 3D UI unit 130. For example, when the user's finger contacts or approaches the touch panel 120, the operation determination unit 160 determines whether a touch operation has occurred based on the change in capacitance of the corresponding detection unit. Similarly, when the user's finger contacts or approaches the 3D UI unit 130, the operation determination unit 160 determines whether a touch operation has occurred on the 3D UI unit 130 based on the change in capacitance of the corresponding detection unit.

[0027] The display control unit 170 displays images and videos on the display 110, and displays GUI images related to the operation of the 3D UI unit 130 in the area where the 3D UI unit 130 is located. The display control unit 170 also switches the image displayed on the display 110 to a different image when the operation determination unit 160 determines that a touch operation has occurred.

[0028] Next, an overview of the input display device of this embodiment will be described with reference to Figure 4. Figures 4(A) and (B) schematically show the view of a GUI image displayed in the area where a conventional 3D UI unit exists, and Figures 4(C) and (D) schematically show the view of a GUI image displayed in the area where the 3D UI unit of this embodiment exists. Here, a cylindrical knob is used as an example of the 3D UI unit, and a gauge (see gauge 34 in Figure 2) is used as an example of the GUI image.

[0029] As shown in Figure 4(A), when viewing the GUI image 180 displayed on the display 110 in a conventional 3D UI unit 30, light L1 (shown by a dashed line) from the GUI image 180 passes through the 3D UI unit 30 and enters the user's viewpoint P1. Furthermore, as shown in Figure 4(B), when the user's viewpoint P1 moves to viewpoint P2, light L2 (shown by a dashed line) from the GUI image 180 enters viewpoint P2 at a greater angle than light L1. As a result, the user views the GUI image 180 from an oblique angle, causing the GUI image 180 to appear smaller and reducing visibility.

[0030] In contrast, the three-dimensional UI section 130 of this embodiment utilizes the height of the side surface of the three-dimensional UI section and provides a function to specularly reflect the GUI image 180 onto its side surface 132. The three-dimensional UI section 130 has a surface 132, a bottom surface 134, and a side surface 136 connecting the surface 132 and the bottom surface 134, and a function to reflect the GUI image 180 onto part or all of the side surface 132 is added.

[0031] As shown in Figure 4(C), when viewing the GUI image 180 displayed on the display 110 in the three-dimensional UI unit 130 of this embodiment, in addition to the light L1 from the GUI image 180, light L1a that has been specularly reflected off the side surface 136 from the GUI image 180 is incident on the user's viewpoint P1. Similarly, as shown in Figure 4(D), when the user's viewpoint P1 moves to viewpoint P2, in addition to the light L2 from the GUI image 180, light L2a that has been specularly reflected off the side surface 136 is incident on viewpoint P2. As a result, the user views the GUI image 180 not only with light L1 and L2 from the GUI image 180, but also with light L1a and L2a reflected off the side surface 136 of the three-dimensional UI unit 130, thus expanding the display area of ​​the GUI image 180 compared to conventional displays and improving visibility.

[0032] Figure 5(A) shows an example where the GUI image (gauge) of the 3D UI section of this embodiment is hidden, and Figure 5(B) shows an example where the GUI image (gauge) of the 3D UI section of this embodiment is displayed. Here, in order to achieve more accurate specular reflection of the GUI image, the side surface 136 of the 3D UI section 130 is made vertical, and an example is shown in which the side surface 136 is made of a mirror-finished transparent part (made of acrylic).

[0033] As shown in Figure 5(A), a volume icon 182 is displayed approximately in the center of the area where the 3D UI section 130 is located. At this time, the gauge indicating the amount of volume rotation is not displayed, and no image is displayed on the side surface 136 due to the reflection of the gauge. On the other hand, as shown in Figure 5(B), when the gauge (GUI image) 184 is displayed on the display 110 along the outer circumference of the bottom surface 134 of the 3D UI section 130, the gauge 184 is specularly reflected by the side surface 136 of the 3D UI section 130, and the reflected image 184A is visible. As a result, when the 3D UI section 130 is viewed from an oblique direction, the display area of ​​the gauge 184 is extended to the side surface 136, improving visibility.

[0034] In the above example, the side surface 136 of the three-dimensional UI part 130 was given a mirror finish, meaning the part itself was given a mirror finish. However, this is just one example, and other methods may be used to reflect the GUI image. For example, in the three-dimensional UI part 130A shown in Figure 6(A), a mirror-like glossy film or reflective film 190 may be wrapped around the side surface 136 of the knob 130, or in the three-dimensional UI part 130B shown in Figure 6(B), a mirror coating 194 may be applied to the side surface 136. Furthermore, as shown in Figure 6(C), a mirror or reflective surface 192 may be formed on the back side of the film 190, and a frosted glass-like texture 194 may be formed on the front side for tactile feel. This makes it possible to achieve both tactile feel and a mirror finish on both sides of the film 190.

[0035] Next, a second embodiment of the present invention will be described. Figure 7 is a block diagram showing the configuration of the input display device of the second embodiment, and the same reference numerals are used for components identical to those in Figure 3. The input display device 100A according to the second embodiment further includes a gaze detection camera 200 that detects the user's viewpoint or gaze direction (hereinafter referred to as viewpoint) and a gaze detection unit 210.

[0036] Figure 8 shows an example layout when the input display device 100A is mounted in a vehicle. The display 110 with a touch panel 120 is mounted on the central console of the vehicle, and the gaze detection camera 200 is mounted, for example, close to and above the display 110. Two gaze detection cameras 200 may be installed to detect the viewpoints of both the driver and the passenger in the front seat.

[0037] The gaze detection camera 200 captures the driver's face and provides the captured image data to the gaze detection unit 210 of the controller 140. The gaze detection unit 210 detects the driver's viewpoint by processing the image data from the gaze detection camera 200. The gaze detection unit 210 does this, for example, by extracting feature points of the eyeballs from the image data of the face. Based on the relative positional relationship between the display 110 and the driver's face (for example, the relative positional relationship is known from seat position information and standard human physical characteristics), the gaze detection unit 210 calculates the driver's viewpoint relative to the 3D UI unit 130.

[0038] If the gaze of a passenger in the front passenger seat is to be detected, the gaze detection camera 200 provides the gaze detection unit 210 with image data of the passenger's face. Furthermore, if the vehicle is equipped with a driver monitoring system (DMS) for monitoring the driver's condition, information regarding the driver's gaze provided by the DMS may be used instead of the gaze detection camera 200.

[0039] The display control unit 170 controls the display of the GUI image based on the viewpoint detected by the gaze detection unit 210 so that the reflection of the GUI image to the user's viewpoint is appropriate. For example, as shown in Figure 9(A), when the user's viewpoint changes (for example, when the viewpoint moves further to the right), if the size of the reflected image 184A of the GUI image does not match the size of the side surface 136, a gap (blank area) B will be created in a part of the upper edge of the side surface 136, reducing the expansion effect of the GUI image and decreasing visibility.

[0040] In this embodiment, the display control unit 170 changes the GUI image according to the user's viewpoint, so that the visibility of the GUI image does not decrease even when the user's viewpoint changes. The display control unit 170 calculates, from the user's viewpoint, where and how large a gap B will occur on the side surface 136. The calculation method is not particularly limited, but the display control unit 170 may calculate the gap B from, for example, the detected viewpoint, the display position and size of the GUI image on the display, the position of the 3D UI section 130, the position and height of the side surface 136, or it may simply register the relationship between the viewpoint and the gap in a lookup table or the like and refer to it.

[0041] The display control unit 170 controls the display of the GUI image so that some or all of the calculated gap is eliminated, that is, so that the GUI image in the positional relationship between the calculated gap and the mirror surface is expanded. For example, as shown in Figure 9(B), the display control unit 170 causes the expanded GUI image 186 to be displayed on the display 110. As a result, as shown in Figure 9(C), the expanded GUI image 186 is mirror-reflected by the side surface 136, and the user can see the expanded GUI image 186A reflected by the side surface 136. The expanded GUI image 186A eliminates at least a portion of the gap B of the side surface 136, and the visibility of the GUI image can be maintained even if the user's viewpoint changes.

[0042] Furthermore, in the Driver Monitoring System (DMS), if it is possible to detect the gaze of both the driver and the passenger in the front seat, the DMS will detect the gaze of the person whose eyes are directed towards the display 110. If both are looking at the display at the same time, the DMS will prioritize detecting the driver's gaze for safety reasons.

[0043] Figure 10 shows a method for detecting the gaze of both the driver and the passenger. Figures 10(A) and (B) show the positional relationship between the display 110 and the driver U1 and passenger U2 when viewed from above. In Figure 10(A), a lenticular lens 112 is placed in front of the display 110, and in Figure 10(B), a parallax barrier 114 is placed in front of the display 110. By using the lenticular lens 112 and the parallax barrier 114, a structure is created that separates the displayed image depending on the viewpoint, providing two types of video output to the driver and passenger respectively. In the example shown in the figure, the driver U1 can see the image indicated as "right," and the passenger U2 can see the image indicated as "left." This makes it possible to control the display of GUI images that are suitable for the gaze or direction of each of the driver and passenger.

[0044] In the above embodiment, a cylindrical knob (button) was used as an example of the three-dimensional UI element, but the three-dimensional UI element is not limited to this and may take on other shapes. Also, the sides of the three-dimensional UI element do not necessarily have to be perpendicular to the bottom / top surface and may be inclined. In the above embodiment, a volume icon and gauge were used as examples of GUI images, but these are just examples, and the GUI image may be an icon or display related to other operations.

[0045] Although preferred embodiments of the present invention have been described in detail above, the present invention is not limited to any particular embodiment, and various modifications and changes are possible within the scope of the gist of the invention as described in the claims. [Explanation of Symbols]

[0046] 100: Input display device 110: Display 120: Touch panel 130: 3D UI section (operation section) 132: Surface 134: Bottom 136: Side view 180, 184: GUI images 184A: Reflected GUI image 186: Extended GUI image 186A: Reflected extended GUI image

Claims

1. A display for displaying images, A capacitive touch panel mounted on the aforementioned display, having at least one transparent three-dimensional operating section on its surface, A detection means for detecting touch operations, including proximity to the touch panel, The area of ​​the display where the operation unit is located includes a display control means capable of displaying an image related to the operation, The operation unit includes a top surface and a side surface connected to the top surface, and the side surface is provided with a function to reflect images related to the operation, in an input display device.

2. The input display device according to claim 1, wherein the aforementioned side surface includes a mirror-finished surface.

3. The input display device according to claim 1, wherein the aforementioned side surface includes a reflective film or a mirror coating.

4. The input display device according to claim 1, wherein the aforementioned side surface includes a film having a reflective surface on the back side and a texture on the front side.

5. The input display device further includes detection means for detecting the user's viewpoint or gaze direction. The input display device according to claim 1, wherein the display control means changes the image related to the operation according to the viewpoint or line of sight direction detected by the detection means.

6. The input display device according to claim 5, wherein the display control means expands the image reflected on the side surface by expanding the image relating to the operation.

7. The aforementioned three-dimensional operating part is a cylindrical knob, The detection means detects the amount of rotation of the knob, The display control means displays a gauge corresponding to the detected amount of rotation along the outer circumference of the knob. The side surface of the knob reflects the gauge, as described in claim 1.

8. The input display device according to claim 5, wherein the display is mounted on a vehicle, and the detection means detects the gaze position of the driver and / or passenger.

Images