Display system, control device and method of vibration output device, and program

Abstract

To allow reliable focusing on an object intended by a user.SOLUTION: A display system comprises a display device 100 which synthesizes and displays an object on an image in a three-dimensional space, and a vibration output device 101 which performs vibration output. The display system comprises control means 111 which controls the vibration output device 101 to perform the vibration output when the object is focused by a focusing operation for the display device 100. When the focusing operation is performed for a plurality of objects A and B which overlap each other when viewed from a user, the control means 111 changes the vibration output according to a position of focus.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a display system, a control device for a vibration output device, a method, and a program. [Background technology]

[0002] In mixed reality (MR) technology and virtual reality (VR) technology that use a head-mounted display (HMD), some technologies enable a focus operation to focus on an object displayed on the HMD. If vibration output is provided as feedback that the user has focused on an object through the focus operation, the user can easily recognize that they are focusing on an object.

[0003] Patent Document 1 discloses a technology for vibrating the contact position at a vibration magnitude corresponding to a parameter amount of data associated with an object in a touch panel on which a user directly touches a display screen to operate an object displayed on the screen. In Patent Document 1, when a moving object abuts against another object, the other object and the moving object are moved together, and the contact position is vibrated at a vibration magnitude corresponding to the total parameter amount of both objects moving together.

[0004] Patent Document 2 discloses a technique in which a haptic effect is determined based at least in part on a virtual object, and a haptic output device is configured to output the haptic effect. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2010-15239 [Patent Document 2] Japanese Patent Publication No. 2020-42827 Summary of the Invention [Problem to be solved by the invention]

[0006] When a plurality of objects overlap from the user's perspective, it is desired to enable the user to reliably focus on the object that the user intends. The techniques disclosed in Patent Documents 1 and 2 do not provide support for reliably enabling the user to reliably focus on the object that the user intends when a plurality of objects overlap from the user's perspective.

[0007] The present invention has been made in view of the above-mentioned points, and has as its object to enable a user to reliably focus on an intended object. [Means for solving the problem]

[0008] The display system of the present invention is a display system comprising a display device that synthesizes and displays an object into an image in three-dimensional space, and a vibration output device that outputs vibration, and is further provided with a control means that controls the vibration output device to output vibration when an object is focused on by a focus operation on the display device, and is characterized in that when multiple objects that overlap as seen by the user are focused on, the control means changes the vibration output depending on the position of the focus. [Effects of the Invention]

[0009] According to the present invention, it becomes possible to reliably focus on an object intended by the user. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a diagram showing a configuration of a display system according to a first embodiment. [Figure 2] FIG. 1 is a diagram illustrating a functional configuration of a display system according to a first embodiment. [Figure 3] 4 is a flowchart showing processing in the display system according to the first embodiment. [Figure 4] FIG. 10 is a diagram illustrating focus when two objects are displayed. [Figure 5] FIG. 10 is a diagram illustrating focus when two objects are displayed. [Figure 6] FIG. 10 is a diagram illustrating focus when two objects are displayed. [Figure 7] FIG. 10 is a diagram illustrating a process for determining whether two objects overlap. [Figure 8] 10 is a flowchart showing processing in a display system according to a second embodiment. [Figure 9] FIG. 10 is a diagram illustrating focus when two objects are displayed. [Figure 10] FIG. 10 is a diagram illustrating a state in which an object in the background is completely hidden by an object in the foreground. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. (First embodiment) A display system according to a first embodiment will be described with reference to FIGS. 1 is a diagram showing the configuration of a display system according to the first embodiment. As shown in FIG. 1, the display system includes a display device 100 and a vibration output device 110.

[0012] The display device 100 is a head-mounted display device (HMD) that is worn on the user's head, and displays an object synthesized with an image in three-dimensional space.

[0013] The vibration output device 110 is a ring-shaped device that outputs vibration, and includes a button 110a with a built-in optical track pad (hereinafter referred to as OTP). Although the vibration output device 110 has been described as a ring-shaped device, the present invention is not limited to this. The vibration output device 110 may be any device that can provide vibration to the user, and may be, for example, a glove-shaped device worn on the hand, or a bracelet-shaped device worn on the wrist.

[0014] The display device 100 and the vibration output device 110 communicate wirelessly using Bluetooth (registered trademark). Note that the communication method between the display device 100 and the vibration output device 110 is not limited to this, and may be, for example, wired communication or wireless communication using Wi-Fi.

[0015] FIG. 2 is a diagram showing the functional configuration of the display system according to the first embodiment. The display device 100 includes a control unit 101, an imaging unit 102, an image display unit 103, an orientation sensor unit 204, a content DB (database) 105, an object generation unit 106, and a communication unit 107. The control unit 101 is responsible for overall control of the display device 100. In this embodiment, the control unit 101 functions as a determining unit and a display control unit as referred to in the present invention. The imaging unit 102 is composed of two cameras positioned near the user's left and right eyes when the display device 100 is worn, and acquires an image (hereinafter referred to as a captured image) of the real space, which is a three-dimensional space. The image display unit 103 is configured by, for example, an organic EL panel, includes optical systems positioned in front of the left and right eyes of the user when the display device 100 is worn, and displays images that can be observed by the user. The posture sensor 104 is composed of an acceleration sensor, an angular velocity sensor, a geomagnetic sensor, etc., and is capable of acquiring posture information and coordinate information of the user wearing the display device 100. The content DB 105 stores object data and the like. The object generation unit 106 reads out an object from the content DB 105 and generates an object to be combined with the captured image. The communication unit 107 performs wireless communication using Bluetooth.

[0016] In the display device 100, the control unit 101 performs image processing to cancel aberrations inherent in the image capture unit 102 and the image display unit 103 based on captured image data sent from the image capture unit 102 and orientation information sent from the orientation sensor 104. The control unit 101 then performs processing to combine the captured image with an object. The two cameras constituting the image capture unit 102 can acquire distance information about the environment in which a user wearing the display device 100 is located by measuring distances using a stereo camera, and the orientation sensor 104 can also acquire orientation information and coordinate information about the user. Based on this information, the control unit 101 cooperates with the object generation unit 106 to change the position coordinates, orientation, and size of the object, thereby generating an image that makes an object that is not actually present appear to be present in the actual location. The control unit 101 displays an image in which the object is combined with the captured image on the image display unit 103. By observing the combined image displayed on the image display unit 103, a user wearing the display device 100 can experience various mixed realities that combine CG data with real space.

[0017] The vibration output device 110 includes a control unit 111 , a vibration output unit 112 , a communication unit 113 , and an operation unit 114 . The control unit 111 is responsible for overall control of the vibration output device 110. In this embodiment, the control unit 111 functions as the control means of the present invention. The vibration output unit 112 is configured to output vibrations by combining a device that generates vibrations, such as an actuator or a motor, with an IC for controlling the device, etc. The vibration output unit 112 is configured so that at least one of the vibration intensity, frequency, vibration duration, and number of vibrations of the vibration output is variable. The communication unit 113 performs wireless communication using Bluetooth. The operation unit 114 is configured with a button 110a incorporating an OTP.

[0018] In this embodiment, when selecting an object to be displayed on the image display unit 103 of the display device 100, there are three types of focus operations for focusing on the object. Note that the user referred to below is assumed to be wearing the display device 100 and the vibration output device 110. The first focus operation is an operation of aligning a pointer displayed on the display device 100 with an object. In this embodiment, the vibration output device 110 is used, and information on the relative movement direction of the OTP received from the user by the vibration output device 110 and information on pressing the button 110a are sent to the display device 100. The display device 100 performs various processes corresponding to the information sent from the vibration output device 110. Specifically, similar to mouse pointer operation used on a general personal computer, the user can move the pointer displayed on the screen and align the pointer with a desired object by placing a finger on the OTP and rubbing it in any direction. Pressing the button 110a while the pointer is aligned with the object is a selection operation.

[0019] The second focus operation is an operation of directing a ray (a beam of light) displayed on the display device 100 toward an object. In this operation, the user's hand and its orientation in the captured image are recognized by color and shape, and the user moves a pointer on an extension of the pointer in the direction the index finger is pointing. As shown in FIGS. 4 and 5, a ray 401 extending in the direction pointed by the index finger 402 in the captured image is displayed as a CG pointer, and the user moves their hand to direct the ray 401 toward a desired object. Note that the vibration output device 110 may be provided with an acceleration sensor or an angular velocity sensor, and the sensor information may be used in combination with image recognition information to improve the coincidence rate between the direction the user is pointing and the direction of the ray 401. When a predetermined time or more has elapsed while the ray 401 is directed toward the object, a selection operation is performed.

[0020] The third focus operation is called hand tracking, which is an operation in which the user's hand displayed on the display device 100 touches an object. Similar to the ray operation, this recognizes the user's hand and its orientation in the captured image based on color and shape, and determines, for example, contact between the index finger and the object. As shown in FIG. 6, the user moves their hand so that the index finger 601 in the captured image touches the desired object. When the distance between the index finger 601 and the object is equal to or less than a predetermined distance, contact is determined. When a predetermined time or more has elapsed while the index finger 601 is in contact with the object, a selection operation is performed.

[0021] Of these focus operations, ray operation has the advantage of being intuitive, fast in response speed, and easy to focus on even objects that are far from the user. However, it has the disadvantage of being difficult to focus on objects that overlap from the user's perspective. Pointer operation, while less responsive, also has the advantage of being easy to focus on even objects that are far from the user. However, like ray operation, it has the disadvantage of being difficult to focus on objects that overlap from the user's perspective. The display system may be provided with a physical contact device such as a physical button, touchpad, touch panel, cross key, joystick, trackpad device, etc., and the operation may be to move the pointer or ray with the physical contact device. In addition, in the case of ray operation or hand tracking, a selection action may be performed when the user forms their hand in a specific shape.

[0022] The vibration output device 110 outputs vibration as feedback for focusing on an object through a focus operation. The vibration output device 110 also outputs vibration as feedback for performing a selection operation.

[0023] Hereinafter, the processing in the display system according to the first embodiment will be described in detail with reference to FIGS. 3 is a flowchart showing the processing in the display system, which is executed when a focus operation is started by a focus operation. 4 to 6 are diagrams for explaining focusing when two objects A and B are displayed. FIGS. 4 and 5 show an example of focusing by ray operation. FIG. 6 shows an example of focusing by hand tracking. As shown in FIGS. 4 to 6, the depth direction as seen from the user is defined as the z direction, the up and down direction as the y direction, and the left and right direction as the x direction.

[0024] 3, in step S301, the control unit 101 of the display device 100 determines whether or not two objects A and B overlap in the z direction in the captured image to be displayed on the image display unit 103. As shown in FIGS. 4 and 6, if it is determined that objects A and B overlap in the z direction, the process proceeds to step S302. As shown in FIG. 5, if it is determined that objects A and B do not overlap in the z direction, the process proceeds to step S306.

[0025] FIG. 7 is a diagram illustrating the process of determining whether objects A and B overlap. Whether objects A and B overlap in the z direction is determined based on the coordinate and size data of objects A and B, respectively, and the user's coordinate data. For example, as shown in FIG. 7(a), consider a line 701 that connects the coordinates of a polygon that forms the surface of object B in the background with the user's coordinates. If there is a point in the polygon that forms the surface of object A in the foreground that overlaps with line 701, it is determined that objects A and B overlap in the z direction.

[0026] In step S302, the control unit 101 of the display device 100 determines whether or not object A is focused. If object A is focused, the process proceeds to step S303. If object A is not focused, the process proceeds to step S304. In FIG. 4(a), object A is focused by directing a ray 401 at object A. Also, in FIG. 6(a), object A is focused by hand tracking.

[0027] In step S303, in response to the determination that object A is focused in step S302, the vibration output unit 112 of the vibration output device 110 performs vibration output A under the control of the control unit 111.

[0028] In step S304, the control unit 101 of the display device 100 determines whether or not object B is focused. If object B is focused, the process proceeds to step S305. If object B is not focused, the process proceeds to step S310. In FIG. 4(b), object B is focused by directing a ray 401 at object B. Also, in FIG. 6(b), object B is focused by hand tracking.

[0029] In step S305, in response to the determination of focus on object B in step S304, the vibration output unit 112 of the vibration output device 110 performs vibration output B under the control of the control unit 111. The vibration output B is a vibration output different from the vibration output A in step S303. In this embodiment, as represented by the vibration output characteristic line 403 in FIGS. 4(a) and 6(a), the vibration output A generates a predetermined vibration. Furthermore, as represented by the vibration output characteristic line 404 in FIGS. 4(b) and 6(b), the vibration output B generates a predetermined vibration intermittently twice. Note that the vibration output A and the vibration output B may be different vibration outputs, and are not limited to this. For example, a large vibration may be generated when object A in the foreground is focused, and a small vibration may be generated when object B in the background is focused.

[0030] In step S306, the control unit 101 of the display device 100 determines whether or not object A is focused. If object A is focused, the process proceeds to step S307. If object A is not focused, the process proceeds to step S308. In FIG. 5(a), object A is focused by directing ray 401 at object A.

[0031] In step S307, in response to the focus determination on object A in step S306, in the vibration output device 110, under the control of the control unit 111, the vibration output unit 112 performs vibration output C. The vibration output C is a vibration output different from the vibration outputs A and B, as represented by the vibration output characteristic line 501 in FIGS. 5(a) and 5(b).

[0032] In step S308, the control unit 101 of the display device 100 determines whether or not object B is focused. If object B is focused, the process proceeds to step S309. If object B is not focused, the process proceeds to step S310. In FIG. 5(b), object B is focused by directing ray 401 at object B.

[0033] In step S309, in response to the determination of focus on object B in step S308, in the vibration output device 110, under the control of the control unit 111, the vibration output unit 112 performs vibration output C. As represented by the characteristic line 501 of the vibration output in Figures 5(a) and 5(b), the vibration output C is the same as the vibration output in step S307.

[0034] In step S310, the control unit 101 of the display device 100 determines whether or not to end the focus operation. If the focus operation is not to be ended, the process returns to step S301. If the focus operation is to be ended, the process exits from this flowchart.

[0035] As described above, when multiple overlapping objects are focused on from the user's perspective, the vibration output is changed according to the position of the focus. Specifically, if object A and object B are overlapping from the user's perspective, vibration output A is produced when object A is focused on, and vibration output B is produced when object B is focused on. This allows the user to easily recognize which of objects A and B that overlap from the user's perspective they are focused on. Furthermore, since the vibration output changes when the focused object is changed, the user can easily recognize that the focused object has changed. This allows the user to reliably focus on the object they intend.

[0036] Furthermore, when the focus is on a single object that is not overlapping from the user's perspective (objects A and B in Figure 5), vibration output C is performed. This is because with a single object, the user can visually clearly recognize which object is being focused on, even if the vibration output is not different for each object.

[0037] (Second embodiment) Next, a second embodiment will be described with reference to Figures 7 to 10. The configuration of the display system and the focus operation are the same as those in the first embodiment, and the following description will focus on the differences from the first embodiment. Hereinafter, the processing in the display system according to the second embodiment will be described in detail with reference to FIGS. Fig. 8 is a flowchart showing the processing in the display system, which is executed when a focus operation is started by a focus operation. Figure 9 is a diagram explaining focusing when two objects A and B are displayed, and shows an example of focusing by ray operation. As shown in Figure 9, the depth direction as seen by the user is defined as the z direction, the up / down direction as the y direction, and the left / right direction as the x direction. Figure 10 is a diagram explaining a state in which object B in the background is completely hidden by object A in the foreground.

[0038] In the flowchart of Fig. 8, in step S801, the control unit 101 of the display device 100 determines whether two objects A and B overlap in the z direction in the captured image displayed on the image display unit 103. As shown in Fig. 9, if it is determined that objects A and B overlap in the z direction, the process proceeds to step S802. If it is determined that objects A and B do not overlap in the z direction, the process proceeds to step S806. In this embodiment, if objects A and B overlap in the z direction, the control unit 101 identifies the overlapping area.

[0039] As in the first embodiment, whether objects A and B overlap in the z direction is determined based on the coordinate and size data of each of objects A and B and the coordinate data of the user. In this embodiment, as shown in FIG. 7(a), a line 701 is assumed to connect the coordinates of a polygon constituting the surface of object B in the background with the coordinates of the user, and an area 703 of object A in the foreground that exists inside the point that overlaps with line 701 is identified. Also, as shown in FIG. 7(b), a line 702 is assumed to connect the coordinates of a polygon constituting the surface of object A in the background with the coordinates of the user, and an area 704 of object B in the background that exists inside the point that overlaps with line 702 is identified. The area where these areas 703 and 704 overlap is defined as the area where objects A and B overlap as seen from the user.

[0040] In step S802, the control unit 101 of the display device 100 determines whether or not the focus is on an area where objects A and B do not overlap. If the focus is on an area where objects A and B do not overlap, the process proceeds to step S803. If the focus is not on an area where objects A and B do not overlap, the process proceeds to step S804. In FIG. 9(a), the focus is achieved by directing the ray 401 to an area of ​​object A where objects A and B do not overlap.

[0041] In step S803, in response to the determination in step S802 that the focus is on the area where objects A and B do not overlap, the vibration output unit 112 of the vibration output device 110 performs vibration output A under the control of the control unit 111.

[0042] In step S804, the control unit 101 of the display device 100 determines whether or not the area where objects A and B overlap has been focused. If the area where objects A and B overlap has been focused, the process proceeds to step S805. If the area where objects A and B overlap has not been focused, the process proceeds to step S810. In FIG. 9(b), the area where objects A and B overlap has been focused by directing the ray 401.

[0043] In step S805, in response to the determination in step S804 that the focus is on the area where objects A and B overlap, the vibration output unit 112 of the vibration output device 110 performs vibration output B under the control of the control unit 111. When focusing on the area where objects A and B overlap, it is not clear whether the focus is on object A or object B, so vibration output B is set to a vibration output different from the vibration output A in step S804. In this embodiment, as represented by the vibration output characteristic line 901 in FIG. 9( a), vibration output A generates a predetermined vibration. Furthermore, as represented by the vibration output characteristic line 902 in FIG. 9( b), vibration output B generates a predetermined vibration intermittently twice. Note that vibration output A and vibration output B may be different vibration outputs, and are not limited to this.

[0044] In step S806, the control unit 101 of the display device 100 determines whether or not the focus is on object A. If the focus is on object A, the process proceeds to step S807. If the focus is not on object A, the process proceeds to step S808.

[0045] In step S807, in response to the determination of focus on object A in step S806, in the vibration output device 110, the vibration output unit 112 performs vibration output A under the control of the control unit 111. The vibration output A is the same as the vibration output in step S804.

[0046] In step S808, the control unit 101 of the display device 100 determines whether or not the focus is on object B. If the focus is on object B, the process proceeds to step S809. If the focus is not on object B, the process proceeds to step S810.

[0047] In step S809, in response to the determination of focus on object B in step S808, in the vibration output device 110, the vibration output unit 112 performs vibration output A under the control of the control unit 111. The vibration output A is the same as the vibration output in step S804.

[0048] In step S810, the control unit 101 of the display device 100 determines whether or not to end the focus operation. If the focus operation is not to be ended, the process returns to step S801. If the focus operation is to be ended, the process exits from this flowchart.

[0049] As described above, when multiple overlapping objects are focused on from the user's perspective, the vibration output is changed depending on the position of the focus. Specifically, if object A and object B overlap from the user's perspective, vibration output A is performed when the focus is on an area where objects A and B do not overlap, and vibration output B is performed when the focus is on an area where objects A and B overlap. When focusing on an area where objects A and B overlap, it is not necessarily clear which of object A or B the user wants to focus on. Therefore, vibration output B is performed when the focus is on an area where objects A and B overlap, prompting the user to clarify the object to focus on, enabling the user to reliably focus on the object they intend.

[0050] Furthermore, as shown in the left diagram of Fig. 10, when object B (shown by the dotted line) in the background is completely hidden by object A in the foreground, the user may not be aware of the existence of object B. In this case, when object A in the background is focused, different vibration outputs are generated depending on whether the focus is on the area where objects A and B do not overlap or the area where objects A and B overlap. This makes it possible for the user to recognize that object B is hidden behind object A.

[0051] Note that, when object B in the background is completely hidden by object A in the foreground, it is not possible to focus on object B by operating with a ray or a pointer. Therefore, object B in the background may be displayed so that it can be seen by the user. For example, by performing a selection action while vibration output B is being generated by focusing on an area where objects A and B overlap, the control unit 101 of the display device 100 swaps the z-direction coordinates of objects A and B, placing object B in front of object A and displaying it so that it can be seen by the user. Alternatively, object A may be shifted in the x-direction or y-direction, or object A in the foreground may be made transparent so that object B can be seen. Object B may also be highlighted by applying a color, for example. Furthermore, if the number of vibrations is included as a parameter of the vibration caused by vibration output B, object B may be displayed so that it can be seen by performing an operation such as pressing a button the same number of times as the number of vibrations.

[0052] In the first and second embodiments, when object B in the background is completely hidden by object A in the foreground and the focus is on object A in the background, a predetermined vibration output may be generated. In the first embodiment, the predetermined vibration output is different from vibration outputs A to C, and in the second embodiment, it is different from vibration outputs A and B. This allows the user to recognize that object B is hidden behind object A.

[0053] In the first embodiment, as shown in Fig. 9(b), when an area where objects A and B overlap is focused, it may be treated as if the foreground object A is focused. Alternatively, the first embodiment and the second embodiment may be combined. That is, when objects A and B overlap as seen from the user, different vibration outputs may be produced when object A is focused, when object B is focused, and when the area where objects A and B overlap is focused.

[0054] In the first and second embodiments, an example was described in which two objects A and B overlap as seen from the user, but the vibration output may be changed according to the number of objects. Also, the vibration output may be changed according to the distance between multiple overlapping objects as seen from the user.

[0055] Furthermore, when there are multiple vibration output devices, different vibration outputs may be generated among the multiple vibration output devices. For example, when a user wears the vibration output device 110 of a ring-shaped device on both hands, the hand performing the focus operation is determined and the vibration output is changed accordingly.

[0056] Although the present invention has been described above with reference to the embodiments, the above embodiments are merely illustrative of specific examples of how the present invention can be implemented, and the technical scope of the present invention should not be construed as being limited by these embodiments. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features. In the above embodiment, the vibration output device 110 itself functions as a control device for a vibration output device to which the present invention is applied, but this is not limited to this. For example, the display device 100 may function as a control device for a vibration output device to which the present invention is applied. Furthermore, an information processing device that functions as a control device for a vibration output device to which the present invention is applied may be installed as a separate entity from the display device 100 and the vibration output device 110. (Other embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0057] The disclosure of this embodiment includes the following configuration. (Configuration 1) A display system including a display device that displays an object by combining it with an image of a three-dimensional space, and a vibration output device that outputs vibrations, a control means for controlling the vibration output device to output a vibration when an object is focused on by a focus operation on the display device, The display system is characterized in that the control means, when a plurality of overlapping objects as seen from the user are focused on, changes vibration output in accordance with the position of the focus. (Configuration 2) The display system described in configuration 1 is characterized in that, when a first object and a second object overlap as seen from the user, the control means controls so that a first vibration output is performed when the first object is focused on, and a second vibration output is performed when the second object is focused on. (Configuration 3) The display system according to configuration 2, wherein the control means controls the display system so that a third vibration output is performed when the focus is on a single object that is not overlapped as seen from the user. (Configuration 4) The display system described in configuration 1 is characterized in that the control means controls the display system so that, when a first object and a second object overlap as seen by the user, a first vibration output is performed when the user focuses on an area where the first object and the second object do not overlap, and a second vibration output is performed when the user focuses on an area where the first object and the second object overlap. (Configuration 5) 5. The display system according to any one of configurations 1 to 4, wherein the focus operation is an operation of pointing a pointer displayed on the display device to an object. (Configuration 6) 6. The display system according to any one of configurations 1 to 5, wherein the focus operation is an operation of directing a ray displayed on the display device toward an object. (Configuration 7) 7. The display system according to any one of configurations 1 to 6, wherein the focus operation is an operation of touching an object displayed on the display device with the user's hand. (Configuration 8) A display system according to any one of configurations 2 to 4 and configurations 5 to 7 dependent on configurations 2 to 4, characterized in that it comprises a determination means for determining whether the first object and the second object overlap as viewed from the user, based on the coordinates and sizes of each of the plurality of first objects and the second object and the user's coordinates. (Configuration 9) The display system according to any one of configurations 1 to 8, further comprising a display control means for displaying an object at the back so that the object can be seen by the user when the object at the back is completely hidden by an object at the front as seen by the user. [Explanation of symbols]

[0058] 100: display device, 101: control unit, 102: imaging unit, 103: image display unit, 104: attitude sensor unit, 105: content DB, 106: object generation unit, 107: communication unit, 110: vibration output device, 111: control unit, 112: vibration output unit, 113: communication unit, 114: operation unit

Claims

1. A display system including a display device that displays an object by combining it with an image of a three-dimensional space, and a vibration output device that outputs a vibration, a control means for controlling the vibration output device to output a vibration when an object is focused on by a focus operation on the display device, The display system is characterized in that the control means, when a plurality of objects that overlap as seen from the user are focused on, changes vibration output in accordance with the position of the focus.

2. The display system according to claim 1, characterized in that the control means controls the display so that, when a first object and a second object overlap as seen by the user, a first vibration output is performed when the first object is focused on, and a second vibration output is performed when the second object is focused on.

3. 3. The display system according to claim 2, wherein the control means controls to output a third vibration when a single object that is not overlapped as seen from the user is focused on.

4. The display system of claim 1, characterized in that the control means controls the display so that, when a first object and a second object overlap as seen by the user, a first vibration output is performed when the user focuses on an area where the first object and the second object do not overlap, and a second vibration output is performed when the user focuses on an area where the first object and the second object overlap.

5. 5. The display system according to claim 1, wherein the focus operation is an operation of pointing a pointer displayed on the display device to an object.

6. 5. The display system according to claim 1, wherein the focus operation is an operation of directing a ray displayed on the display device toward an object.

7. 5. The display system according to claim 1, wherein the focus operation is an operation of touching an object displayed on the display device with the user's hand.

8. A display system according to any one of claims 2 to 4, characterized in that it comprises a determination means for determining whether the first object and the second object overlap as viewed by the user based on the coordinates and sizes of each of the plurality of first objects and the second object and the user's coordinates.

9. The display system according to any one of claims 1 to 4, further comprising a display control means for displaying an object at the back so that it can be seen by the user when the object at the back is completely hidden by an object at the front as seen by the user.

10. A control device that controls a vibration output device that outputs vibration, a control means for controlling a display device to output vibrations when the display device focuses on an object by performing a focus operation on the display device that displays the object in combination with an image of a three-dimensional space; The control device for a vibration output device, wherein the control means changes vibration output depending on a position of focus when a plurality of objects that overlap as seen from the user are focused on.

11. A control method for controlling a vibration output device that outputs vibration, comprising: a control step of controlling a display device to output a vibration when the display device focuses on an object by a focus operation on the display device that displays the object synthesized with an image of a three-dimensional space, The control method for a vibration output device, wherein, in the control step, when a plurality of objects that overlap as seen from the user are focused on, the vibration output is changed according to the position of the focus.

12. A program for controlling a vibration output device that outputs vibration, causing a computer to function as a control means for controlling a display device to output vibrations when the display device focuses on an object by performing a focus operation on the display device that displays the object synthesized with an image of a three-dimensional space; The program is characterized in that the control means changes vibration output depending on the position of the focus when multiple objects that are overlapping as seen from the user are focused on.

Images