Electronic apparatus, method of controlling same, and program

Abstract

To make it possible for a user to easily recognize and read information in a VR space and an AR space by displaying information in a mode matched with a user behavior.SOLUTION: An electronic apparatus has: a display control unit for performing control to display a virtual text object superimposed on a video in a real space on a screen; first detection means for detecting a user position in the real space; and second detection means for detecting a visual line of the user viewing the screen. Display control means performs control to display the text object with a first font if a first condition is satisfied, and displays the text object with a second font different from the first font if the first condition is not satisfied.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to an electronic device, a control method thereof, and a program. [Background technology]

[0002] Virtual reality (VR) and augmented reality (AR) are technologies that combine the real and virtual worlds to provide simulated experiences. In recent years, VR and AR technologies have been used not only in games and entertainment but also in various fields such as education and medicine.

[0003] Therefore, in VR and AR spaces, it is necessary to provide various information to users. One method of providing such information is to display the information in the form of a text object. Patent Document 1 describes that in a volumetric space such as VR or AR, when multiple users exist, a message that is appropriately translated according to the user is displayed in a position visible to each user. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] U.S. Patent Application Publication No. 2021 / 0074070 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the above-mentioned Patent Document 1 only assumes that a user will stand still and look at a displayed message in a volumetric space. For example, it is conceivable that a user will view a displayed message while moving within a volumetric space. In such a case, a message that is displayed assuming a user will stand still and look at it may be difficult for the user to recognize or read.

[0006] Therefore, the electronic device of the present invention aims to make information easier for users to recognize and read by displaying information in a form that matches the user's behavior in a VR or AR space. [Means for solving the problem]

[0007] In order to achieve the above object, the electronic device of the present invention comprises a display control means for controlling the display of a virtual text object on a screen by superimposing it on an image of real space, a first detection means for detecting the position of a user in the real space, and a second detection means for detecting the line of sight of the user looking at the screen, wherein the display control means controls the display of the text object in a first font if a first condition is met, that is, the position of the text object displayed superimposed on the image has not changed relative to the position of the user detected by the first detection means, and the position of the user's line of sight detected by the second detection means is the position of the text object, and if the first condition is not met, controls the display of the text object in a second font different from the first font. [Effects of the Invention]

[0008] According to the present invention, by displaying information in a form that matches the user's actions in a VR space or an AR space, the user can easily recognize and read the information. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is an external view of a display control device 100 according to the present embodiment. [Figure 2] 1 is a block diagram showing a configuration of a smartphone 200 as an example of a display control device 100 according to the present embodiment. [Figure 3] 1 is an external view of VR goggles on which a smartphone 200 according to the present embodiment can be worn. [Figure 4] 1 is an external view of controllers 240 and 250 capable of communicating with a smartphone 200 according to the present embodiment. [Figure 5] 10 is a flowchart for explaining a process in which the smartphone 200 displays virtual information (objects) in accordance with the user's actions in real space, according to the present embodiment. [Figure 6] 10 is a diagram illustrating an example of a screen of a VR space application on a smartphone 200 according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0011] FIG. 1 is an external view of a display control device 100. The display control device 100 is realized by an electronic device such as a smartphone, a tablet terminal, or a digital camera. FIG. 1(a) is an external view of the front of the display control device 100, and FIG. 1(b) is an external view of the back of the display control device 100. The display control device 200 has a display 105. On the display 105, AR images (AR content) and VR images (VR content) can be displayed in AR and VR, respectively.

[0012] The display 105 is a display unit that displays images and various information. The display 105 is configured integrally with a touch panel 106a, as will be described later, and is capable of detecting touch operations on the display surface of the display 105.

[0013] As shown in the figure, the operation unit 106 includes a touch panel 106a and operation units 106b, 106c, 106d, and 106e. The operation unit 106b is a power button that accepts an operation to switch the power of the display control device 100 on and off. The operation units 106c and 106d are volume buttons that increase and decrease the volume of the audio output from the audio output terminal 112a and the speaker 112b. The operation unit 106e is a home button for displaying a home screen on the display 105.

[0014] The audio output terminal 112a is an earphone jack for outputting audio to an earphone, an external speaker, etc. The speaker 112b is a built-in speaker for producing audio.

[0015] The imaging unit 115 on the rear surface of the display control device 100 is a camera capable of capturing images.

[0016] 2 is a block diagram showing the configuration of a smartphone 200 as an example of the display control device 100. A CPU 201, a memory 202, a nonvolatile memory 203, an image processing unit 204, a display 205, an operation unit 206, a recording medium I / F 207, an external I / F 209, and a communication I / F 210 are connected to an internal bus 250. An audio output unit 212 and an attitude detection unit 213 are also connected to the internal bus 250. The units connected to the internal bus 250 are configured to be able to exchange data with each other via the internal bus 250.

[0017] The CPU 201 is a control unit that controls the entire smartphone 200, and is configured with at least one processor or circuit. The memory 202 is configured with, for example, a random access memory (RAM) such as a volatile memory that uses semiconductor elements. The CPU 201 controls each unit of the smartphone 200 using the memory 202 as a work memory in accordance with a program stored in, for example, the nonvolatile memory 203. The nonvolatile memory 203 stores image data, audio data, other data, various programs for the operation of the CPU 201, and the like. The nonvolatile memory 203 is configured with, for example, a flash memory, a read-only memory (ROM), or the like.

[0018] Under the control of the CPU 201, the image processing unit 204 performs various image processing on images stored in the nonvolatile memory 203 or the recording medium 208, video signals acquired via the external I / F 209, images acquired via the communication I / F 210, and the like. The image processing performed by the image processing unit 204 includes A / D conversion, D / A conversion, image data encoding, compression, decoding, enlargement / reduction (resizing), noise reduction, color conversion, and the like. The image processing unit 204 also performs various image processing, such as panoramic expansion, mapping, and conversion of omnidirectional images or VR images, which are wide-range images having data of a wide range even if they are not omnidirectional. The image processing unit 204 may be configured with a circuit block dedicated to performing specific image processing. Depending on the type of image processing, the CPU 201 may perform image processing according to a program without using the image processing unit 204.

[0019] Display 205 displays images, GUI screens constituting a Graphical User Interface (GUI), and the like under the control of CPU 201. CPU 201 generates a display control signal in accordance with a program, and controls each unit of smartphone 200 to generate a video signal to be displayed on display 205 and output the video signal to display 205. Display 205 displays a video based on the output video signal. Note that smartphone 200 itself may only have an interface for outputting a video signal to be displayed on display 205, and display 205 may be configured as an external monitor (such as a television).

[0020] The operation unit 206 is an input device for receiving user operations, including a character information input device such as a keyboard, a pointing device such as a mouse or a touch panel, a button, a dial, a joystick, a touch sensor, a touch pad, etc. The touch panel is an input device that is configured as a plane overlaid on the display 205 and outputs coordinate information according to the position of contact.

[0021] A recording medium I / F 207 can be loaded with a recording medium 208 such as a memory card, CD, or DVD, and reads data from or writes data to the loaded recording medium 208 under the control of the CPU 201. The external I / F 209 is an interface that connects to an external device via a wired cable or wirelessly and inputs and outputs video signals and audio signals. The communication I / F 210 is an interface that communicates with external devices, the Internet 211, etc., and transmits and receives various data such as files and commands. It is also possible to communicate with wirelessly connected controllers 240, 250, etc., which will be described later.

[0022] The audio output unit 212 outputs audio from video and music data, operation sounds, ringtones, various notification sounds, etc. The audio output unit 212 includes an audio output terminal 112a for connecting earphones or the like, and a speaker 112b, but audio output may also be performed via wireless communication or the like.

[0023] The attitude detection unit 213 detects the attitude of the smartphone 200 with respect to the direction of gravity and the tilt of the attitude with respect to each of the yaw, roll, and pitch axes. Based on the attitude detected by the attitude detection unit 213, it is possible to determine whether the smartphone 200 is held horizontally, held vertically, pointed up, pointed down, or at an angle. The attitude detection unit 213 can use at least one of an acceleration sensor, a gyro sensor, a geomagnetic sensor, a direction sensor, an altitude sensor, etc., and can also use a combination of two or more sensors.

[0024] The self-position / surrounding environment estimation unit 214 estimates the self-position and surrounding environment in the real space of the smartphone 200 or the VR goggles 300 described below.

[0025] The "self-position" refers to the position of the smartphone 200 or the VR goggles 300 (described later) in real space. For example, it is expressed by three parameters that represent the position in a coordinate system in which a predetermined position within a predetermined range of space is defined as the origin and three mutually orthogonal axes are defined as the X-axis, Y-axis, and Z-axis. Furthermore, it may be expressed by adding three parameters that represent the attitude (orientation).

[0026] The imaging unit 215 is a camera capable of capturing images. The captured images can be used for various detection processes in the gesture detection unit 206b, the self-position and surrounding environment estimation unit 214, and the like, which will be described later. It is also possible to output an image of the outside world to the display 205.

[0027] The operation unit 206 includes a touch panel 206a. The CPU 201 can detect the following operations or states on the touch panel 206a. A finger or pen that has not been touching the touch panel 206a touches the touch panel 206a again, that is, the start of touching (hereinafter referred to as touch-down). The touch panel 206a is in a state where a finger or a pen is touching the touch panel 206a (hereinafter referred to as Touch-On). A finger or pen is moved while touching the touch panel 206a (hereinafter referred to as Touch-Move). The finger or pen that was touching the touch panel 206a is removed from the touch panel 206a, that is, the end of touch (hereinafter referred to as touch-up). A state in which nothing is touching the touch panel 206a (hereinafter referred to as Touch-Off)

[0028] When a touch down is detected, a touch on is also detected at the same time. After a touch down, a touch on is usually continued to be detected unless a touch up is detected. If a touch move is detected, a touch on is also detected at the same time. Even if a touch on is detected, a touch move is not detected unless the touch position has moved. When it is detected that all fingers or pens that were touching have touched up, a touch off is detected.

[0029] These operation states and the position coordinates of the finger or pen touching the touch panel 206a are notified to the CPU 201 via the internal bus 205. The CPU 201 determines what kind of operation (touch operation) has been performed on the touch panel 206a based on the notified information. Regarding touch-move, the movement direction of the finger or pen moving on the touch panel 206a can also be determined for each vertical and horizontal component on the touch panel 206a based on changes in the position coordinates. If a touch-move of a predetermined distance or more is detected, it is determined that a slide operation has been performed.

[0030] An operation in which a finger is touched to the touch panel 206a, moved quickly for a certain distance, and then released is called a flick. In other words, a flick is an operation in which a finger is quickly traced across the touch panel 206a as if flicking. When a touch-move of a predetermined distance or more at a predetermined speed or more is detected and a touch-up is then detected, it can be determined that a flick has been performed (it can be determined that a flick occurred following a slide operation).

[0031] Furthermore, a touch operation in which multiple points (for example, two points) are touched simultaneously and the touch positions are brought closer together is called a pinch in, and a touch operation in which the touch positions are moved farther apart is called a pinch out. Pinch out and pinch in are collectively called a pinch operation (or simply pinch). The touch panel 206a may be of any of various types, such as a resistive film type, a capacitance type, a surface acoustic wave type, an infrared type, an electromagnetic induction type, an image recognition type, or an optical sensor type. There are types that detect a touch by contact with the touch panel, and types that detect a touch by the approach of a finger or a pen to the touch panel, and either type is acceptable.

[0032] The operation unit 206 also includes a gesture detection unit 206b. The CPU 201 acquires a gesture image of the user's hand or the like via the imaging unit 215, and the gesture detection unit 206b can detect a predetermined gesture.

[0033] FIG. 3 is an external view of VR goggles (head-mounted adapter) 300 to which a smartphone 200 can be attached. By attaching the smartphone 200 to the VR goggles 300, it is also possible to use the smartphone 200 as a head-mounted display. An insertion slot 301 is an insertion slot for inserting the smartphone 200. The entire smartphone 200 can be inserted into the VR goggles 300 with the display surface of the display 205 facing toward a headband 302 (i.e., toward the user) for fixing the VR goggles 300 to the user's head. With the VR goggles 300 with the smartphone 200 attached attached to their head, the user can view the display 205 of the smartphone 200 without holding the smartphone 200 in their hands. In this case, when the user moves their head or entire body, the posture of the smartphone 200 also changes. An posture detection unit 213 detects a change in the posture of the smartphone 200, and the CPU 201 performs VR display processing based on this change in posture. In this case, detecting the orientation of the smartphone 200 by the orientation detection unit 213 is equivalent to detecting the orientation of the user's head (the direction in which the user's gaze is facing). The imaging lens 303 enables the imaging unit 215 of the smartphone 200 to acquire images of real space and images of the user's gestures even when the smartphone 200 is attached to the VR goggles 300. The smartphone 200 and the VR goggles 300 may be integrated into a single housing. The smartphone 200 may also detect the user's gaze, facial expression, and the like, and use the information as an operating means for the smartphone 200.

[0034] FIG. 4 is an external view of the controllers 240 and 250 that can communicate with the smartphone 200. As shown in FIG.

[0035] When the user holds the holding portion 441 of the grip-type controller 240 with his / her hand and operates a member on the operation surface 442, an operation event is notified to the smartphone 200.

[0036] Furthermore, ring-type controller 250 is composed of ring operation unit 451 and ring unit 452 to be worn on user's finger 453. Ring operation unit 451 may be a push-button or the like, or may be a member capable of detecting finger contact, such as a rotary dial or optical trackpad.

[0037] In the VR goggles 300, not only can virtual images be displayed, but also images of real space (video see-through images) captured by the imaging lens 303 can be displayed. Hereinafter, an example will be described in which an image of real space is displayed in the VR goggles when the user wears and uses the VR goggles 300 into which the smartphone 200 is inserted.

[0038] Fig. 5 is a flowchart for explaining a process in which the smartphone 200 displays a virtual text object in accordance with the user's actions in real space according to the present embodiment. Fig. 6 is an example of a screen of a VR space application on the smartphone 200 according to the present embodiment.

[0039] When the smartphone 200 detects that it is being worn with the VR goggles 300 or when a predetermined operation is performed by the user, it executes a process of displaying a virtual text object superimposed on an image in real space in accordance with the user's actions in real space.

[0040] In this process, the font of the text object is switched between condition 1 (when the relative position between the user and the text object remains unchanged and the user is gazing at the text object) and condition 2 (otherwise). This allows the text object to be displayed in a way that is easy for the user to recognize and read.

[0041] In S501, the CPU 201 reads the software to be executed and performs initialization processing of flags and the like.

[0042] In S502, the CPU 201 displays an image of the real space captured through the imaging lens 303 on the display 205 as a video see-through image 600. Fig. 6(d) is an example of a screen in which the video see-through image 600 is displayed on the display 205.

[0043] In S503, the CPU 201 determines whether movement of the user in the real space has been detected. If it is determined that movement of the user in the real space has been detected, the process proceeds to S504; otherwise, the process proceeds to S505.

[0044] There are several possible methods for determining the movement of the user in real space. For example, if the orientation detection unit 213 or the self-position / surrounding environment estimation unit 214 detects that the smartphone 200 is moving in real space, it may be determined that the user is moving in real space. Alternatively, the video see-through image 600 may be analyzed to detect the movement of the user in real space. Alternatively, the movement of the user may be detected from a change in the user's position information in real space.

[0045] In S504, the CPU 201 updates the stored position information of the user in response to the user's movement in the real space.

[0046] In S505, the CPU 201 determines whether or not the user's gaze has been detected in the real space. If it is determined that the user's gaze has been detected, the process proceeds to S506, and if not, the process proceeds to S507.

[0047] There are several possible methods for determining whether a user is gazing in real space. For example, it may be determined that a user is gazing when a state in which a specific direction is displayed on the display 205 by the posture detection unit 213 or the self-position / surrounding environment estimation unit 214 included in the smartphone 200 continues for a certain period of time. Alternatively, it may be determined that a user is gazing when the smartphone 200 is capable of detecting a line of sight and the line of sight detection continues for more than a predetermined period of time at a specific position.

[0048] In S506, the CPU 201 updates the stored information on the gaze position of the user in response to the user gazing at a specific point in the real space.

[0049] In S507, CPU 201 determines whether or not virtual information (objects) are displayed superimposed on the image of real space. Objects include text objects, icons, setting GUI panels, etc. If it is determined that an object is displayed superimposed on the image of real space, the process proceeds to S508; otherwise, the process proceeds to S514.

[0050] The object may be one that is displayed in advance and superimposed on the image of real space, or one that starts to be displayed superimposed on the image of real space in response to a user operation, such as a setting GUI panel 601 in Fig. 6 (described later).

[0051] In S508, the CPU 201 stores the display position of the object that is displayed superimposed on the image of the real space.

[0052] In S509, the CPU 201 determines whether the object is a text object. If it is determined to be a text object, the process proceeds to S510; if not, the process proceeds to S514.

[0053] In S510, CPU 201 determines whether the display position of the text object stored in S508 has changed relative to the position of the user in real space stored in S504. The display position of the text object changes relative to the user's position, for example, when the display position of the text object is fixed and the user himself moves, or when the user himself does not move but the display position of the text object moves. The display position of the text object does not change relative to the user's position, for example, when the text object is displayed following the user's movement, or when the display position of the text object is fixed and the user himself does not move. If it is determined that the display position of the text object has changed relative to the position of the user in real space, proceed to S511; otherwise, proceed to S513.

[0054] In the real space, the text object is basically displayed at a programmed position, and therefore it is possible to recognize this on the smartphone 200. Then, based on the detection of the user's movement in the real space in S503 and the data on the display position of the text object in S508, it is possible to detect whether there has been a change in the relative position of the user and the text object in the real space.

[0055] In S511, CPU 201 determines whether the user is gazing at the text object. To do so, it determines whether the display position of the text object stored in S508 matches the user's gaze position stored in S506. If they match, that is, if it is determined that the user is gazing at the text object, proceed to S513; otherwise, proceed to S512.

[0056] In S512, the CPU 201 sets the font of the text object to Mincho. This setting is used when the position of the text object and the user's gaze position have changed relatively, or when the display position of the text object and the user's gaze position have not changed relatively and the user is not gazing at the text object. In such cases, it is necessary to make the existence of the text object easier for the user to notice. Therefore, the font is set to Mincho, which is a font that makes the existence of the text object easier to notice.

[0057] In S513, the CPU 201 sets the font of the text object to Gothic. This setting is used when the display position of the text object and the user's gaze position have not changed relative to each other and the user is gazing at the text object. In such a case, it is necessary to make the content of the text object easy for the user to understand. Therefore, Gothic, a font that makes the content of the text object easy to understand, is used.

[0058] In S514, the CPU 201 displays the text object in the font set in S512 or S513 (overlapping it on the image in real space).

[0059] In S515, the CPU 201 determines whether or not another operation has been performed. If another operation has been performed, the process proceeds to S516; if not, the process proceeds to S517. An example of another operation is an operation to display the setting GUI panel 601 in real space.

[0060] In S516, the CPU 201 performs other processes corresponding to other operations.

[0061] In S517, the CPU 201 determines whether or not an end operation has been performed. If an end operation has been performed, the process ends; if not, the process proceeds to S502.

[0062] 6(a) is an example of a screen in which a setting GUI panel 601 is displayed on a video see-through image 600. The setting GUI panel 601 is displayed superimposed on the video see-through image 600. Also superimposed on this are various applications 602 to 604. In this example, the selection cursor is positioned on application 602. Furthermore, guidance 605 explaining the contents of application 602, a time display 606, and the like are superimposed and displayed. Also superimposed is a gaze pointer 620 indicating the user's gaze position.

[0063] Here, it is assumed that the setting GUI panel 601 is always displayed at a position equidistant from the user. In other words, even if the user moves within real space, the setting GUI panel 601 is always displayed following the user. In other words, it can be said that the position of the setting GUI panel 601 and the user in real space does not change relative to each other. Also, in FIG. 6(a), the gaze pointer 620 indicating the user's gaze position is on the guidance 605, so it can be seen that the user is gazing at the text object of the guidance 605.

[0064] In this way, the relative positions of the setting GUI panel 601 and the user in real space do not change, and the user is gazing at the text object of guidance 605. Therefore, condition 1 is met, and the font of the text object of guidance 605 is set to Gothic so that the user can easily understand the content of the text object.

[0065] At this time, the setting GUI panel 601 is displayed in a manner that follows the user, so the characters "Apps" and the time display 606 displayed on the setting GUI panel 601 also follow the user. That is, the characters "Apps" and the time display 606 do not change relative to the user's position in real space. However, although the user is gazing at the guidance 605, they are not gazing at the characters "Apps" or the time display 606. Therefore, the characters "Apps" and the time display 606 meet condition 2, and so the font is set to Mincho to make the presence of the text object easier to notice.

[0066] FIG. 6(b) shows an example in which the user has moved their gaze position from the state shown in FIG. 6(a) to the time display 606. There is no relative change between the time display 606 and the user's position in real space, and the user is gazing at the time display 606, so this satisfies condition 1. Therefore, the font of the text object is set to Gothic to make the content of the text object easier to understand. On the other hand, although there is no relative change between the guidance 605 and the word "Apps" and the user's position in real space, the user is not gazing at the guidance 605 or the word "Apps," so this satisfies condition 2. Therefore, the font is set to Mincho to make the user more aware of the existence of the text object.

[0067] 6(c) shows an example in which the posture detection unit 213 detects that the user has turned their head from left to right (arrow 430) from the state shown in FIG. 6(b), and the display positions of the objects displayed superimposed on the video see-through image 600 and the image of real space are updated accordingly. In this example, the setting GUI panel 601 is displayed while tracking in front of the user. Therefore, when the user is gazing at the time display 606, the font of the time display 606 satisfies condition 1, and so it remains displayed in Gothic font, unchanged from FIG. 6(b).

[0068] In FIG. 6(d), a warning message 607 as an object is arranged in a form written (superimposed) on a wall 610 in the real space. In other words, this warning message 607 is not displayed so as to follow the movement of the user in the real space. Therefore, when the user in the real space walks in the direction of the arrow 631, the position of the user and the position of the warning message 607 change relatively. Furthermore, the user is walking in the direction of the arrow 631 while gazing at the painting hanging on the wall 610 at the position indicated by the gaze pointer 620, and therefore is not gazing at the warning message 607. From the above, since condition 2 is met, the warning message 607 is displayed in the Mincho font.

[0069] 6(e) shows a state in which the user is facing a wall 610. The wall 610 is displayed over the entire surface of the display 205 as a video see-through image 600. In addition, a notice 607 is displayed superimposed on the wall 610.

[0070] 6(e), the user's gaze position (the position of the gaze pointer 620) is on the picture, not on the warning 607. Therefore, the warning 607 is displayed in the Mincho font.

[0071] Fig. 6(f) shows a state in which the user is gazing at warning statement 607, following the state in Fig. 6(e). The relative positions of the user and warning statement 607 remain unchanged, and the user is gazing at warning statement 607. Therefore, condition 1 is met, and the font of warning statement 607 is set to Gothic to make it easier for the user to understand the content of the text object.

[0072] FIG. 6(g) is a diagram illustrating representative examples of fonts. Font 632 is a Japanese Mincho font, and font 633 is a Japanese Gothic font. Font 634 is an English serif font, and font 635 is an English sans serif font. In the case of condition 1, font 633 Gothic is suitable for Japanese, and font 635 sans serif is suitable for English. In the case of condition 2, font 632 Mincho is suitable for Japanese, and font 634 serif is suitable for English.

[0073] Note that text objects in real space are not necessarily displayed facing the user. When a user tries to understand the content of a text object, there is a high possibility that the user will try to read the text object from the front. Therefore, a condition that the user faces the text object directly can be added to condition 1.

[0074] Note that text objects in real space are not necessarily displayed distorted to the user. When a user tries to understand the content of a text object, there is a high possibility that the user will try to read the text object from a position where it is easy to read (a position where all characters of the distorted text can be read at an equal distance). Therefore, a condition that the text object is distorted to the user may be added to condition 1. Note that this may also be changed to a condition that the text object is not distorted. The above control may be performed only in a situation where a flat GUI displayed in real space, such as the setting GUI panel 601, is being used.

[0075] In the above example, the font used for condition 1 is a Gothic (sans serif) font, and the font used for condition 2 is a Mincho (serif) font, but this is not limited to this. For example, there are various types of Gothic fonts, such as regular Gothic, round Gothic, and square Gothic.

[0076] Furthermore, conditions 1 and 2 may be further subdivided. For example, even if there is no change in the relative position of the user and the text object in real space, there are cases where the user is moving and cases where the user is stationary. The former may be condition 1-1 and the latter may be condition 1-2, and control may be performed to use an appropriate font for each. For example, among regular Gothic, round Gothic, and square Gothic, the font most suitable for condition 1-1 is regular Gothic, while the font most suitable for condition 1-2 is square Gothic, so control may be performed to switch fonts in this way.

[0077] Note that the font selected in Condition 1 (including Conditions 1-1 and 1-2 above) and Condition 2 may be dynamically changed rather than being predetermined. Alternatively, the optimal font may be selected from among the fonts provided. When selecting a font, some or all of the following conditions may be adopted:

[0078] Users are more likely to notice the existence of text objects with fonts that have higher density (where the area occupied by characters per unit area is larger), sans serif fonts rather than serif fonts, and Mincho fonts rather than Gothic fonts. Also, when a text object is displayed following the user's movement in real space, users are more likely to notice the existence of a text object with Mincho fonts rather than Gothic fonts. On the other hand, users can understand the content of a text object more easily with Gothic fonts than with Mincho fonts.

[0079] In addition to the selection of fonts, the text object may be shaped to be optimal from the following viewpoints related to the readability of the text object. Font type Font weight Font size Font color Text information layout Background on which to place the text ·Text information movement Display Media User actions Surrounding environment

[0080] Note that when detecting changes between conditions 1 and 2 and switching the font, the user may feel uncomfortable, which may hinder their understanding of the content of the text object. Therefore, it is possible to aim to reduce the user's discomfort by switching to a third font (intermediate font) before switching from the first font to the second font.

[0081] Additionally, if there is a possibility of switching fonts depending on conditions 1 and 2, variable fonts may be used. Digital fonts are usually divided into styles such as character width, weight, and italics, but variable fonts allow the entire font family to be managed as a single font file. Because it is possible to specify character width, weight, and italic strength in multiple stages, it is possible to make the transition from the first font to the second font smoother and less awkward.

[0082] Furthermore, when switching from the first font to the second font that is optimal for condition 2, there may be a large difference in fonts, which may cause a sense of incongruity for the user. In such a case, control may be performed so that a fourth font (the font that is next most optimal for condition 2 after the second font) is selected under condition 2.

[0083] In condition 1-1, the user is moving in the real space, so if the user is too focused on understanding the content of the text object, there is a risk of bumping into a wall, etc. Therefore, even if condition 1 is met, in condition 1-1, control may be performed to select a font that makes it easy to see the image of the real space in the background at the same time, rather than the optimal first font.

[0084] However, the display components (602 to 606) of the setting GUI panel 601 are not limited to these. The setting GUI panel 601 may also be a panel showing the progress or status of the game, or a GUI panel for editing moving images.

[0085] Although the above description concerns control in real space, it can also be applied to virtual space.

[0086] (Other embodiments) The various controls described above as being performed by the CPU of smartphone 200 may be performed by a single piece of hardware, or multiple pieces of hardware (e.g., multiple processors or circuits) may share the processing to control the entire smartphone.

[0087] Furthermore, although the present invention has been described in detail based on preferred embodiments thereof, the present invention is not limited to these specific examples, and various forms within the scope of the gist of the present invention are also included in the present invention. Furthermore, each of the above-described examples merely shows one example of the present invention, and each example can be combined as appropriate.

[0088] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0089] The disclosure of this embodiment includes the following configurations and methods.

Claims

1. a display control means for controlling the display of a virtual text object on a screen in such a manner that the virtual text object is superimposed on an image of real space; a first detection means for detecting a position of a user in the real space; a second detection means for detecting a line of sight of the user looking at the screen, the display control means controls the display of the text object in a first font when a first condition is satisfied, that is, the position of the text object displayed superimposed on the video image has not changed relative to the position of the user detected by the first detection means, and the position of the user's line of sight detected by the second detection means is the position of the text object; and when the first condition is not satisfied, controls the display of the text object in a second font different from the first font. An electronic device characterized by:

2. the display control means controls to display the text object in the second font when a second condition is satisfied, that is, the position of a text object displayed superimposed on the video image has not changed relative to the position of the user detected by the first detection means, and the position of the user's gaze detected by the second detection means is not the position of the text object, or the position of the text object displayed superimposed on the video image has changed relative to the position of the user detected by the first detection means; 2. The electronic device according to claim 1, wherein the electronic device is a semiconductor device.

3. When the text object is displayed superimposed on the video in accordance with the movement of the user's position, or when the display position of the text object is fixed and the user's position does not move, the position of the text object displayed superimposed on the video does not change relative to the user's position detected by the first detection means.

3. The electronic device according to claim 1 or 2.

4. when either the position at which the text object is displayed superimposed on the video or the position of the user moves, the position of the text object displayed superimposed on the video changes relative to the position of the user detected by the first detection means; 3. The electronic device according to claim 2.

5. a determination means for determining whether or not the position of the text object displayed superimposed on the video image has changed relative to the position of the user detected by the first detection means; 2. The electronic device according to claim 1, further comprising:

6. The display control means the first font is a Gothic or sans serif font; the second font is a Mincho font or a serif font; 2. The electronic device according to claim 1, wherein the electronic device is a semiconductor device.

7. the display control means controls the display of the text object in the first font and the second font while displaying the text object in the first font and the second font, when there is a change between when the first condition is satisfied and when the first condition is not satisfied.

2. The electronic device according to claim 1, wherein the electronic device is a semiconductor device.

8. the first font, the second font, and the third font are variable fonts; 8. The electronic device according to claim 7,

9. a display control step of controlling the display of the virtual text object on the screen so as to be superimposed on the image of the real space; a first detection step of detecting a position of a user in the real space; a second detection step of detecting a line of sight of the user looking at the screen, The display control step controls the display of the text object in a first font if a first condition is satisfied, that is, the position of the text object displayed superimposed on the video image has not changed relative to the position of the user detected in the first detection step, and the position of the user's line of sight detected in the second detection step is the position of the text object; and controls the display of the text object in a second font different from the first font if the first condition is not satisfied. A method for controlling an electronic device.

10. A program for causing a computer to execute the method for controlling an electronic device according to claim 9.

11. A computer-readable storage medium having recorded thereon a program for causing a computer to execute the electronic device control method according to claim 9.

Images