Control system, control method, and computer program

Abstract

This technology provides a way to reduce the cognitive load on users during spatial and physical transitions. [Solution] The display is controlled to show a connected space, which is a virtual space connecting a first space to a second space, within the user's field of view. The display is controlled to show spatial images from the perspective of the virtual body, corresponding to the virtual body's position in the connected space and the virtual body's line of sight, in accordance with the user's actions to move the virtual body corresponding to the user through the connected space. Furthermore, as the virtual body moves through the connected space toward the second space, the display is controlled to represent the change in the virtual body's field of view from the field of view corresponding to the first form in the first space to the field of view corresponding to the second form in the second space using spatial images displayed on the display.

Description

Technical Field

[0001] The present disclosure relates to a control system and a control method.

Background Art

[0002] Conventionally, a technique for providing a user with an artificial experience by displaying a virtual space through a display that spreads across the user's field of vision is known. In particular, VR (Virtual Reality) technology is known.

[0003] In recent years, a virtual space has also been displayed through a head-mounted display as a wearable device (see, for example, Patent Document 1). A user can feel a strong sense of reality in the virtual space that spreads across the entire field of vision through the head-mounted display. For example, a first-person perspective spatial image corresponding to the user's line of sight is displayed on the display.

[0004] In the virtual space, a virtual body is given to the user. The virtual body is, for example, an avatar. By using an avatar, it is possible to provide the user with an experience that is impossible in the real world through the virtual space.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] The virtual space and the virtual body may be significantly different from the real world in order to provide the user with a new experience. In this case, in the virtual space, the user experiences a discontinuous environmental change, which is an instantaneous or rapid environmental change that is not normally experienced in the real world.

[0007] The change in a user's field of view upon entering a virtual space is abrupt due to differences in the structure, appearance, and worldview between the space before entry and the space after entry. Furthermore, the change in the user's field of view is abrupt when the virtual body provided to the user is significantly different from the user's appearance before entering the virtual space, especially when the height and / or body shape are significantly different.

[0008] Therefore, in the initial stages of entering a virtual space, users need to recognize discontinuous or rapid changes in their surroundings and their own bodies, understand the situation, and act accordingly, resulting in a significant cognitive load. This cognitive load can occur not only when entering a virtual space from the real world, but also when entering a second virtual space from the first.

[0009] Therefore, according to one aspect of this disclosure, it is desirable to provide a technology that can reduce the cognitive load on the user during spatial transitions. [Means for solving the problem]

[0010] According to one aspect of this disclosure, a control system is provided for controlling a display that extends into the user's field of view. The control system comprises a first display control unit and a second display control unit.

[0011] The first display control unit is configured to control the display to show a connected space, which is a virtual space connecting the first space to the second space, in the user's field of view, in order to change the space visible to the user from the first space to the second space.

[0012] The second display control unit is configured to control the display to show a spatial image from the perspective of the virtual body, corresponding to the virtual body's position in the connected space and its line of sight, in accordance with the user's actions, in order to move the virtual body corresponding to the user through the connected space.

[0013] The second display control unit controls the display so that, as the virtual body moves through the connected space toward the second space, the change in the virtual body's field of view from the field of view corresponding to the first form in the first space to the field of view corresponding to the second form in the second space is represented by a spatial image displayed on the display.

[0014] This control system reduces the cognitive load caused by discontinuous spatial transitions by providing a connecting space. This system does not discontinuously change the virtual body's field of view to that of the second form when the virtual body enters a second space; instead, it changes the field of view continuously or gradually within the connecting space. Therefore, it reduces the user's cognitive load regarding visual perception during spatial transitions.

[0015] According to one aspect of this disclosure, a virtual body may have a third form for the connection space that is different from the first and second forms in at least a portion of the connection space. In the process of moving from the first space through the connection space to the second space, the virtual body may change from the first form to the third form, and further change from the third form to the second form.

[0016] In this case, the second display control unit may be configured to control the display so that, as the virtual body having a third form moves through the connected space toward the second space, the field of view of the virtual body having a third form gradually changes from the field of view corresponding to the first form in the first space to the field of view corresponding to the second form in the second space.

[0017] According to one aspect of this disclosure, the height and / or body shape may differ between the first and second forms. In this case, the second display control unit may control the display to represent the change in the virtual body's field of view that occurs when the virtual body's height and / or body shape gradually changes to the height and / or body shape corresponding to the second form as the virtual body moves through the connected space, using a spatial image displayed on the display.

[0018] According to one aspect of this disclosure, the second display control unit may control the display to represent in a spatial image the changes in the field of view of the virtual body that occur when the virtual body having a third form moves through a connected space, and the height and / or body shape of the virtual body gradually changes from a first height and / or body shape corresponding to a first form to a second height and / or body shape corresponding to a second form.

[0019] Such control reduces the cognitive load on the user when the user's height and / or build in the second space differs significantly from their height and / or build at the entrance to the connecting space, compared to discontinuously changing the field of view in accordance with the change in height and / or build upon entering the second space.

[0020] According to one aspect of this disclosure, the size ratio of the virtual body to the space may differ between the first space and the second space. In this case, the second display control unit may be configured to control the display in a spatial image to represent the change in the virtual body's field of view that occurs when the virtual body moves through the connected space, gradually changing from the viewpoint when the virtual body views the first space in its first form to the viewpoint when the virtual body views the second space in its second form.

[0021] According to one aspect of this disclosure, the second display control unit may be configured to control the display so as to represent the changes in the virtual body's field of view that occur when the virtual body moves at a speed that is a predetermined multiple of the actual user's movement speed, or at a preset movement speed, in a spatial image during the process of the virtual body moving through the connected space.

[0022] According to one aspect of this disclosure, the third form may be a virtual body with an appearance that does not explicitly define its body shape. In this case, when the second form in the second space differs significantly from the first form in the first space, the cognitive load on the user can be reduced compared to when the body shape is represented discontinuously.

[0023] According to one aspect of the present disclosure, each of the first space and the second space may be a virtual space. For example, both the first space and the second space may be virtual spaces. For example, one of the first space and the second space may be a virtual space, and the other may be a real space. The control system can display, on the display, a space image corresponding to the visual field of the virtual body in, for example, the first space or the second space as the real space.

[0024] According to one aspect of the present disclosure, the second display control unit can control the display to express the height and / or moving speed of the line of sight of the virtual body in the connection space by using the display of an object through the space image.

[0025] According to one aspect of the present disclosure, the connection space may be a virtual space in which a plurality of gates are arranged at intervals from the first space toward the second space. According to one aspect of the present disclosure, the second display control unit can control the display to express the height and / or moving speed of the line of sight of the virtual body in the connection space by using the display of the plurality of gates through the space image. According to such control, it is possible to express changes in the height, body type, and / or moving speed of the virtual body through the display of the object.

[0026] According to one aspect of the present disclosure, the control system may further include an acquisition unit configured to acquire body information capable of specifying the height of the user. According to one aspect of the present disclosure, the second display control unit can control the display to express the size of the object in the space image in a size based on the difference between the height of the user specified from the body information and the height of the virtual body.

[0027] According to such control, it is possible to display an object in the virtual space to the user in a size corresponding to the height of the virtual body while considering the user's visual sensation in the real world.

[0028] According to one aspect of this disclosure, the control system may further include a sound control unit configured to control a speaker so that the speaker outputs ambient sounds corresponding to the position of the virtual body. According to one aspect of this disclosure, the sound control unit can control the speaker such that as the virtual body approaches a second space in the connected space, the volume of the ambient sounds corresponding to the second space output by the speaker changes. Such control makes it possible to reduce the cognitive load caused by spatial transitions not only visually but also auditorily.

[0029] According to one aspect of this disclosure, the second display control unit can set a virtual body to a second form having an appearance corresponding to the avatar image by placing an avatar image corresponding to the second form in the boundary area between the connection space and the second space, and by assigning an appearance corresponding to the avatar image to the virtual body when the virtual body overlaps with the avatar image.

[0030] This type of control allows users operating a virtual body to feel as if they are putting on clothes when assigning an appearance to the virtual body, thereby reducing cognitive load compared to suddenly changing the appearance of the virtual body at the entrance to the second space.

[0031] According to one aspect of this disclosure, the second display control unit may, at a specific point in the connection space, replace a virtual body having a first appearance that has entered the connection space from the first space with a third appearance, and may also place an avatar image corresponding to the first appearance around the specific point.

[0032] This type of control allows the virtual body to change from an appearance corresponding to a first form to a third form with an abstract or simple appearance, as if removing clothes. Therefore, it is possible to reduce the cognitive load compared to abruptly changing the appearance of the virtual body from the first form to the second form.

[0033] According to one aspect of this disclosure, the spatial image displayed on the screen may be a first-person perspective spatial image corresponding to the gaze of a virtual body that corresponds to the user's gaze. The spatial image may also be displayed from other perspectives, such as a third-person perspective. However, displaying from a first-person perspective may impose a high cognitive burden on the user in response to sudden changes accompanying spatial transitions, due to the stronger sense of immersion the user has in the virtual space compared to displaying from other perspectives.

[0034] Therefore, applying the above-described control to a system that displays spatial images from a first-person perspective makes it possible to effectively reduce cognitive load.

[0035] According to one aspect of this disclosure, a control method for controlling a display that extends into the user's field of view may be provided. The control method may include, as a first process, a process of controlling the display to show a connected space, which is a virtual space connecting the first space to the second space, in order to change the space that appears in the user's field of view from a first space to a second space.

[0036] The control method may include, as a second process, a process of controlling the display to show a spatial image as seen from the virtual body, corresponding to the virtual body's position in the connected space and the virtual body's line of sight, in accordance with the user's actions, in order to move the virtual body corresponding to the user through the connected space.

[0037] The second process may include controlling the display so that, as the virtual body moves through the connected space toward the second space, the change in the virtual body's field of view from the field of view corresponding to the first form in the first space to the field of view corresponding to the second form in the second space is represented by a spatial image displayed on the display.

[0038] This control method, like the control system described above, can reduce the user's cognitive load regarding visual perception.

[0039] According to one aspect of this disclosure, a virtual body may have a third form for the connection space that is different from the first and second forms in at least a portion of the connection space. In the process of moving from the first space through the connection space to the second space, the virtual body may change from the first form to the third form, and further change from the third form to the second form.

[0040] In this case, the second process may include controlling the display such that, as the virtual body having a third form moves through the connected space toward the second space, the field of view of the virtual body having a third form changes from the field of view corresponding to the first form in the first space to the field of view corresponding to the second form in the second space.

[0041] According to one aspect of this disclosure, the height and / or body shape may differ between the first and second forms. The second process may include controlling the display to represent the changes in the virtual body's field of view that occur when the virtual body's height and / or body shape gradually changes to the height and / or body shape corresponding to the second form as the virtual body moves through the connected space, using spatial images displayed on the display.

[0042] According to one aspect of this disclosure, the second process may include controlling the display to represent in a spatial image the changes in the virtual body's field of view that occur when the virtual body having a third form moves through a connected space and its height and / or body shape gradually changes from a first height and / or body shape corresponding to a first form to a second height and / or body shape corresponding to a second form.

[0043] According to one aspect of this disclosure, the size ratio of the virtual body to space may differ between the first space and the second space. In this case, the second process may include controlling the display to represent in a spatial image the change in the virtual body's field of view that occurs as the virtual body moves through the connected space, gradually changing from the viewpoint when the virtual body views the first space in its first form to the viewpoint when the virtual body views the second space in its second form.

[0044] According to one aspect of this disclosure, the second process may include controlling the display to represent in a spatial image the changes in the virtual body's field of view that occur when the virtual body moves at a speed that is a predetermined multiple of the actual user's movement speed, or at a preset movement speed, as the virtual body moves through the connected space.

[0045] According to one aspect of this disclosure, in the control method described above, the third form may be the form of a virtual body having an appearance in which the body shape is not explicitly defined.

[0046] According to one aspect of this disclosure, the second process may include controlling the display to represent the height and / or speed of the line of sight of a virtual body in connected space, using the representation of an object through a spatial image.

[0047] According to one aspect of this disclosure, the connection space may be a virtual space in which a plurality of gates are arranged at intervals from a first space to a second space. The second process may include controlling the display to represent the height of the line of sight and / or the speed of movement of a virtual body in the connection space, using the display of the plurality of gates through a spatial image.

[0048] According to one aspect of this disclosure, the control method may further include obtaining physical information that can identify the user's height. The second process may include controlling the display to represent the size of an object in a spatial image based on the difference between the user's height identified from the physical information and the height of the virtual body.

[0049] According to one aspect of this disclosure, the control method involves controlling a speaker to output ambient sounds corresponding to the position of a virtual body, and further includes controlling the speaker such that the volume of ambient sounds corresponding to the second space output by the speaker changes as the virtual body approaches the second space in the connected space.

[0050] According to one aspect of this disclosure, the second process may include placing an avatar image corresponding to the second form in the boundary area between the connection space and the second space, and when the virtual body overlaps with the avatar image, assigning the virtual body an appearance corresponding to the avatar image, thereby setting the virtual body to a second form having an appearance corresponding to the avatar image.

[0051] According to one aspect of this disclosure, the second process may include, at a specific point in the connected space, replacing a virtual body having a first form that entered the connected space from the first space with a third form, and placing an avatar image corresponding to the first form around the specific point.

[0052] According to one aspect of this disclosure, a computer program may be provided for causing a computer to perform the first and second processes in the control method described above. The computer program may be recorded on a computer-readable recording medium. [Brief explanation of the drawing]

[0053] [Figure 1] This is a block diagram showing the electrical configuration of electronic equipment. [Figure 2] This diagram illustrates the configuration of an exemplary connection space as seen from the source space. [Figure 3] This diagram illustrates the configuration of an example of a connected space as seen from within the connected space. [Figure 4] This diagram illustrates the arrangement of multiple gates in a connected space. [Figure 5] Figures 5A, 5B, and 5C are explanatory diagrams (part 1) regarding avatar switching. [Figure 6] Figures 6A and 6B are explanatory diagrams (part 2) regarding avatar switching. [Figure 7] This diagram illustrates how height changes can be represented in connected space. [Figure 8] This diagram illustrates how the relative height change of a virtual body is represented by adjusting the size of objects in the connected space. [Figure 9] This is a flowchart (part 1) representing the display control process (display control process A) executed by the processor. [Figure 10] This is a flowchart (part 2) representing the display control process (display control process A) executed by the processor. [Figure 11] This is a flowchart illustrating the sound control process performed by the processor. [Figure 12] This is a flowchart that partially represents the display control process (display control process B) executed by the processor. [Modes for carrying out the invention]

[0054] Exemplary embodiments of the present disclosure are described below with reference to the drawings. [First Embodiment] The electronic device 10 of this embodiment, shown in Figure 1, is a device capable of displaying an XR (Cross Reality) space in front of the user's field of vision. An XR space is an example of a virtual space. Examples of XR spaces include VR (Virtual Reality) spaces, AR (Augmented Reality) spaces, and MR (Mixed Reality) spaces. The electronic device 10 may be configured as a wearable device, including a head-mounted display.

[0055] The electronic device 10 includes a processor 11, memory 13, storage 15, communication interface 17, display 21, speaker 23, camera 25, and sensor 27.

[0056] The processor 11 is configured to execute processing according to a computer program stored in the storage 15. The memory 13 is a primary storage device equipped with RAM and is used as a work area when the processor 11 is executing processing.

[0057] Storage 15 is a secondary storage device and consists of electrically rewritable non-volatile memory such as flash memory. Storage 15 stores computer programs as well as various data used when executing processes that follow the computer programs.

[0058] The communication interface 17 is configured to enable wireless communication. For example, the communication interface 17 is configured as a short-range wireless communication interface. Examples of short-range wireless communication interfaces include a wireless LAN (Local Area Network) interface of the IEEE 802.11 standard and a communication interface of the Bluetooth® standard.

[0059] The display 21 is configured to display a spatial image in front of the user's field of view. The spatial image, in this context, is a three-dimensional image representing space and objects within that space. Examples of spatial images include VR spatial images, AR spatial images, and MR spatial images.

[0060] The display 21 includes multiple optical systems and a display body corresponding to the user's left and right eyes for displaying spatial images to the user using XR technology. The display 21 is controlled by the processor 11.

[0061] Speaker 23 is equipped with multiple speakers corresponding to the left and right ears to enable three-dimensional sound. Speaker 23 outputs ambient sounds corresponding to the position of the user's virtual body in the virtual space.

[0062] Camera 25 is provided on the electronic device 10 so as to be able to capture images in front of the user when the electronic device 10 is attached. Camera 25 comprises multiple cameras. Camera 25 is used, for example, to determine the three-dimensional shape of the real space around the user and to recognize objects. Camera 25 is also used to determine the user's position, posture, gaze, and movement.

[0063] Camera 25 is also used to capture the movements of the user's hands and fingers. Camera 25 is used, for example, to receive input from the user's hands and fingers regarding the graphical user interface displayed to the user through the display 21.

[0064] Sensor 27 includes a gyroscope and an accelerometer for detecting the user's posture and movement. Sensor 27 may also include a depth sensor. Sensor 27 may also include various other sensors necessary for spatial image processing. Sensor 27 may also include a sensor for detecting the user's height and / or line of sight.

[0065] A distinctive feature of the electronic device 10 in this embodiment is that when the user wearing the electronic device 10 wishes to change the space visible in their field of view from a first space SP1 to a second space SP2, the connected space SP3 is displayed through the display 21 that extends into the user's field of view.

[0066] The first space SP1 is the space that the user sees through the display 21. The connected space SP3 is a virtual space that connects the first space SP1 and the second space SP2. The connected space SP3 has an entrance P1 at the boundary with the first space SP1 and an exit P2 at the boundary with the second space SP2. The connected space SP3 is provided to reduce the cognitive load on the user when changing the space perceived by the user through the display 21 from the first space SP1 to the second space SP2.

[0067] In the following, the first space SP1 will also be referred to as the source space SP1, and the second space SP2 will also be referred to as the destination space SP2. The connected space SP3 corresponds to the intervening space between the source space SP1 and the destination space SP2. The source space SP1 is not necessarily the space in which the user is actually located. The source space SP1 may be a virtual space in which the virtual body of the user wearing the electronic device 10 exists.

[0068] In one example, the source space SP1 and the destination space SP2 are virtual spaces. In another example, one of the source space SP1 and the destination space SP2 is a real space. If the source space SP1 is a real space, the connected space SP3 is displayed as part of the real space that is the source space SP1. The real space on which the connected space SP3 is displayed can be called a type of virtual space, or more specifically, an augmented reality space.

[0069] The exemplary connection space SP3 shown in Figure 2 is a tunnel-like virtual space. The user can move through the connection space SP3 via a virtual body, as shown in Figure 3. For example, the user can move through the connection space SP3 via the virtual body by manipulating it through the graphical user interface described above. This manipulation corresponds to the user's actions to move the virtual body through the connection space SP3 to the destination space SP2.

[0070] The movement of the virtual body may be linked to the user's movements, which are determined using the camera 25 and sensor 27. In this case, the user can move through the virtual body on the connected space SP3 by formal or actual walking movements. The user can move the virtual body to the destination space SP2 by manipulating the virtual body to pass through the connected space SP3.

[0071] The virtual body is a virtual body that corresponds to the field of view provided to the user wearing the electronic device 10 via the display 21. That is, the display 21 displays a spatial image from a viewpoint relative to the virtual body. In order to display the virtual space and / or real space to the user via the display 21 from a first-person or third-person viewpoint relative to the virtual body, the virtual body is configured with information that defines its appearance.

[0072] For example, a virtual body may be configured with information defining its height. In this case, the virtual body's eye level and body shape can be uniquely determined by its height. Alternatively, a virtual body may be configured with information defining its body shape. A virtual body may also be configured with information defining its eye level. Eye level and body shape are also information that defines the appearance of the virtual body.

[0073] The appearance of a virtual body includes at least a partially transparent appearance. The appearance of a virtual body may be defined by the shape, position, and / or size of body parts. This may define the height, eye level, and body shape of the virtual body.

[0074] Furthermore, the movement speed of a virtual body can be defined independently of its height, body type, and the user's walking motion. Depending on the virtual body's appearance, such as when it has no legs or is riding a vehicle, the concept of walking may not exist. Even when a virtual body has legs, a mode of movement in which the virtual body moves in a parallel direction while keeping the legs stationary is also conceivable.

[0075] The user is shown, via the display 21, an image of the space as seen from the perspective of the virtual body, which corresponds to the virtual body's position and line of sight in the virtual space. The flow of the field of view, corresponding to the virtual body's height, body shape, and / or movement speed, is represented by the spatial image displayed on the display 21.

[0076] Virtual bodies are given an appearance. In the following, a virtual body with an appearance will be specifically referred to as an avatar.

[0077] In the exemplary connection space SP3, as shown in Figure 4, multiple gates 30 are arranged at intervals from the source space SP1 to the destination space SP2. The gates 30 are, for example, torii gates.

[0078] Figure 4 shows how multiple gates 30 are arranged at intervals in the connection space SP3, from the entrance P1 located at the boundary with the source space SP1 to the exit P2 located at the boundary with the destination space SP2.

[0079] Multiple gates 30 are examples of objects placed in the connection space SP3 to represent the height of the virtual body's line of sight and / or its movement speed. One or more objects may be placed in the connection space SP3 to represent the height of the virtual body's line of sight and / or its movement speed.

[0080] For example, in the connecting space SP3, multiple rings may be placed at intervals from the entrance P1 to the exit P2 of the connecting space SP3, serving as objects to replace multiple gates 30. Multiple guide poles may also be installed at intervals in the connecting space SP3. The objects may also be the inner walls of a tunnel with a pattern.

[0081] The connecting space SP3 is not limited to a tunnel shape. The connecting space SP3 may also be staircase-shaped. If the connecting space SP3 is staircase-shaped, the staircase structure itself can function as an object for representing the height of the virtual body's line of sight and / or its movement speed.

[0082] Figures 5A, 5B, 5C, 6A, and 6B illustrate the switching of avatars during the virtual body's movement from entrance P1 to exit P2 in the connected space SP3. In the examples shown in Figures 4, 5A, 5B, 5C, 6A, and 6B, the source space SP1 and destination space SP2 are virtual spaces.

[0083] As shown in Figure 4, the source space SP1 has a first avatar AV1, and the destination space SP2 has a second avatar AV2. In the example shown in Figure 4, the second avatar AV2 is the same height as the first avatar AV1, and shares the same eye level and movement speed.

[0084] In the example shown in Figure 5A, near the entrance P1 in the connected space SP3, an avatar image 41 is placed to transform the user's virtual body from the first avatar AV1 to the connected space avatar AV3, in order to compensate for the discontinuity in appearance and body shape between the first avatar AV1 and the second avatar AV2. This avatar image 41 is a three-dimensional model of the connected space avatar AV3. The avatar image 41 functions as support information to facilitate understanding of the transformed appearance.

[0085] Near the exit P2 of the connection space SP3, an avatar image 43 is placed to transform the user's virtual body into a second avatar AV2. This avatar image 43 is a three-dimensional model of the second avatar AV2. The avatar image 43 functions as support information to facilitate understanding of the transformed appearance.

[0086] The dotted lines in Figures 5A, 5B, 5C, 6A, and 6B represent avatar images 41, 43, and 45 placed in the connection space SP3 as support information to facilitate understanding of the form to be transformed into, or as support information to facilitate understanding that the pre-transformation form is being discarded when transforming into the next form by retaining the pre-transformation form during the transformation.

[0087] According to this embodiment, in order to reduce the cognitive load caused by the instantaneous switching of the virtual body's appearance from one appearance to another, the appearance of the virtual body can be transitioned as if putting on or taking off a costume.

[0088] Figure 5B illustrates that when a virtual body having the appearance of the first avatar AV1 shown in Figure 5A overlaps with the avatar image 41 for the connection space near the entrance P1 of the connection space SP3, the virtual body is given an appearance corresponding to the avatar image 41 for the connection space, and the virtual body transforms into the avatar AV3 for the connection space.

[0089] When the user's virtual body transforms into avatar AV3 for the connected space, avatar image 45, which corresponds to the first avatar AV1 before the transformation, is displayed around the position at the time of transformation (particularly on the transition source space SP1 side), like a costume that the virtual body has taken off. In Figures 5B, 5C, 6A, and 6B, avatar image 45, shown by a dotted line, corresponds to the three-dimensional model of the discarded first avatar AV1.

[0090] In other words, at a specific point in the connection space SP3, the appearance of the virtual body having the appearance of the first avatar AV1 that entered the connection space SP3 from the source space SP1 is replaced with the appearance corresponding to avatar image 41, and avatar image 45 corresponding to the first avatar AV1 is placed at the specific point.

[0091] Figure 5C illustrates how the user's virtual body, transformed into avatar AV3 for the connected space, is moving towards exit P2 of the connected space SP3, with an image of the first avatar AV1 (i.e., avatar image 45) placed there as supporting information to facilitate understanding that the user is shedding their previous form by retaining their previous form when transforming into the next form.

[0092] Figure 6A illustrates that when a user's virtual body, which has the appearance of avatar AV3 for the connected space (see Figure 5C), overlaps with avatar image 43 corresponding to the second avatar AV2 near exit P2 of the connected space SP3, the virtual body is given the appearance corresponding to avatar image 43, and the virtual body transforms into the second avatar AV2.

[0093] Figure 6B illustrates the process of a user's virtual body, having transformed into the second avatar AV2, moving from the exit P2 of connection space SP3 to the destination space SP2, with an image of avatar AV3 for the connection space (i.e., avatar image 41) placed around the position at the time of transformation (particularly on the connection space SP3 side) to facilitate understanding that the user's virtual body, having transformed into the second avatar AV2, is shedding its previous form by retaining its previous form during the transformation.

[0094] Figure 7 illustrates that when the height of the second avatar AV2 is shorter than that of the first avatar AV1, the electronic device 10 represents through the display 21 the change in the virtual body's field of view that occurs when the virtual body (particularly avatar AV3) moves from the entrance P1 to the exit P2 of the connection space SP3, and the height of the virtual body gradually changes from that of the first avatar AV1 to that of the second avatar AV2. The fact that the height of the second avatar AV2 is different from that of the first avatar AV1 corresponds to the fact that the appearance of the second avatar AV2 is different from that of the first avatar AV1. "Height" may be read as "body type". For example, when the heights are different, the amount of movement of each body part, including stride length, is also different. The amount of movement of each body part can be set to a predetermined ratio of the user's height and body type to the height and body type of each avatar.

[0095] The representation of changes in the field of view in this connected space SP3 is done to suppress the increased cognitive load on the user in understanding spatial changes caused by discontinuous changes in the field of view associated with switching virtual bodies from the first avatar AV1 to the second avatar AV2.

[0096] A change in height can correspond to a change in the avatar's size ratio to space. When representing the change in field of view in a spatial image as the size ratio R2 of the second avatar AV2 to the destination space SP2 becomes smaller compared to the size ratio R1 of the first avatar AV1 to the source space SP1, the electronic device 10 controls the display 21 so that the multiple gates 30 are displayed larger or higher as they approach the exit P2 of the connection space SP3, as shown in Figure 8. As an example, consider a case where objects similar to those in the real world exist in the source space SP1 and the destination space SP2, and the first avatar AV1 is an adult avatar of 180cm, while the second avatar AV2 is a child avatar of 110cm. In this case, as shown in Figure 8, the change in field of view due to the decrease in height can be represented in a spatial image by controlling the display 21 so that the multiple gates 30 are displayed larger or higher as they approach the exit P2 of the connection space SP3.

[0097] The height of the real-world user corresponding to the first avatar AV1 and the second avatar AV2 is, of course, constant. The electronic device 10 gives the user the sensation of being miniaturized or shorter by setting the virtual space objects that the user sees through the display 21 to be larger or taller. Conversely, by setting the virtual space objects that the user sees through the display 21 to be smaller or shorter, the device can give the user the sensation of being giant or taller.

[0098] After the user's virtual body moves to the destination space SP2, the electronic device 10 needs to display the spatial image on the display 21 within the field of view of the second avatar AV2, which is the avatar of the destination space SP2.

[0099] However, if the spatial image displayed on the display 21 is discontinuously switched from a spatial image matched to the field of view of the first avatar AV1 to a spatial image matched to the field of view of the second avatar AV2, the user's cognitive load increases due to the discontinuous change in the field of view displayed on the display 21. Therefore, in this embodiment, a continuous or gradual change in the field of view is represented during the process in which the user's virtual body moves to the connected space SP3.

[0100] Specifically, in the electronic device 10 of this embodiment, the processor 11 executes the display control process A shown in Figures 9 and 10, thereby avoiding discontinuous changes in the field of view caused by differences in the appearance of the avatar in the source space SP1 and the destination space SP2, and reducing the cognitive load on the user.

[0101] A display control process (display control process A) according to an exemplary embodiment of the present disclosure will be described with reference to Figures 9 and 10. When display control process A is started, the processor 11 waits until the user makes an operation to instruct a transition to another space (S110). When the operation is made (Yes in S110), the processor 11 acquires the user's physical information (S120). The physical information can be acquired, for example, by reading from the storage 15.

[0102] The physical information may include information describing the user's height, which has been pre-entered by the user through the operation of the user interface, or it may include information describing the user's height and / or line of sight, which has been measured using the sensor 27. Based on the height information described in the height information, a certain percentage of the height may be determined as the line of sight, or the height may be determined based on the line of sight information.

[0103] In the subsequent S130, the processor 11 acquires destination space information. The destination space information includes the size of the destination space SP2 and the user's avatar information in the destination space SP2. The avatar information includes information describing the avatar's height in the destination space SP2. The avatar's line of sight height can be determined from this avatar information, particularly the height information.

[0104] In the subsequent S140, the processor 11 determines the size ratio R1 between the source space SP1 and the avatar in the source space SP1, and the size ratio R2 between the destination space SP2 and the avatar in the destination space SP2.

[0105] If the spatial size of the source space SP1 and the spatial size of the destination space SP2 are the same, the difference between the size ratio R1 of the avatar in the source space SP1 and the size ratio R2 of the avatar in the destination space SP2 can be understood as the size ratio of the avatars in the source space SP1 and the destination space SP2 (i.e., the first avatar AV1 and the second avatar AV2).

[0106] Spatial size is the standard size of a space. Each space may be given its own standard size depending on the design environment, implementation environment, and the design of the content's worldview. For example, in two spaces with the same spatial size, the same type of object may be represented at the same size. On the other hand, in two spaces with different spatial sizes, the same type of object may be represented at different sizes. For example, when creating content in which dwarves experience a land of giants, it is possible to create a chair for the giants at 10 meters and a dwarf at 1.8 meters if the spatial size is the same as reality, but it is also possible to create a chair for the giants at 1 meter and a dwarf at 0.18 meters if the spatial size is smaller than reality.

[0107] In the subsequent S150, the processor 11 calculates the change ΔR of the size ratios R1 and R2. In one example, the change ΔR is the increment of the size ratio R2 relative to the size ratio R1 ΔR = R2 / R1. In another example, the change ΔR is the difference between the avatar size R1 and the avatar size R2 ΔR = R2 - R1.

[0108] In the subsequent S160, the processor 11 controls the display 21 to display the connection space SP3 based on the change amount ΔR in front of the transition source space SP1 that the user is viewing through the display 21.

[0109] The processor 11 controls the display 21 to display a connected space SP3 in which, for example, multiple gates 30 are arranged as follows: The processor 11 sets the height H1 of the leading gate 30A (see Figure 8), which is the leading gate 30 closest to the entrance P1 of the connected space SP3, to height H0 (H1=H0), and sets the height H2 of the last gate 30B, which is the last gate 30 closest to the exit P2 of the connected space SP3, to H2=H1 / ΔR using height H0 and increment ΔR. In another example, the processor 11 sets the height H2 of the last gate 30B to H2=H0-ΔR using difference ΔR.

[0110] When the total number of gates 30 from the first gate 30A to the last gate 30B is N, the processor 11 sets the height of the n-th gate 30 to α(n)*(H2 - H1) + H1 for n = 2, …, N. The first gate 30 is the first gate 30A, and the N-th gate 30 is the last gate 30B. α(n) is a function of n such that α(n)>0, α(n)>α(n - 1), and α(N)=1. α(n) is, for example, a linear function. α(n) is, for example, α(n)=(n - 1) / (N - 1).

[0111] The height H0 is set based on, for example, the user's height and the size ratio R1 between the source space SP1 and the avatar in the source space SP1. According to another example, the height H0 is set based only on the user's height. According to still another example, the height H0 is set to a fixed height.

[0112] In addition, when H1 = H2 (in other words, when R1 = R2), the processor 11 arranges the N gates 30 in the connection space SP3 at equal intervals D0.

[0113] On the other hand, when H1 ≠ H2 (in other words, when R1 ≠ R2), the processor 11 arranges the plurality of gates 30 at unequal intervals according to the change in the height of the gates 30. For example, when the processor 11 gradually sets the gates 30 higher from the first gate 30A to the last gate 30B, the processor 11 gradually increases the interval between adjacent gates 30.

[0114] For example, the processor 11 can set the distance between the n-th gate 30 and the (n + 1)-th gate 30 to β(n)*D0 for n = 1, …, N - 1. β(n) is set as a function that satisfies the inequality 0 < β(n) < β(n - 1) < 1 when R2 > R1 and satisfies the inequality β(n)>β(n - 1)>1 when R2 < R1. β(n) is, for example, a linear function of n. When R1 = R2, β(n)=1.

[0115] In the subsequent S170, the processor 11 places an avatar image 41 for the connection space near the entrance P1 in the connection space SP3, sized to correspond to the user's height. The reason for placing the avatar image 41 sized to correspond to the user's height is that when the spatial image is displayed on the display 21 from the viewpoint of the virtual body, changes in the height of the virtual body are actually represented relatively by adjusting the scale of the spatial image around the avatar.

[0116] In the subsequent S180, the processor 11 places an avatar image 43 for the destination space near the exit P2 of the connection space SP3, sized to correspond to the user's height. The reason for placing the avatar image 43 at a size corresponding to the user's height is as described above.

[0117] In the subsequent S190, the processor 11 waits until the user's virtual body enters the connection space SP3 through entrance P1, and the virtual body overlaps with the avatar image 41 for the connection space, which is placed near entrance P1 of the connection space SP3. When the processor 11 determines that the virtual body has entered the connection space SP3 through entrance P1 and that the virtual body has overlapped with the avatar image 41 (Yes in S190), it executes the process in S200.

[0118] In S200, the processor 11 transforms the user's virtual body into the avatar AV3 for the connected space by assigning an appearance to the avatar image 41 for the connected space. The processor 11 further places the avatar image 45 corresponding to the first avatar AV1 in the source space SP1 around the transformation point (especially on the source space SP1 side) as support information to facilitate understanding that the user is shedding their pre-transformation appearance by retaining their pre-transformation appearance during the transformation.

[0119] After executing the process in S200, the processor 11 maintains the user's virtual body in the avatar AV3 for the connected space and controls the display 21 to display a spatial image of the virtual body as a reference to the virtual body, either a first-person or third-person viewpoint, corresponding to the virtual body's position in the connected space SP3 and the virtual body's line of sight, until the virtual body reaches the exit P2 of the connected space SP3 (S210, S220).

[0120] In other words, the processor 11 controls the display 21 to display an image of the space that appears in the virtual body's field of view in the connected space SP3. The virtual body's gaze is controlled, for example, in response to the virtual body's operation through a graphical user interface. The virtual body's gaze may also be controlled in conjunction with the user's gaze movements detected by the sensor 27. In other words, the virtual body's gaze may correspond to the user's gaze.

[0121] As described above, in the connection space SP3, the multiple gates 30 are arranged such that their height and spacing gradually change from the first gate 30A to the last gate 30B, according to the change amount ΔR of the size ratio R1,R2. Therefore, as the virtual body moves through the connection space SP3 toward the destination space SP2, the change in the virtual body's field of view corresponding to this height change is represented in the spatial image displayed on the display 21, as if the virtual body's height were gradually changing to match the height of the second avatar AV2.

[0122] Similarly, as the virtual body moves through the connection space SP3 toward the destination space SP2, the change in the virtual body's field of view corresponding to the change in movement speed (and / or stride length) is represented by the spatial image displayed on the display 21, as if the virtual body's movement speed (and / or stride length) were gradually changing to match the movement speed (and / or stride length) corresponding to the second avatar AV2.

[0123] In other words, in this embodiment, by controlling the height and spacing of the gates 30, changes in the field of view of the virtual body in the connected space SP3 are represented in a way that corresponds to changes in height and movement speed (and / or stride length) from the first avatar AV1 to the second avatar AV2. Changes in the field of view include changes in line of sight height and line of sight flow.

[0124] The processor 11 repeatedly performs the display of spatial images corresponding to the virtual body's position and line of sight, as described above, until the virtual body reaches the exit P2 of the connected space SP3, and thereby the virtual body overlaps with the avatar image 43 placed near the exit P2 (S210, S220).

[0125] When the virtual body reaches the exit P2 of the connection space SP3 and overlaps with the avatar image 43 (Yes in S220), the processor 11 transforms the user's virtual body into an avatar for the transition space, i.e., a second avatar AV2, by assigning an appearance corresponding to the avatar image 43 to the virtual body (S230).

[0126] The processor 11 further places the avatar image 41 corresponding to the avatar AV3 in the connection space SP3 around the transformation point (especially on the connection space SP3 side) as support information to facilitate understanding that the pre-transformation form is being shed by retaining the pre-transformation form during the transformation. Subsequently, the processor 11 terminates the display control process A shown in Figures 9 and 10 and controls the display 21 to display a spatial image of the second avatar AV2 from a first-person or third-person perspective, corresponding to the movement of the second avatar AV2 in the transition destination space SP2.

[0127] In addition, when the virtual body passes through connection space SP3, the processor 11 may control the display 21 to close connection space SP3, or it may leave connection space SP3 open and perform display control of the display 21 regarding the destination space SP2.

[0128] Figure 11 illustrates the sound control process that the processor 11 repeatedly executes in parallel with the display control process A described above when the virtual body is in the connection space SP3. In this sound control process, the processor 11 determines the distance L1 from the source space SP1 and the distance L2 to the destination space SP2 of the virtual body's current position (S310).

[0129] Distance L1 corresponds to the distance L1 from the entrance P1 of the connected space SP3 to the current position. Distance L2 corresponds to the distance L2 from the exit P2 of the connected space SP3 to the current position. Distances L1 and L2 can be understood as distances evaluated in the reference coordinate system before adjusting the height and spacing of gate 30 according to the change amount ΔR.

[0130] In the subsequent S320, the processor 11 determines the sound pressure of the ambient sound in the source space SP1 and the sound pressure of the ambient sound in the destination space SP2 at the location where the virtual body exists, based on the distances L1 and L2.

[0131] For example, the processor 11 can set the sound pressure attenuation rate at a distance L from the sound source to γ(L), determine the sound pressure of the ambient sound in the source space SP1 at the location where the virtual body exists as γ(L1)·P1, and determine the sound pressure of the ambient sound in the destination space SP2 as γ(L2)·P2. P1 is the sound pressure of the ambient sound at the point adjacent to the connection space SP3 in the source space SP1, and P2 is the sound pressure of the ambient sound at the point adjacent to the connection space SP3 in the destination space SP2.

[0132] In the subsequent S330, the processor 11 controls the speaker 23 so that a mixed sound, which is an ambient sound obtained by mixing the ambient sound components of the source space SP1 and the destination space SP2 in a ratio corresponding to the sound pressure determined above, is output from the speaker 23.

[0133] The processor 11 repeatedly performs sound control processing in this manner, thereby controlling the speaker in the connected space SP3 to output an ambient sound, which is a mixture of the ambient sound from the source space SP1 and the ambient sound from the destination space SP2, from the virtual body at a mixing ratio corresponding to its position. Here, the method for generating the mixed sound has been described assuming that there is no sound source that generates ambient sound in the connected space SP3, but there may be a sound source that generates ambient sound in the connected space SP3. In this case, the ambient sound component from the corresponding sound source in the connected space SP3 can also be added to generate the mixed sound output from the speaker 23.

[0134] The above explanation of display control process A and the additional sound control process was based on the premise that a first avatar AV1 is set in the source space SP1. However, when the source space SP1 is real space and the display 21 displays a spatial image of the real space as seen from the user's viewpoint, the processor 11 can execute a display control process (display control process B) in which S110 to S160 of display control process A shown in Figure 9 are replaced with S410 to S460 shown in Figure 12. For example, when the source space SP1 is real space and the destination space SP2 is a virtual world, in the source space SP1, in order to display a spatial image from the user's viewpoint through the display 21, the electronic device 10 sets the connected space SP3 and the virtual body of the connected space SP3 based on the size of the real space and the user's height, as well as the size of the destination virtual space and the avatar size, and the spatial image as seen from that virtual body is displayed on the display 21.

[0135] When display control process B is started, processor 11 executes processes S410, S420, and S430, similar to processes S110, S120, and S130 in display control process A. Subsequently, in S440, processor 11 determines the size ratio R0 between the real space size and the user's height, and the size ratio R2 between the destination space SP2 and the avatar in the destination space SP2.

[0136] In the subsequent S450, the processor 11 calculates ΔR, which is the change between the size ratios R0 and R2, similar to the processing in S150. In one example, the change ΔR is the increment in the size ratio ΔR = R2 / R0. In another example, the change ΔR is the difference ΔR = R2 - R0.

[0137] In the subsequent S460, the processor 11 controls the display 21 to display the connected space SP3, which has multiple gates 30 based on the change amount ΔR, in front of the current space that the user is viewing through the display 21. The processing in S460 can be understood as being similar to the processing in S160, which can be understood by replacing the size ratio R1 of the virtual transition source space SP1 with the size ratio R0 of the real space.

[0138] In other words, the connected space SP3 is configured such that the heights and spacing of multiple gates 30 can be set to represent the changes in the virtual body's field of view in accordance with the movement of the virtual body, as the height of the virtual body gradually changes from the user's height to the height of the second avatar AV2, and the size of the world relative to the height gradually changes from the size of the real world to the size of the second space SP2.

[0139] Figure 12 illustrates the processing from S410 to S460, but the processor 11 can execute the same processing as the processing from S170 onwards in display control processing A as the processing following S460 in display control processing B. In other words, the processor 11 can execute the processing from S410 to S460 shown in Figure 12 and the processing from S170 to S230 shown in Figures 9 and 10, and then terminate display control processing B.

[0140] In the electronic device 10 of this embodiment described above, the processor 11 controls the display 21 to display the connected space SP3, which connects the first space SP1 to the second space SP2, in the user's field of view by executing display control processing A and display control processing B.

[0141] When displaying the connected space SP3, the processor 11 places multiple gates 30 as objects within the connected space SP3 and controls the display 21 to represent the height of the virtual body's line of sight and / or movement speed in the connected space SP3 by utilizing the display of the multiple gates 30 through the spatial image.

[0142] In particular, the processor 11 adjusts the size of objects within the connected space SP3 that the user visually perceives, specifically the height and spacing of the gates 30, from the entrance P1 to the exit P2 of the connected space SP3, thereby creating an image in the spatial image that shows the relative size of the user's virtual body changing.

[0143] The processor 11 controls the display 21 so that, based on the above adjustments, the change in the virtual body's field of view until it switches to the field of view corresponding to the second avatar AV2 is represented by a spatial image displayed on the display 21.

[0144] In particular, the processor 11 controls the display 21 so that the spatial image displayed on the display 21 represents the change in the virtual body's field of view that occurs when the virtual body's height gradually changes to the height corresponding to the second avatar AV2. The processor 11 controls the display 21 so that the spatial image displayed on the display 21 represents the change in the flow of the virtual body's field of view that occurs when the virtual body's movement speed (and / or stride length) gradually changes to the movement speed (and / or stride length) corresponding to the second avatar AV2.

[0145] Therefore, according to the electronic device 10 of this embodiment, when the user's virtual body transitions from the first space SP1 to the second space SP2, the spatial image that spreads in the user's field of view through the display 21 can be discontinuously changed from the spatial image corresponding to the field of view of the first avatar AV1 to the spatial image corresponding to the field of view of the second avatar AV2, thereby reducing the cognitive load on the user associated with the spatial transition.

[0146] Furthermore, in this embodiment, when the source space SP1 is real space, the change in the user's field of view when the user's virtual body changes from the user's height to the height of the second avatar AV2 is represented using a spatial image, taking into account the user's height.

[0147] For example, by displaying an object (e.g., a gate) with a size corresponding to the difference (e.g., ratio) between the user's height and the virtual body's height, the change in the virtual body's size is represented through the display 21. Therefore, in this embodiment, the cognitive load when transitioning from real space to virtual space can also be reduced.

[0148] In addition, the electronic device 10 of this embodiment mixes the ambient sound of the first space SP1 and the ambient sound of the second space SP2 with a sound pressure or volume corresponding to the position in the connection space SP3 of the virtual body to generate ambient sound for the connection space SP3, and outputs it to the speaker 23.

[0149] Specifically, the electronic device 10 controls the speaker 23 so that as the virtual body approaches the second space SP2 in the connected space SP3, the volume of the ambient sound corresponding to the second space SP2 output by the speaker 23 increases.

[0150] According to this embodiment, since the speaker 23 is controlled to output a mixed sound corresponding to the position of the virtual body as a mixed sound of the ambient sounds of the first space SP1 and the second space SP2, it is possible to further reduce the cognitive load on the user.

[0151] [Other embodiments] This disclosure is not limited to the embodiments described above, and various forms can be adopted. For example, in the above embodiments, an avatar AV3 for a connection space different from the first avatar AV1 and the second avatar AV2 is prepared in the connection space SP3, but an avatar does not need to be prepared in the connection space SP3.

[0152] In other words, in the connected space SP3, the user's virtual body may be transparent. The avatar image 41 for the connected space does not have to be displayed near the entrance P1 of the connected space SP3. That is, at the entrance P1 of the connected space SP3, display control may be performed so that the user's virtual body transforms from the first avatar AV1 to the avatar AV3 for the connected space without overlapping with the avatar image 41. When the user's virtual body transforms into the second avatar AV2 near the exit P2 of the connected space SP3, the avatar image 41 does not have to be displayed at the transformation point as support information to facilitate understanding that the user is shedding their pre-transformation form by leaving the pre-transformation form behind during the transformation.

[0153] Furthermore, the avatar AV3 for the connected space is not limited to the dot-shaped avatar shown in Figure 5A, etc. Other avatars with an appearance that does not explicitly define their body shape, or avatars with any appearance, may be used. The dot-shaped avatar is an example of an avatar with an appearance that does not explicitly define its body shape.

[0154] The changes in the virtual body's field of view represented in the connected space SP3 do not need to be continuous, as long as they reduce the user's cognitive load. For example, changes in the field of view in connected space SP3 may be represented by discontinuous but small, stepwise changes in the field of view. In addition, changes in the virtual body's field of view do not need to be realized in accordance with changes in the virtual body's position in connected space SP3, but may be realized in accordance with the virtual body's stay time in connected space SP3.

[0155] Perception of virtual space is primarily through vision. Therefore, in the above embodiment, to reduce cognitive load, changes in the field of view accompanying avatar changes were gradually represented in the connected space SP3 by controlling the display of spatial images. However, if the concept of auditory changes is added to the avatar, auditory changes may also be gradually represented in the connected space SP3 by acoustic control.

[0156] In the embodiment described above, the stride length and movement speed in the connected space SP3 can be set to any multiplier. For example, if the electronic device 10 is configured to detect the user's walking motion, the avatar can be made to move forward in accordance with the user's walking motion. In this case, by increasing the avatar's movement speed to a predetermined multiple (N times) of the actual movement speed corresponding to the user's walking motion, it is possible to minimize movement in the real world and achieve the effect of moving a long distance in the connected space to reach the second space. The movement speed in the connected space SP3 may also be a predetermined constant speed.

[0157] The first avatar AV1 and the second avatar AV2 may have different movement speeds and the same appearance. The processing for reducing cognitive load in the connected space SP3 regarding movement speed can be applied when the movement speed of the first avatar AV1 and the movement speed of the second avatar AV2 are different, regardless of whether the external appearances of the first avatar AV1 and the second avatar AV2 are different.

[0158] In the embodiment described above, the avatar size in space was represented by adjusting the size of the avatar and the space to match the user's body size. That is, in the embodiment described above, the body size of the first avatar AV1 used by the user in the source space SP1 is adjusted to match the user's body size. On the other hand, the content size of the source space SP1 displayed to the user is adjusted so that the ratio between the body size of the first avatar AV1 and the spatial content size of the source space SP1 is kept constant.

[0159] Similarly, in the destination space SP2, the size of the second avatar AV2 used by the user is adjusted to match the user's body size. Meanwhile, the content size of the destination space SP2 displayed to the user is adjusted so that the ratio between the body size of the second avatar AV2 and the spatial content size of the destination space SP2 remains constant.

[0160] The avatar AV3 for the connected space SP3 is also the user's body size. Because the size of the spatial content changes between the source space SP1 and the destination space SP2, the size of the spatial content in the connected space is also changed to compensate for the difference between them.

[0161] If the size of the second avatar AV2 is smaller than the size of the first avatar AV1, the size of the spatial content displayed to the user will be relatively larger in the destination space SP2 than in the source space SP1, and the height of the gate 30 in the connected space SP3 will increase as it approaches the destination space SP2 from the source space SP1.

[0162] Conversely, if the size of the second avatar AV2 is larger than that of the first avatar AV1, the size of the spatial content in the connected space SP3 (the height of the gate 30) will shrink as it approaches the destination space SP2 from the source space SP1.

[0163] In other words, according to the above embodiment, the size of the content within the connection space SP3, such as the height of the gate 30, is determined by the difference between "the ratio of the size of the source space SP1 to the size of the first avatar AV1 used by the user in the source space SP1" and "the ratio of the size of the destination space SP2 to the size of the second avatar AV2 used by the user in the destination space SP2".

[0164] Therefore, the above embodiment includes the idea of ​​controlling the display to represent changes in the height of the virtual body's line of sight in the connected space and / or changes in its movement speed and / or changes in the ratio of the size of the body to the environment in the first space to the ratio of the size of the body to the environment in the second space, by utilizing the display of objects through spatial images.

[0165] The above embodiment includes the idea of ​​controlling a display to represent changes in the height of the virtual body's line of sight and / or changes in its movement speed and / or changes in the ratio of the size of the body to the environment in the first space to the ratio of the size of the body to the environment in the second space, by utilizing the display of multiple gates through spatial images.

[0166] Alternatively, the user's movements may be adjusted to match the avatar size. In this case, the movements of the first avatar AV1, which operates based on the user's movements, can be scaled up or down relative to the user's movements based on the size of the first avatar AV1, thereby giving the user the sensation that their size has become the same as the first avatar AV1.

[0167] When moving from the source space SP1 through the connection space SP3 to the destination space SP2, the movement of the avatar AV3 for the connection space is gradually enlarged or reduced relative to the user's movement, based on the body size of the second avatar AV2 used by the user in the destination space SP2. This gives the user the sensation that their size has become as if it were the size of the second avatar AV2 when they reach the exit of the connection space SP3 on the destination space SP2 side.

[0168] In addition, the distance of the connecting space SP3 that links the source space SP1 and the destination space SP2, that is, the distance from the connection point with the source space SP1 in the connecting space SP3 to the connection point with the destination space SP2 in the connecting space SP3, may be designed according to the environmental difference between the source space SP1 and the destination space SP2. By designing a longer distance for larger environmental differences, the cognitive load caused by discontinuous spatial transitions can be reduced. In other words, by setting a longer viewpoint movement distance in the connecting space, the cognitive load of the environmental change from the source space SP1 to the destination space SP2 can be reduced. In a simple example, the environmental difference can be the amount of change ΔR mentioned above.

[0169] The movement speed of the virtual body (third avatar AV3) in the connected space SP3 may differ from the movement speed in the source space SP1 and / or destination space SP2. Similar to the distance traveled in the connected space, the movement speed in the connected space may be designed to match the environmental differences between the source space SP1 and the destination space SP2 as the virtual body moves through the connected space SP3.

[0170] The above example includes the idea of ​​controlling the display so that, as the virtual body moves through the connected space, the changes in the virtual body's field of view when moving at a distance and / or speed in the connected space, designed to match the environmental differences between the first and second spaces, are represented by spatial images displayed on the display.

[0171] The function of one component in the above embodiment may be distributed among multiple components. The functions of multiple components may be integrated into one component. Some parts of the configuration of the above embodiment may be omitted. At least some parts of the configuration of the above embodiment may be added to or replaced by the configuration of other above embodiments. Any aspect of the technical concept specified by the wording of the claims constitutes an embodiment of the present disclosure.

[0172] [Technical Concept Disclosed in This Specified Specification] This specification can be understood to disclose the following technical concepts: [Item 1] A control system for controlling the display that is visible to the user, A first display control unit is configured to control the display to show a connected space, which is a virtual space connecting the first space to the second space, in order to change the space visible in the user's field of view from the first space to the second space, A second display control unit is configured to control the display to show a spatial image as seen from the virtual body, corresponding to the virtual body's position in the connected space and the virtual body's line of sight, in accordance with the user's actions, for moving the virtual body corresponding to the user through the connected space. Equipped with, The second display control unit controls the display so that, as the virtual body moves through the connected space toward the second space, the change in the virtual body's field of view from the field of view corresponding to the first form in the first space to the field of view corresponding to the second form in the second space is represented by the spatial image displayed on the display. Control system. [Item 2] The control system described in item 1, The virtual body has a third form for the connection space that is different from the first form and the second form in at least a portion of the connection space, and in the process of moving from the first space through the connection space to the second space, it changes from the first form to the third form, and further changes from the third form to the second form. The second display control unit controls the display so as the virtual body having the third form moves through the connection space toward the second space, the field of view of the virtual body having the third form gradually changes from the field of view corresponding to the first form in the first space to the field of view corresponding to the second form in the second space. Control system. [Item 3] The control system described in item 1, Between the first appearance and the second appearance, there is a difference in height and / or body shape. The second display control unit controls the display to represent the change in the virtual body's field of view, which occurs when the virtual body's height and / or body shape gradually changes from a first height and / or body shape corresponding to the first appearance to a second height and / or body shape corresponding to the second appearance, as the virtual body moves through the connected space, using the spatial image. Control system. [Item 4] The control system described in item 2, Between the first appearance and the second appearance, there is a difference in height and / or body shape. The second display control unit controls the display to represent in the spatial image the change in the virtual body's field of view that occurs when the virtual body having the third form moves through the connected space and its height and / or body shape gradually changes from the first height and / or body shape corresponding to the first form to the second height and / or body shape corresponding to the second form. Control system. [Item 5] A control system described in any one of items 1 to 4, The size ratio of the virtual body to the space differs between the first space and the second space. The second display control unit controls the display so as the virtual body moves through the connected space, the display represents the change in the virtual body's field of view that occurs when the virtual body gradually changes its viewpoint from the viewpoint when it views the first space in its first form to the viewpoint when it views the second space in its second form, using the spatial image. Control system. [Item 6] A control system described in any one of items 1 to 5, The second display control unit controls the display so as the virtual body moves through the connected space, it does so that the changes in the virtual body's field of view that occur when the virtual body moves at a speed that is a predetermined multiple of the actual user's movement speed, or at a preset movement speed, are represented in the spatial image. Control system. [Item 7] A control system as described in item 2 or item 4, The third form is the form of the virtual body having an appearance in which the body shape is not explicitly defined. Control system. [Item 8] A control system described in any one of items 1 to 7, A control system in which both the first space and the second space are virtual spaces, or in which one of the first space and the second space is a virtual space and the other is a real space. [Item 9] A control system described in any one of items 1 to 8, The second display control unit controls the display to represent the height of the virtual body's line of sight and / or movement speed in the connected space, utilizing the display of objects through the spatial image. Control system. [Item 10] A control system described in any one of items 1 to 8, The aforementioned connection space is a virtual space in which a plurality of gates are arranged at intervals from the first space toward the second space, The second display control unit controls the display to represent the height of the virtual body's line of sight and / or movement speed in the connected space, utilizing the display of the plurality of gates through the spatial image. Control system. [Item 11] A control system described in any one of items 1 to 10, An acquisition unit configured to acquire physical information that can identify the user's height. Furthermore, The second display control unit controls the display to represent the size of objects in the spatial image based on the difference between the user's height, identified from the physical information, and the height of the virtual body. Control system. [Item 12] A control system according to any one of items 1 to 11, A sound control unit is configured to control a speaker and cause the speaker to output ambient sounds corresponding to the position of the virtual body. The sound control unit further comprises the following: As the virtual body approaches the second space in the connection space, the sound control unit controls the speaker such that the volume of the ambient sound corresponding to the second space output by the speaker changes. Control system. [Item 13] A control system according to any one of items 1 to 12, The second display control unit places an avatar image corresponding to the second appearance in the boundary area between the connection space and the second space, and when the virtual body overlaps with the avatar image, it assigns the appearance corresponding to the avatar image to the virtual body, thereby setting the virtual body to the second appearance having the appearance corresponding to the avatar image. Control system. [Item 14] A control system according to any one of items 2, 4, and 7, The second display control unit, at a specific point in the connection space, replaces the virtual body having a first appearance that entered the connection space from the first space with the third appearance, and places an avatar image corresponding to the first appearance around the specific point. Control system. [Item 15] A control system described in any one of items 1 to 14, The spatial image displayed on the aforementioned display is a first-person perspective spatial image corresponding to the gaze of the virtual body that corresponds to the user's gaze. Control system. [Item 16] A control method for controlling a display that extends into the user's field of view, The first process involves controlling the display to show a connection space, which is a virtual space connecting the first space to the second space, in order to change the space visible to the user from the first space to the second space. A second process involves controlling the display to show a spatial image as seen from the virtual body, corresponding to the virtual body's position in the connected space and the virtual body's line of sight, in accordance with the user's actions, in order to move the virtual body corresponding to the user through the connected space. Includes, The second process includes controlling the display so that, as the virtual body moves through the connected space toward the second space, the change in the virtual body's field of view from the field of view corresponding to the first form in the first space to the field of view corresponding to the second form in the second space is represented by the spatial image displayed on the display. Control method. [Item 17] The control method described in item 16, The virtual body has a third form for the connection space that is different from the first form and the second form in at least a portion of the connection space, and in the process of moving from the first space through the connection space to the second space, it changes from the first form to the third form, and further changes from the third form to the second form. The second process includes controlling the display such that, as the virtual body having the third form moves through the connected space toward the second space, the field of view of the virtual body having the third form changes from the field of view corresponding to the first form in the first space to the field of view corresponding to the second form in the second space. Control method. [Item 18] The control method described in item 16, Between the first appearance and the second appearance, there is a difference in height and / or body shape. The second process includes controlling the display to represent in the spatial image the changes in the virtual body's field of view that occur when the virtual body moves through the connected space and its height and / or body shape gradually changes from a first height and / or body shape corresponding to the first form to a second height and / or body shape corresponding to the second form. Control method. [Item 19] The control method described in item 17, Between the first appearance and the second appearance, there is a difference in height and / or body shape. The second process includes controlling the display to represent in the spatial image the changes in the virtual body's field of view that occur when the virtual body having the third form moves through the connected space and its height and / or body shape gradually changes from a first height and / or body shape corresponding to the first form to a second height and / or body shape corresponding to the second form. Control method. [Item 20] A control method according to any one of items 16 to 19, The size ratio of the virtual body to the space differs between the first space and the second space. The second process includes controlling the display to represent in the spatial image the change in the virtual body's field of view that occurs when the virtual body moves through the connected space, gradually changing from the viewpoint when the virtual body views the first space in its first form to the viewpoint when the virtual body views the second space in its second form. Control method. [Item 21] A control method described in any one of items 16 to 20, The second process includes controlling the display so that the changes in the virtual body's field of view that occur when the virtual body moves in the connected space at a speed that is a predetermined multiple of the actual user's movement speed, or at a preset movement speed, are represented in the spatial image. Control method. [Item 22] A control method according to item 17 or item 19, The third form is the form of the virtual body having an appearance in which the body shape is not explicitly defined. Control method. [Item 23] A control method described in any one of items 16 to 22, A control method in which both the first space and the second space are virtual spaces, or in which one of the first space and the second space is a virtual space and the other is a real space. [Item 24] A control method described in any one of items 16 to 23, The second process includes controlling the display to represent the height and / or speed of the virtual body's line of sight in the connected space, using the display of objects through the spatial image. Control method. [Item 25] A control method described in any one of items 16 to 23, The aforementioned connection space is a virtual space in which a plurality of gates are arranged at intervals from the first space toward the second space, The second process includes controlling the display to represent the height and / or speed of the virtual body's line of sight in the connected space, using the display of the plurality of gates through the spatial image. Control method. [Item 26] A control method described in any one of items 16 to 25, To obtain physical information that can identify the user's height. It further includes, The second process includes controlling the display to represent the size of an object in the spatial image based on the difference between the user's height, identified from the physical information, and the height of the virtual body. Control method. [Item 27] A control method described in any one of items 16 to 26, Controlling a speaker to output ambient sounds corresponding to the position of the virtual body, wherein the speaker is controlled such that the volume of ambient sounds corresponding to the second space output by the speaker changes as the virtual body approaches the second space in the connection space. A control method that further includes the following. [Item 28] A control method described in any one of items 16 to 27, The second process includes placing an avatar image corresponding to the second form in the boundary area between the connection space and the second space, and when the virtual body overlaps with the avatar image, assigning the virtual body an appearance corresponding to the avatar image, thereby setting the virtual body to the second form having an appearance corresponding to the avatar image. Control method. [Item 29] A control method according to any one of items 17, 19, and 22, The second process includes, at a specific point in the connection space, replacing the virtual body having a first form that entered the connection space from the first space with the third form, and placing an avatar image corresponding to the first form around the specific point. Control method. [Item 30] A control method described in any one of items 16 to 29, The spatial image displayed on the aforementioned display is a first-person perspective spatial image corresponding to the gaze of the virtual body that corresponds to the user's gaze. Control method. [Item 31] A computer program for causing a computer to perform the first process and the second process in the control method described in any one of items 16 to 30. [Item 32] A control system described in any one of items 1 to 8, The second display control unit controls the display to represent changes in the height of the virtual body's line of sight and / or changes in its movement speed in the connected space and / or changes in the ratio of the size of the virtual body to the space in the first space to the ratio of the size of the virtual body to the space in the second space, using the display of objects through the spatial image. Control system. [Item 33] A control system described in any one of items 1 to 8, The aforementioned connection space is a virtual space in which a plurality of gates are arranged at intervals from the first space toward the second space, The second display control unit controls the display to represent changes in the height of the virtual body's line of sight and / or changes in its movement speed and / or changes in the size ratio of the virtual body to the space in the first space to the size ratio of the virtual body to the space in the second space, using the display of the plurality of gates through the spatial image. Control system. [Item 34] A control system for controlling the display that is visible to the user, A first display control unit is configured to control the display to show a connection space, which is a virtual space connecting the first space to the second space, in order to change the space visible in the user's field of view from the first space to the second space, A second display control unit is configured to control the display to show a spatial image as seen from the virtual body, corresponding to the virtual body's position in the connected space and the virtual body's line of sight, in accordance with the user's actions, for moving the virtual body corresponding to the user through the connected space. A control system equipped with the following features. [Item 35] A control system as described in item 34, The first display control unit is a control system configured to set the distance and / or speed of movement of the virtual body from the connection point of the connected space with the first space to the connection point of the second space, according to the environmental difference between the first space and the second space as perceived by the user through the display. [Item 36] A control system as described in item 35, The first display control unit is a control system that sets the travel distance to be longer the greater the environmental difference. [Item 37] A control system as described in item 35 or item 36, The first display control unit is a control system that sets the movement speed higher the greater the environmental difference. [Explanation of Symbols]

[0173] 10...Electronic device, 11...Processor, 13...Memory, 15...Storage, 17...Communication interface, 21...Display, 23...Speaker, 25...Camera, 27...Sensor, 30...Gate, 41, 43, 45...Avatar image, AV1...First avatar, AV2...Second avatar, AV3...Third avatar, P1...Entrance, P2...Exit, SP1...Source space (first space), SP2...Destination space (second space), SP3...Connection space.

Claims

1. A control system for controlling the display that is visible to the user, A first display control unit is configured to control the display to show a connected space, which is a virtual space connecting the first space to the second space, in order to change the space visible in the user's field of view from the first space to the second space, A second display control unit is configured to control the display to show a spatial image as seen from the virtual body, corresponding to the virtual body's position in the connected space and the virtual body's line of sight, in accordance with the user's actions, for moving the virtual body corresponding to the user through the connected space. Equipped with, The second display control unit controls the display so that, as the virtual body moves through the connected space toward the second space, the change in the virtual body's field of view from the field of view corresponding to the first form in the first space to the field of view corresponding to the second form in the second space is represented by the spatial image displayed on the display. Control system.

2. A control system according to claim 1, The virtual body has a third form for the connection space that is different from the first form and the second form in at least a portion of the connection space, and in the process of moving from the first space through the connection space to the second space, it changes from the first form to the third form, and further changes from the third form to the second form. The second display control unit controls the display so as the virtual body having the third form moves through the connection space toward the second space, the field of view of the virtual body having the third form gradually changes from the field of view corresponding to the first form in the first space to the field of view corresponding to the second form in the second space. Control system.

3. A control system according to claim 1, Between the first appearance and the second appearance, there is a difference in height and / or body shape. The second display control unit controls the display to represent the change in the virtual body's field of view, which occurs when the virtual body's height and / or body shape gradually changes from a first height and / or body shape corresponding to the first appearance to a second height and / or body shape corresponding to the second appearance, in the spatial image as the virtual body moves through the connected space. Control system.

4. A control system according to claim 2, Between the first appearance and the second appearance, there is a difference in height and / or body shape. The second display control unit controls the display to represent in the spatial image the change in the virtual body's field of view that occurs when the virtual body having the third form moves through the connected space and its height and / or body shape gradually changes from the first height and / or body shape corresponding to the first form to the second height and / or body shape corresponding to the second form. Control system.

5. A control system according to claim 1, The size ratio of the virtual body to the space differs between the first space and the second space. The second display control unit controls the display so as the virtual body moves through the connected space, the display represents the change in the virtual body's field of view that occurs when the virtual body gradually changes its viewpoint from the viewpoint when it views the first space in its first form to the viewpoint when it views the second space in its second form, using the spatial image. Control system.

6. A control system according to claim 1, The second display control unit controls the display so as the virtual body moves through the connected space, it does so that the changes in the virtual body's field of view that occur when the virtual body moves at a speed that is a predetermined multiple of the actual user's movement speed, or at a preset movement speed, are represented in the spatial image. Control system.

7. A control system according to claim 2, The third form is the form of the virtual body having an appearance in which the body shape is not explicitly defined. Control system.

8. A control system according to any one of claims 1 to 7, A control system in which both the first space and the second space are virtual spaces, or in which one of the first space and the second space is a virtual space and the other is a real space.

9. A control system according to any one of claims 1 to 7, The second display control unit controls the display to represent the height of the virtual body's line of sight and / or movement speed in the connected space, utilizing the display of objects through the spatial image. Control system.

10. A control system according to any one of claims 1 to 7, The aforementioned connection space is a virtual space in which a plurality of gates are arranged at intervals from the first space toward the second space, The second display control unit controls the display to represent the height of the virtual body's line of sight and / or movement speed in the connected space, utilizing the display of the plurality of gates through the spatial image. Control system.

11. A control system according to any one of claims 1 to 7, An acquisition unit configured to acquire physical information that can identify the user's height. Furthermore, The second display control unit controls the display to represent the size of objects in the spatial image based on the difference between the user's height, identified from the physical information, and the height of the virtual body. Control system.

12. A control system according to any one of claims 1 to 7, A sound control unit is configured to control a speaker and cause the speaker to output ambient sounds corresponding to the position of the virtual body. The sound control unit further comprises the following: As the virtual body approaches the second space in the connection space, the sound control unit controls the speaker such that the volume of the ambient sound corresponding to the second space output by the speaker changes. Control system.

13. A control system according to any one of claims 1 to 7, The second display control unit places an avatar image corresponding to the second appearance in the boundary area between the connection space and the second space, and when the virtual body overlaps with the avatar image, it assigns the appearance corresponding to the avatar image to the virtual body, thereby setting the virtual body to the second appearance having the appearance corresponding to the avatar image. Control system.

14. A control system according to claim 2, The second display control unit, at a specific point in the connection space, replaces the virtual body having a first appearance that entered the connection space from the first space with the third appearance, and places an avatar image corresponding to the first appearance around the specific point. Control system.

15. A control system according to any one of claims 1 to 7, The spatial image displayed on the aforementioned display is a first-person perspective spatial image corresponding to the gaze of the virtual body that corresponds to the user's gaze. Control system.

16. A control method for controlling a display that extends into the user's field of view, The first process involves controlling the display to show a connection space, which is a virtual space connecting the first space to the second space, in order to change the space visible in the user's field of view from the first space to the second space. A second process involves controlling the display to show a spatial image as seen from the virtual body, corresponding to the virtual body's position in the connected space and the virtual body's line of sight, in accordance with the user's actions, in order to move the virtual body corresponding to the user through the connected space. Includes, The second process includes controlling the display so that, as the virtual body moves through the connected space toward the second space, the change in the virtual body's field of view from the field of view corresponding to the first form in the first space to the field of view corresponding to the second form in the second space is represented by the spatial image displayed on the display. Control method.

17. A control method according to claim 16, The virtual body has a third form for the connection space that is different from the first form and the second form in at least a portion of the connection space, and in the process of moving from the first space through the connection space to the second space, it changes from the first form to the third form, and further changes from the third form to the second form. The second process includes controlling the display such that, as the virtual body having the third form moves through the connected space toward the second space, the field of view of the virtual body having the third form changes from the field of view corresponding to the first form in the first space to the field of view corresponding to the second form in the second space. Control method.

18. A control method according to claim 16, Between the first appearance and the second appearance, there is a difference in height and / or body shape. The second process includes controlling the display to represent in the spatial image the changes in the virtual body's field of view that occur when the virtual body moves through the connected space and its height and / or body shape gradually changes from a first height and / or body shape corresponding to the first appearance to a second height and / or body shape corresponding to the second appearance. Control method.

19. A control method according to claim 17, Between the first appearance and the second appearance, there is a difference in height and / or body shape. The second process includes controlling the display to represent in the spatial image the changes in the virtual body's field of view that occur when the virtual body having the third form moves through the connected space and its height and / or body shape gradually changes from a first height and / or body shape corresponding to the first form to a second height and / or body shape corresponding to the second form. Control method.

20. A control method according to claim 16, The size ratio of the virtual body to the space differs between the first space and the second space. The second process includes controlling the display to represent in the spatial image the change in the virtual body's field of view that occurs when the virtual body moves through the connected space, gradually changing from the viewpoint when the virtual body views the first space in its first form to the viewpoint when the virtual body views the second space in its second form. Control method.

21. A control method according to claim 16, The second process includes controlling the display so that the changes in the virtual body's field of view that occur when the virtual body moves in the connected space at a speed that is a predetermined multiple of the actual user's movement speed, or at a preset movement speed, are represented in the spatial image. Control method.

22. A control method according to claim 17, The third form is the form of the virtual body having an appearance in which the body shape is not explicitly defined. Control method.

23. A control method according to claim 16, A control method in which both the first space and the second space are virtual spaces, or in which one of the first space and the second space is a virtual space and the other is a real space.

24. A control method according to claim 16, The second process includes controlling the display to represent the height of the virtual body's line of sight and / or its movement speed in the connected space, using the display of objects through the spatial image. Control method.

25. A control method according to claim 16, The aforementioned connection space is a virtual space in which a plurality of gates are arranged at intervals from the first space toward the second space, The second process includes controlling the display to represent the height of the virtual body's line of sight and / or movement speed in the connected space, using the display of the plurality of gates through the spatial image. Control method.

26. A control method according to claim 16, To obtain physical information that can identify the user's height. It further includes, The second process includes controlling the display to represent the size of an object in the spatial image based on the difference between the user's height, identified from the physical information, and the height of the virtual body. Control method.

27. A control method according to claim 16, Controlling a speaker to output ambient sounds corresponding to the position of the virtual body, wherein the speaker is controlled such that the volume of the ambient sounds corresponding to the second space output by the speaker changes as the virtual body approaches the second space in the connection space. A control method that further includes the following.

28. A control method according to claim 16, The second process includes placing an avatar image corresponding to the second form in the boundary area between the connection space and the second space, and when the virtual body overlaps with the avatar image, assigning the virtual body an appearance corresponding to the avatar image, thereby setting the virtual body to the second form having an appearance corresponding to the avatar image. Control method.

29. A control method according to claim 17, The second process includes, at a specific point in the connection space, replacing the virtual body having a first form that entered the connection space from the first space with the third form, and placing an avatar image corresponding to the first form around the specific point. Control method.

30. A control method according to claim 16, The spatial image displayed on the aforementioned display is a first-person perspective spatial image corresponding to the gaze of the virtual body that corresponds to the user's gaze. Control method.

31. A computer program for causing a computer to perform the first process and the second process in the control method according to any one of claims 16 to 30.

Images