Information processor, control method for information processor, and control program for information processor

Abstract

To provide an information processor that enables a user to further easily understand a point and a trigger at which an event occurs in a virtual space.SOLUTION: An information processor includes: a display unit that displays a virtual space image; a display control unit that displays a virtual space on the display unit according to the orientation of the display unit; and a determination unit that determines whether a predetermined first position on the display unit overlaps with a predetermined second position in the virtual space. The display control unit causes the second position on the virtual space image to be displayed in an identifiable manner, and causes the display unit to display a predetermined image corresponding to the second position in response to the determination determined by the determination unit that the first position and the second position overlap.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an information processing device, a control method for an information processing device, and a control program for an information processing device. [Background technology]

[0002] Conventionally, in entertainment facilities, there is a video providing system that provides a video according to a user's location to a display device (head mounted display (HMD)) worn on the user's head (for example, Patent Document 1). In recent years, in addition to VR (Virtual Reality) as in Patent Document 1, XR (Cross Reality) technology including AR (Augmented Reality) and MR (Mixed Reality) has been spreading in various fields. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-121132 A Summary of the Invention [Problem to be solved by the invention]

[0004] In order to provide users with experiences using XR technology, there was a need to make it easier for users to understand the points and triggers at which events occur in the virtual space. [Means for solving the problem]

[0005] An information processing device according to one embodiment of the present invention includes a display unit that displays a virtual space image, a display control unit that causes the display unit to display a virtual space corresponding to the orientation of the display unit, and a determination unit that determines whether a predetermined first position on the display unit and a predetermined second position on the virtual space image overlap. The display control unit displays the second position on the virtual space image in an identifiable manner, and causes the display unit to display a predetermined image corresponding to the second position in response to the determination by the determination unit that the first position and the second position overlap.

[0006] An information processing device according to an embodiment of the present invention may further include a generation unit that generates a virtual space image, and the display control unit may cause the display unit to display the virtual space image generated by the generation unit.

[0007] An information processing device according to an embodiment of the present invention may further include a communication unit that receives a virtual space image, and the display control unit may cause the virtual space image received by the communication unit to be displayed on the display unit.

[0008] In an information processing device according to one embodiment of the present invention, the display control unit may cause the display unit to display a time gauge for a time period set according to the second position, the gauge changing according to the time period during which the first position and the second position overlap.

[0009] In an information processing device according to one embodiment of the present invention, a display control unit may display a predetermined image in response to an overlap between a first location and a second location lasting for a period of time set in accordance with the second location.

[0010] In an information processing device according to an embodiment of the present invention, the first position may be the center of the display unit.

[0011] In the information processing device according to one embodiment of the present invention, the display control unit may change the display mode of the center of the display unit in the virtual space image displayed on the display unit.

[0012] A control method for an information processing device according to one embodiment of the present invention includes the steps of causing the display unit to display a virtual space image corresponding to an orientation of the display unit, and determining whether a predetermined first position on the display unit and a predetermined second position on the virtual space image overlap, wherein the displaying step displays the second position on the virtual space image in an identifiable manner, and causes the display unit to display a predetermined image corresponding to the second position in response to determining that the first position and the second position overlap.

[0013] A control program of an information processing device according to one embodiment of the present invention causes the information processing device to realize a function of displaying a virtual space image on the display unit according to the orientation of the display unit, and a function of determining whether a predetermined first position on the display unit overlaps with a predetermined second position on the virtual space image, the display function displaying the second position on the virtual space image in an identifiable manner, and displaying a predetermined image corresponding to the second position on the display unit according to whether it is determined that the first position and the second position overlap. Effect of the Invention

[0014] According to one embodiment of the present invention, it is possible to provide an information processing device or the like that makes it easier for a user to understand points and triggers at which events occur in a virtual space. [Brief description of the drawings]

[0015] [Figure 1] FIG. 1 is a schematic diagram of a configuration of a video providing system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is an example of a functional block diagram of an information processing device according to an embodiment of the present invention. [Diagram 3] FIG. 3 is a flowchart showing an example of the operation of the information processing device according to one embodiment of the present invention. [Figure 4] 4(a) to 4(c) are diagrams showing an example of transition of the display unit of an information processing device according to one embodiment of the present invention. [Diagram 5] FIG. 5 is an example of an event information table stored in the video providing system according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An information processing apparatus according to an embodiment of the present invention will be described below with reference to the accompanying drawings. However, the present invention is not limited to the embodiment.

[0017] <System configuration> 1 is a diagram showing an example of the configuration of a video providing system according to an embodiment of the present invention. The video providing system 500 is a system in which, by XR technology, an image is provided to an information processing device 100 (100A, 100B, 100C) held by a user according to the position and orientation of the information processing device 100. The information processing device 100 is a display device having a communication function that can be held and moved by a user, and may be, for example, a smartphone, a tablet, a notebook computer, a wearable terminal (glasses-type device, watch-type device, smart glasses, etc.), an HMD, etc.

[0018] Hereinafter, an embodiment of the present invention will be described as an image providing system using AR technology, with a case in which a smartphone or tablet is used as the information processing device 100. Here, AR refers to a technology that extends the real world by overlaying computer-generated images or pictures on the real world in front of the user, and the user can view an image in which virtual world content is displayed on an image of the real world via the camera of the information processing device 100. However, the present invention is not limited to this, and VR and MR may also be used.

[0019] The video providing system 500 includes an information processing device 100, a video providing server 200, and a database server 400. The video providing server 200 is connected to the information processing device 100 via a network 300, and transmits virtual video (content) to the information processing device 100 as necessary. The content provided by the video providing server 200 may be, but is not limited to, moving images, text messages, music, and the like.

[0020] The information processing device 100 is a communication terminal of a user who uses a video providing service realized by a video providing system. An application for using the video providing service (hereinafter, also referred to as an "AR application") is installed in the information processing device 100, and various information may be transmitted and received between the information processing device 100 and the video providing server 200. Although only three information processing devices 100 are shown in FIG. 1, the number of information processing devices 100 may be the same as the number of users who use the video providing service.

[0021] The network 300 includes a wireless network and a wired network, and is a mobile communication system such as wireless LAN (WLAN), LTE (long term evolution), fourth generation communication (4G), fifth generation communication (5G), etc. Note that the network 300 is not limited to these examples, and may be, for example, a short-range wireless communication (Bluetooth (registered trademark)), an optical fiber line, etc.

[0022] The database server 400 stores the virtual video provided by the video providing system 500 and information on users who use the video providing system 500 .

[0023] <Information processing device> Next, an information processing device 100 according to an embodiment of the present invention will be described with reference to Fig. 2. The information processing device 100 includes a control unit 110, a communication unit 120, a display unit 130, an input / output unit 140, various sensors 150, and a storage unit 170.

[0024] The communication unit 120 is implemented as hardware such as a network adapter, communication software, or a combination of these, and transmits and receives various data to and from the video providing server 200 via the network 300. For example, the communication unit 120 receives a virtual space image from the video providing server 200.

[0025] The display unit 130 is a display that displays moving images transmitted from the control unit 110. The display unit 130 also displays images of the real world acquired via a camera provided in the information processing device 100. Note that, in one embodiment of the present invention, the display unit 130 may be a touch panel, a touch display, or the like that is integrated with the input / output unit 140 described later.

[0026] The input / output unit 140 includes an input device that inputs various operations to the information processing device 100, and an output device that outputs processing results processed by the information processing device 100. The input device includes, for example, a touch panel, a touch display, a camera, and a microphone, and the output device includes, for example, a display, a touch panel, a speaker, etc.

[0027] The sensor 150 includes an acceleration sensor that detects the inclination of the information processing device 100 and a gyro sensor that detects the direction (orientation) of the information processing device 100. The sensor 150 may also include a GPS (Global Positioning System) sensor that acquires position information of the information processing device 100 and a geomagnetic sensor.

[0028] The storage unit 170 stores various programs and various data required for the operation of the information processing device 100. That is, the storage unit 170 stores programs related to applications of a video providing service. The storage unit 170 includes, for example, a flash memory and a memory (such as a RAM (Random Access Memory) or a ROM (Read Only Memory)) that provides a working area for the control unit 110.

[0029] The control unit 110 is typically a processor, and is realized by a CPU, a GPU, etc. The control unit 110 realizes the functions of an acquisition unit 111, a display control unit 112, a determination unit 113, and a generation unit 114 by reading a program stored in the storage unit 170 and executing a code or instruction included in the read program.

[0030] The acquisition unit 111 acquires information about the position and orientation of the device itself from various sensors 150. For example, the acquisition unit 111 acquires latitude and longitude as position information of the device itself from a GPS sensor. The position information may include altitude. The acquisition unit 111 also acquires information about the inclination of the device itself from a gyro sensor and an acceleration sensor.

[0031] The display control unit 112 causes the display unit 130 to display a virtual space corresponding to the orientation of the device itself, based on the information acquired by the acquisition unit 111. The display control unit 112 also causes a predetermined second position in the virtual space, which is a point at which an event occurs, to be displayed in an identifiable manner. The determination unit 113 performs various determination processes. For example, the determination unit 113 determines whether or not a predetermined first position on the display unit 130 overlaps with a predetermined second position in the virtual space (details will be described later).

[0032] In one embodiment of the present invention, information on the virtual space image may be acquired in advance from the database server 400 according to the area in which the information processing device 100 is located, and stored in the storage unit 170. Then, the generation unit 114 of the information processing device 100 may generate a virtual space image according to the position and orientation of the information processing device 100 based on the information stored in the storage unit 170. Alternatively, the generation unit 114 may process an image captured by a camera to generate a virtual space image. The video displayed on the display unit 130 by the display control unit 112 may be a composite image of an image captured by the camera and an image generated by the generation unit 114.

[0033] Alternatively, the video providing server 200 may generate a virtual space image according to the position information of the information processing device 100 and transmit the virtual space image to the information processing device 100 .

[0034] <Examples of operation and display screen> Next, functions of the information processing device 100 according to an embodiment of the present invention will be described based on a flowchart and an example of a display screen of the information processing device 100. Fig. 3 is a flowchart showing the operation of the information processing device 100, and Figs. 4(a) to (c) are examples of display screens of the information processing device 100. Note that the drawings are merely examples, and the present invention is not limited to these.

[0035] First, the acquisition unit 111 acquires information about the position and inclination of the device from various sensors 150. As a result, the display control unit 112 causes the display unit to display a virtual space image according to the orientation of the display unit 130 (step S11). Here, FIG. 4(a) is an example of a display screen of the display unit 130 on which the virtual space image is displayed. In the screen D10, an object 12 is displayed as a virtual space image on an image of the real world (in the example of the figure, trees, buildings including a passage extending in the depth direction of the screen, etc.) captured by a camera. Note that the image of the virtual space may be displayed so as to cover the image of the real world. The object 12 indicates a point at which an event occurs when the user takes a predetermined action, and is set according to the position of the information processing device 100, that is, the position in the real world. Note that the object 12 may or may not be displayed according to the position of the information processing device 100. In other words, when the information processing device 100 is within a predetermined distance from the object 12 and is within the angle of view of the information processing device 100, the object 12 may be displayed. This allows the video provider to display the virtual image in a manner that the video provider wishes to show to the user. Information regarding the points is stored in the database server 400, and the communication unit 120 may receive in advance information regarding the points set in the area in which the information processing device 100 is located.

[0036] FIG. 5 shows an example of a data table related to points at which events occur, stored in the database server 400. Table TB10 may store an identifier (event ID (IDentifier)) that uniquely identifies an event occurring at each point, the position coordinates of the point, content provided in response to the occurrence of the event, and the startup time until the event occurs. For example, an event identified by the event ID "action_001" displays "video A" as content when the user takes a predetermined action at the point at coordinates (x1, y1, z1). The predetermined action and the startup time until the event occurs will be described later. Also, the figure is an example, and the stored information is not limited to this. For example, the coordinates of the point may be spherical coordinates instead of orthogonal linear coordinates.

[0037] When the user recognizes that the object 12 is a trigger that generates an event, the user moves the information processing device 100 so as to overlap the marker 11 with the object 12. The determination unit 113 determines whether or not a predetermined first position on the display unit 130 and a predetermined second position in the virtual space overlap (step S12 in FIG. 3). The predetermined first position on the display unit 130 may be, for example, the center point of the display unit 130. The predetermined second position in the virtual space may refer to the above-mentioned point. That is, the determination unit 113 determines whether or not the center point in the camera's shooting range is located on the object 12, based on the position information and tilt of the information processing device 100.

[0038] As shown in screen D10 in FIG. 4(a), the display control unit 112 may change the display mode of the center of the display unit 130 in the image displayed on the display unit 130. In the example of screen D10, the display control unit 112 causes the display unit 130 to display a marker 11 indicating the center of the display unit 130. The shape of the marker 11 is not limited to a circle, and any shape is acceptable as long as it allows the user to recognize that it is the center. For example, the display control unit 112 may blink, color, or make transparent the vicinity of the center of the display unit 130. Alternatively, the center of the display unit 130 may be indicated by a beam-like line extending toward the user.

[0039] If the first position and the second position do not overlap (NO in step S12), the process returns to step S11, and the display of the virtual space image according to the orientation of the display unit 130 continues.

[0040] When it is determined that the first position (the center of the display unit 130) and the second position (point) overlap (YES in step S12 in FIG. 3), the display control unit 112 may display on the display unit 130 a time gauge for a time period set according to the second position, the gauge changing according to the time period during which the first position and the second position overlap. FIG. 4(b) shows an example of a display screen on which a time gauge is displayed. The time gauge 21 is displayed at the center of the screen D20. As described above, the time period until an event occurs is set for each point in the table TB10 in FIG. 5, and the display control unit 112 displays the time gauge 21 so that it expires within the set time period. For example, the event identified by the event ID "action_001" may display a time gauge 21 for "2 seconds". The display form of the time gauge 21 is not limited to that shown in the figure, and may be in the shape of a time bar, and any form is acceptable as long as it allows the user to recognize the time until activation.

[0041] When the overlap between the first position and the second position has lasted for a period of time set according to the second position, the display control unit 112 may cause the display unit 130 to display a predetermined image according to the second position (step S13 in FIG. 3). FIG. 4(c) shows an example of a display screen on which a predetermined image is displayed. The example of screen D30 is an event identified by, for example, event ID "action_001", and "video A" in which birds 31, 32 as virtual images fly out from the center (i.e., the position of object 12) is displayed by the display control unit 112.

[0042] Thereafter, if an operation to end the AR application from the user is detected (YES in step S14), the process ends. If an end operation is not detected, the process returns to step S11 and the display of the virtual space continues.

[0043] Thus, according to one embodiment of the present invention, an event corresponding to the second position occurs by overlapping the first position on the display unit 130 of the information processing device 100 with the second position in the virtual space. At this time, since the second position, which is the point where the event occurs, is displayed in an identifiable manner, it is possible to realize a video providing service that is easier for the user to understand and allows the user to become immersed in the video. In addition, there is also an advantage that the event in the virtual space can be enjoyed by only inputting operations to the screen, and no separate controller is required.

[0044] According to one embodiment of the present invention, information regarding the time until an event occurs is provided to a user in the form of a time gauge, allowing the user to enjoy the video providing service without worrying about when the event will occur.

[0045] Although the present invention has been described based on the drawings and examples, it should be noted that those skilled in the art can easily make various modifications and corrections based on the present disclosure. Therefore, it should be noted that these modifications and corrections are included in the scope of the present invention. For example, the functions included in each component, each step, etc. can be rearranged so as not to cause logical contradictions, and multiple components, steps, etc. can be combined into one or divided. In addition, the configurations shown in the above embodiments may be appropriately combined. For example, each function described as being included in the information processing device 100 may be executed by the video providing server 200.

[0046] For example, in the above description, the first position is described as the center of the display unit 130, but the present invention is not limited to this, and may be set at a location other than the center of the screen. For example, the first position may be the top edge of the screen, or may change over time or depending on the location. Alternatively, the user may be able to set the first position.

[0047] In the above description, the time gauge is displayed when the first position and the second position overlap, but the time gauge may not be displayed and the user may be made aware of the occurrence of an event by other means. For example, a countdown sound may be output or text may be displayed indicating that an event will occur.

[0048] In the above description, the object is displayed in such a manner that the second position is identifiable. However, the second position does not have to be displayed in an identifiable manner. For example, the user may obtain a hint regarding the second position, and may move the information processing device 100 in various directions to search for the second position. That is, the object 12 may be transparent.

[0049] In the above, the image providing system using AR technology has been described. However, the present invention is also applicable to an image providing system using VR technology or MR technology. When using VR technology, the display unit 130 in the information processing device 100 may be realized by an HMD, or the information processing device 100 itself may be realized by an HMD.

[0050] The programs of the embodiments of the present disclosure may be provided in a state stored in a storage medium readable by an information processing device. The storage medium can store the programs in a "non-transitory tangible medium." The programs include, for example, software programs and control programs. When each functional unit of the information processing device 100 is realized by software, the information processing device 100 functions as an acquisition unit 111, a display control unit 112, a determination unit 113, and a generation unit 114 by the processor executing the programs loaded on the memory.

[0051] The storage medium may, where appropriate, include one or more semiconductor-based or other integrated circuits (ICs) (e.g., field programmable gate arrays (FPGAs), application specific ICs (ASICs), etc.), hard disk drives (HDDs), hybrid hard drives (HHDs), optical disks, optical disk drives (ODDs), magneto-optical disks, magneto-optical drives, floppy diskettes, floppy disk drives (FDDs), magnetic tapes, solid state drives (SSDs), RAM drives, secure digital cards or drives, any other suitable storage media, or any suitable combination of two or more of these. The storage medium may, where appropriate, be volatile, non-volatile, or a combination of volatile and non-volatile.

[0052] Furthermore, the program of the present disclosure may be provided to the information processing device 100 via any transmission medium (such as a communication network or broadcast waves) capable of transmitting the program.

[0053] In addition, each embodiment of the present disclosure may be realized in the form of a data signal embedded in a carrier wave in which a program is embodied by electronic transmission. The program of the present disclosure may be implemented using, for example, a script language such as JavaScript (registered trademark) or Python, C language, Go language, Swift, Koltin, Java (registered trademark), or the like. [Explanation of symbols]

[0054] 100 Information processing device 110 Control section 111 Acquisition Department 112 Display control unit 113 Judgment section 114 Generation part 120 Communications Department 130 Display section 140 Input / output section 150 Sensors 170 Storage section

Claims

1. A display unit that displays a virtual space image; a display control unit that causes a virtual space image corresponding to an orientation of the display unit to be displayed on the display unit; a determination unit that determines whether a predetermined first position on the display unit and a predetermined second position in the virtual space overlap; Equipped with The display control unit displays the second position on the virtual space image in an identifiable manner, and, in response to the determination unit determining that the first position and the second position overlap, displays a predetermined image corresponding to the second position on the display unit.

2. a generation unit for generating the virtual space image; Further equipped with The display control unit causes the virtual space image generated by the generation unit to be displayed on the display unit. The information processing device according to claim 1 .

3. A communication unit for receiving the virtual space image; Further equipped with The display control unit causes the virtual space image received by the communication unit to be displayed on the display unit. The information processing device according to claim 1 .

4. The display control unit causes the display unit to display a time gauge for a time period set according to the second position, the time gauge changing according to a time period during which the first position and the second position overlap. The information processing device according to claim 1 .

5. The display control unit displays the predetermined image in response to an overlap between the first location and the second location spanning a time period set in accordance with the second location. The information processing device according to claim 4.

6. The first position is a center of the display unit. The information processing device according to claim 1 .

7. The display control unit changes a display mode of a center of the display unit in the virtual space image displayed on the display unit, The information processing device according to claim 1 .

8. A step of displaying a virtual space image according to an orientation of a display unit on the display unit; determining whether a predetermined first position on the display unit overlaps with a predetermined second position in the virtual space; Including, The display step is a control method for an information processing device, in which the second position on the virtual space image is displayed in an identifiable manner, and a predetermined image corresponding to the second position is displayed on the display unit in response to a determination that the first position and the second position overlap.

9. In the information processing device, A function of displaying a virtual space image on the display unit according to the orientation of the display unit; a function of determining whether a predetermined first position on the display unit and a predetermined second position in the virtual space overlap; Realize this, The display function is a control program of an information processing device that displays the second position on the virtual space image in an identifiable manner, and, when it is determined that the first position and the second position overlap, displays a predetermined image corresponding to the second position on the display unit.

Images