Program, information processing method and information processing device

Abstract

To provide a program, an information processing method and an information processing device capable of displaying objects displayed on a globe in a captured image as an augmented reality in a real space outside the globe area.SOLUTION: An augmented reality system causes a computer 2 to execute processing of: when a globe 1 in a real space is held up thereto, displaying a globe in a first virtual space in response to the movement of the globe 1, and displaying a plurality of objects at prescribed positions on the first virtual space; when receiving selection of any of the objects, displaying the selected object in a second virtual space different from the first virtual space, in a glove area of the first virtual space; and when receiving a command to extend the object in the second virtual space into the real space, three-dimensionally displaying the object in the second virtual space as an augmented reality in a real space outside the globe area.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a program, an information processing method, and an information processing device. [Background technology]

[0002] In recent years, products using an Augmented Reality (AR) system have been developed. In the AR system, an application program for recognizing an AR marker is installed in an information device having a camera and a display unit, such as AR glasses (AR goggles) such as Apple (registered trademark) Vision Pro, a smartphone, a tablet terminal, a portable game machine, etc. When an object with an AR marker is photographed using such an information device, the information device recognizes the AR marker and displays content corresponding to the recognized AR marker superimposed on the photographed image. This allows a user to view an image in which the content is superimposed on the photographed object on the display unit. Patent Document 1 discloses a system that photographs a three-dimensional object and displays additional information such as an image or text superimposed on the three-dimensional object in the photographed image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2013-92407 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the invention of Patent Document 1 has a problem in that objects displayed on a globe (three-dimensional object) in a captured image cannot be displayed as augmented reality in real space outside the globe area.

[0005] One aspect is to provide a program or the like that can display an object displayed on a globe in a captured image as augmented reality in real space outside the globe area. [Means for solving the problem]

[0006] A program relating to one aspect causes a computer to execute a process in which, when a globe in real space is held up, a globe in a first virtual space is displayed in response to the movement of the globe, and multiple objects are displayed at predetermined positions in the first virtual space; when a selection of any of the objects is accepted, the selected object is displayed in a second virtual space different from the first virtual space, within the globe area of ​​the first virtual space; and when a command is accepted to extend an object in the second virtual space into real space, the object in the second virtual space is displayed as augmented reality in real space outside the globe area. Effect of the Invention

[0007] In one aspect, it is possible to display an object displayed on a globe in a captured image as augmented reality in real space outside the globe area. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a schematic diagram showing a configuration example of an AR system. [Diagram 2] FIG. 2 is a block diagram showing an example of the configuration of a user terminal. [Diagram 3] FIG. 11 is an explanatory diagram showing an example of the record layout of a dinosaur DB and a position image DB. [Figure 4] FIG. 13 is an explanatory diagram showing an example of a selection screen for multiple dinosaurs in the first virtual space. [Diagram 5] FIG. 11 is an explanatory diagram showing an example of a screen displaying a second virtual space. [Figure 6] FIG. 13 is an explanatory diagram showing an example of a dinosaur expansion screen. [Figure 7]FIG. 13 is an explanatory diagram showing an example of a screen on which a dinosaur is enlarged to life-size using a slider. [Figure 8] FIG. 13 is an explanatory diagram showing an example of a screen displaying dinosaurs and images in a second virtual space. [Figure 9] 11 is a flowchart showing a processing procedure for displaying a dinosaur in a globe area as an augmented reality in a real space outside the globe area. [Figure 10] 13 is a flowchart showing the processing steps of a subroutine for processing to display a plurality of dinosaurs in the first virtual space. [Figure 11] 13 is a flowchart showing the procedure of a subroutine for displaying dinosaurs as augmented reality in real space outside the globe area. [Figure 12] 13 is a flowchart showing a processing procedure for enlarging or reducing a dinosaur displayed as augmented reality. [Figure 13] 13 is a flowchart showing the procedure of a subroutine for enlarging or reducing a dinosaur. [Figure 14] 13 is a flowchart showing a processing procedure when switching from the second virtual space to the first virtual space. [Figure 15] 13 is a flowchart showing a processing procedure for displaying dinosaurs and images in a second virtual space. [Figure 16] FIG. 13 is an explanatory diagram showing an example of a dinosaur expansion screen in the second embodiment. [Figure 17] 13 is a flowchart showing a processing procedure for displaying a second dinosaur together with a first dinosaur in a real space outside the globe area as augmented reality. [Figure 18] FIG. 13 is an explanatory diagram showing an example of a display screen of a board. [Figure 19] 13 is a flowchart showing a processing procedure for displaying a plurality of boards. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, the present invention will be described in detail with reference to the drawings showing embodiments thereof.

[0010] (Embodiment 1) The first embodiment relates to a form in which, when a globe (earth ball) in real space is held up, an object in the globe area is displayed as an augmented reality in real space outside the globe area. The object includes, for example, animals such as dinosaurs, elephants, pandas, or giraffes, vehicles such as airplanes or ships, national flags, plants, or buildings. Note that, in this embodiment, an example in which the object is a dinosaur is described, but the present invention can be applied to other types of objects in the same manner.

[0011] 1 is a schematic diagram showing a configuration example of an AR system. The AR system of this embodiment includes a globe 1, which is a sphere having AR markers (markers) on its surface, and an information processing terminal 2. The globe 1 is used as a subject, and is photographed by the information processing terminal 2.

[0012] The globe 1 is, for example, a hollow sphere made of plastic, but is not limited to this. A map of the earth is printed on the surface of the globe 1, and AR markers are attached at appropriate positions on the map. For example, in the example shown in Fig. 1, each of the oceans and continents on the map is given a name, and each country and city is given a country name and city name. These characters constitute the AR markers.

[0013] Although omitted in Fig. 1, if the map includes national borders of each country, the national borders may also be included as AR markers. If the rivers, plains, mountain ranges, etc. on the map are colored in a specific color according to the terrain, each color may also be included as an AR marker.

[0014] In addition, the oceans in the map may be marked with a specific mark to identify each ocean, ocean area, etc. Or, a specific mark may be marked to identify an animal such as a ship, a turtle, or a seagull, etc. Furthermore, the continents in the map may be marked with a specific mark to identify each continent, region, etc.

[0015] The information processing terminal 2 is a terminal device of each user. The information processing terminal 2 is, for example, an information processing device such as AR glasses such as Apple Vision Pro, a smartphone, a mobile phone, a wearable device such as Apple Watch (registered trademark), a tablet, a game machine, or a personal computer terminal. For the sake of simplicity, the information processing terminal 2 will be read as a user terminal 2 below.

[0016] When a user terminal 2 according to this embodiment is held up over globe 1 in real space, it displays a globe in a first virtual space in response to the movement of globe 1, and also displays a plurality of dinosaurs at predetermined positions in the first virtual space.

[0017] When the user terminal 2 receives a selection of any one of the multiple dinosaurs, the user terminal 2 displays the selected dinosaur in a second virtual space different from the first virtual space within the globe area of ​​the first virtual space. When the user terminal 2 receives a command to extend the dinosaur in the second virtual space into real space, the user terminal 2 displays the dinosaur in the second virtual space as augmented reality in the real space outside the globe area.

[0018] Augmented reality makes it possible to display objects in a virtual space in a way that is full of realism. However, with conventional technology, it is difficult to achieve a realistic feel when displaying objects such as dinosaurs superimposed on a globe in a virtual space.

[0019] In contrast, in addition to the virtual space where an object is superimposed on a globe, the area occupied by the globe in the real space can be regarded as one spherical virtual space (first virtual space) to realize an augmented reality with a high sense of realism. The first virtual space has a second virtual space where the object is displayed large and the object can be freely rotated and observed by rotating the globe, and a real space outside the globe area where the object is displayed even larger. By using such a first virtual space, an augmented reality with a high sense of realism can be realized.

[0020] 2 is a block diagram showing an example of the configuration of the user terminal 2. The user terminal 2 includes a control unit 21, a storage unit 22, a communication unit 23, an input unit 24, a display unit 25, an imaging unit 26, and a large-capacity storage unit 27. Each component is connected by a bus B.

[0021] The control unit 21 includes an arithmetic processing device such as a central processing unit (CPU), a micro-processing unit (MPU), a graphics processing unit (GPU), a field programmable gate array (FPGA), a digital signal processor (DSP), or a quantum processor. The control unit 21 reads out and executes a control program 2P (program product) stored in the storage unit 22, thereby performing various information processing or control processing related to the user terminal 2. Note that, although the control unit 21 is described in FIG. 2 as being a single processor, it may be a multiprocessor.

[0022] The storage unit 22 includes memory elements such as a RAM (Random Access Memory) and a ROM (Read Only Memory), and stores a control program 2P or data required for the control unit 21 to execute processing. The storage unit 22 also temporarily stores data required for the control unit 21 to execute arithmetic processing. The communication unit 23 is a communication module for performing processing related to communication, and transmits and receives information to and from an external information processing device, etc., via the network N.

[0023] The input unit 24 may be a controller, a gesture input device for sensing a gesture, a keyboard, a mouse, or a touch panel integrated with the display unit 25. When using a gesture input device, the control unit 21 recognizes the position and shape of the hand from an image of the hand captured by the image capture unit 26, for example. The control unit 21 determines the gesture (movement) of the hand based on the recognized position and shape of the hand. The control unit 21 performs a corresponding operation according to the determined hand gesture. For example, when the control unit 21 determines a gesture such as pinching with the thumb and index finger, it may perform an operation such as selecting an object at the position of the hand.

[0024] Display unit 25 is a liquid crystal display or an organic EL (electroluminescence) display, etc., and displays various information according to instructions from control unit 21. Photography unit 26 is, for example, a photography device such as a CCD (Charge Coupled Device) camera or a CMOS (Complementary Metal Oxide Semiconductor) camera. Note that photography unit 26 does not need to be built into user terminal 2, but may be configured to be externally connected directly to user terminal 2 and capable of taking photographs.

[0025] The mass storage unit 27 includes a recording medium such as a hard disk drive (HDD) or a solid state drive (SSD). The mass storage unit 27 includes a dinosaur DB (database) 271 and a position image DB 272. The dinosaur DB 271 stores information about dinosaurs. The position image DB 272 stores images (e.g., background images) at each position.

[0026] In this embodiment, the storage unit 22 and the large-capacity storage unit 27 may be configured as an integrated storage device. The large-capacity storage unit 27 may be configured with a plurality of storage devices. Furthermore, the large-capacity storage unit 27 may be an external storage device connected to the user terminal 2.

[0027] FIG. 3 is an explanatory diagram showing an example of the record layout of the dinosaur DB 271 and the position image DB 272.

[0028] The dinosaur DB 271 includes a dinosaur ID column, a scientific name column, a size column, a habitat column, an image column, and a description column. The dinosaur ID column stores a unique dinosaur ID for identifying each dinosaur. The scientific name column stores the scientific name of the dinosaur. The size column stores the size (total length) of the dinosaur.

[0029] The habitat column stores the habitat of the dinosaur or the longitude and latitude of the habitat, etc. The image column stores image data of the dinosaur. The description column stores description information of the dinosaur, including food, habitat period, characteristics, etc.

[0030] The position image DB 272 includes an image ID column, a longitude and latitude column, and an image column. The image ID column stores a unique image ID to identify an image at each position. The longitude and latitude column stores the longitude and latitude of the position. The image column stores image data of the image at each position.

[0031] The above-mentioned storage formats of each DB are merely examples, and other storage formats may be used as long as the relationships between the data are maintained.

[0032] Continuing, this is an explanatory diagram for explaining the processing operation of the AR system. The user terminal 2 acquires information about dinosaurs from an external information processing device (for example, a server device). The information about dinosaurs includes the scientific name, size, habitat, image, and description information of the dinosaur.

[0033] The user terminal 2 assigns a dinosaur ID to the acquired information about the dinosaur. The user terminal 2 associates the scientific name, size, habitat, image, and explanatory information of the dinosaur with the assigned dinosaur ID and stores the dinosaur's scientific name, size, habitat, image, and explanatory information as one record in the dinosaur DB 271. Note that the information about the dinosaur may be stored in the dinosaur DB 271 in advance.

[0034] 4 is an explanatory diagram showing an example of a selection screen for a plurality of dinosaurs in the first virtual space. The screen includes a first virtual space display field 11a and a dinosaur display field 11b.

[0035] The first virtual space display field 11a is a display field that displays the first virtual space. The first virtual space is a virtual space that includes a globe area and a real space area outside the globe area. The dinosaur display field 11b is a display field that displays the scientific name and image of the dinosaur. The dinosaur display field 11b is provided according to the number of dinosaurs.

[0036] When the user terminal 2 is held up over the globe 1 in real space, it captures a photographed image of the globe 1 having markers on its surface via the photographing unit 26, and displays the photographed image of the globe 1 including the globe area in the first virtual space display area 11a.

[0037] The user terminal 2 identifies the position of the globe area displayed in the captured image based on the marker information in the captured image of the globe 1. For example, the user terminal 2 recognizes AR marker information according to the globe area displayed in the captured image. The user terminal 2 extracts position information based on the recognized AR marker information.

[0038] The user terminal 2 refers to the extracted location information and acquires information about the corresponding multiple dinosaurs (such as the scientific name, size, habitat, image, or description information of the dinosaur) from the dinosaur DB 271 based on the habitat of each dinosaur. The user terminal 2 acquires the longitude and latitude corresponding to the habitat of each acquired dinosaur. For example, the user terminal 2 may acquire the longitude and latitude corresponding to the habitat using a Google Map service or the like.

[0039] The user terminal 2 displays a globe in the first virtual space, and also displays a number of dinosaurs at predetermined positions in the first virtual space. As shown in the figure, the user terminal 2 displays the held-up globe 1 in the first virtual space display field 11a, and also displays the scientific name and image of each dinosaur in the corresponding dinosaur display field 11b at the longitude and latitude corresponding to the habitat of each dinosaur in the first virtual space.

[0040] When the user terminal 2 detects a movement such as a rotation of the globe 1 or a change in the relative position between the user terminal 2 and the globe 1, the user terminal 2 recaptures the captured image of the globe 1 in response to the detected movement. The user terminal 2 displays the recaptured captured image of the globe 1 in the first virtual space display field 11a.

[0041] Similar to the above-described process, the user terminal 2 re-specifies the position of the globe area displayed in the captured image based on the marker information in the captured image of the globe 1. The user terminal 2 refers to the re-specified position and acquires information on the corresponding multiple dinosaurs from the dinosaur DB 271. The user terminal 2 displays the scientific name and image of each dinosaur in the corresponding dinosaur display column 11b at the longitude and latitude corresponding to the acquired habitat of each dinosaur.

[0042] When the user terminal 2 accepts a selection of any of the dinosaurs displayed at a specified position in the first virtual space, it displays the selected dinosaur in a second virtual space, which is different from the first virtual space, within the globe area of ​​the first virtual space.

[0043] Specifically, when the user terminal 2 receives a touch (click) operation on the dinosaur display area 11b, it passes the selected dinosaur ID and image data to the display screen of the second virtual space (Figure 5) and transitions to the display screen of the second virtual space.

[0044] Note that the above-mentioned touch operation is not limited to this. For example, when the input unit 24 is a gesture input device, the user terminal 2 may accept a selection operation of a dinosaur displayed in the dinosaur display field 11b when it determines a gesture such as pinching with the thumb and index finger based on the position and shape of the hand recognized from the image of the hand. Note that, for the sake of simplicity, an example of a touch operation will be described below, but the same can be applied to a selection operation when a gesture input device or the like is used.

[0045] Fig. 5 is an explanatory diagram showing an example of a screen displaying the second virtual space. Note that the same reference numerals are used for the same contents as in Fig. 4, and the description thereof will be omitted. The screen includes a second virtual space display field 11c and a switching button 11d.

[0046] The second virtual space display field 11c is a display field that displays the second virtual space. The second virtual space is a virtual space within the globe region of the first virtual space. The switching button 11d is a button for switching from the second virtual space to the first virtual space.

[0047] The user terminal 2 receives the dinosaur ID and image data passed from the dinosaur selection screen (FIG. 4). The user terminal 2 displays the received dinosaur image data in the second virtual space display field 11c. When the user terminal 2 receives a touch operation of the switching button 11d, it switches from the second virtual space to the first virtual space and returns to the dinosaur selection screen (FIG. 4).

[0048] When the user terminal 2 receives a command to extend the dinosaur displayed in the second virtual space display field 11c into real space, it passes the dinosaur ID and image data to the dinosaur's extended screen (Figure 6) and transitions to the dinosaur's extended screen.

[0049] The command may be, for example, a touch operation by the user on the dinosaur in the second virtual space, or a touch operation on a button (not shown) that expands the dinosaur into the real space. Furthermore, the command may be a drag-and-drop operation or a swipe operation on the dinosaur in the second virtual space. For example, the user terminal 2 drags and drops the dinosaur in the second virtual space into the real space, and expands the dinosaur to the position where it was dropped in the real space.

[0050] Fig. 6 is an explanatory diagram showing an example of a dinosaur expansion screen. The expansion screen is a screen for displaying dinosaurs in the second virtual space as an augmented reality in the real space outside the globe area. Note that the same reference numerals are used for the contents that overlap with Figs. 4 and 5, and the description thereof will be omitted.

[0051] The screen includes a real-space dinosaur display field 11f and a slider 11e. The real-space dinosaur display field 11f is a display field that displays dinosaurs in real space outside the globe area as augmented reality. The slider 11e is a slider for enlarging dinosaurs to life size as augmented reality.

[0052] The user terminal 2 receives the dinosaur ID and image data passed from the display screen of the second virtual space (FIG. 5). The user terminal 2 calculates the initial display size of the dinosaur in the second virtual space. For example, the user terminal 2 acquires the size of the dinosaur from the dinosaur DB 271 based on the dinosaur ID. The user terminal 2 calculates the initial display size based on the acquired dinosaur size using a predetermined ratio (for example, 10%).

[0053] The user terminal 2 displays the dinosaur in the second virtual space as an augmented reality in the real space outside the globe area of ​​the first virtual space according to the calculated initial display size of the dinosaur in the second virtual space. As shown in the figure, the dinosaur in the second virtual space is displayed in the real space dinosaur display field 11f according to the initial display size of the dinosaur.

[0054] When the user terminal 2 receives a touch operation of the slider 11e, the user terminal 2 enlarges the dinosaur to a life-size display size as an augmented reality, or reduces the dinosaur to its initial display size. As shown in the figure, the slider 11e is provided in a horizontally long bar-shaped area such that the size of the dinosaur is smaller on the left side and larger on the right side. The size of the dinosaur can be changed by performing an operation of sliding the slider 11e left and right.

[0055] In the present embodiment, the slider 11e is used to enlarge or reduce the size of the dinosaur, but the present invention is not limited to this. For example, a button for enlarging or reducing the size of the dinosaur may be provided.

[0056] For example, when the user terminal 2 receives an operation of sliding the slider 11e to the left, the size of the dinosaur displayed as augmented reality is gradually reduced to the initial display size of the dinosaur based on the size of the dinosaur. Alternatively, when the user terminal 2 receives an operation of sliding the slider 11e to the right, the size of the dinosaur displayed as augmented reality is gradually increased to a life-size display size based on the size of the dinosaur.

[0057] 7 is an explanatory diagram showing an example of a screen in which a dinosaur is enlarged to life-size by the slider 11e. As shown in the figure, when the user terminal 2 receives an operation of sliding the slider 11e to the rightmost side, the dinosaur displayed as augmented reality is enlarged to a life-size display size based on the size of the dinosaur. The user terminal 2 displays the dinosaur enlarged to the life-size display size in the real-space dinosaur display field 11f.

[0058] The enlargement process is not limited to the above. For example, the user terminal 2 may perform the enlargement process by referring to the size of an object in the real space outside the globe area of ​​the first virtual space as a reference. For example, the user terminal 2 refers to the diameter of the globe 1 in the real space outside the globe area of ​​the first virtual space, and calculates the enlargement ratio to be used in the enlargement process based on the life-size display size of the dinosaur to be enlarged. The user terminal 2 performs the enlargement process of the dinosaur according to the calculated enlargement ratio.

[0059] Alternatively, the user terminal 2 calculates the magnification ratio based on the distance between the position in real space where the object is placed and the position of the user terminal 2. The user terminal 2 may perform magnification processing of the dinosaur according to the calculated magnification ratio. By using the magnification processing, a dramatic effect can be obtained in which the dinosaur becomes larger the further away it is from the dinosaur.

[0060] Next, a process of displaying a dinosaur in the second virtual space together with an image at a position corresponding to the dinosaur will be described.

[0061] 8 is an explanatory diagram showing an example of a screen displaying a dinosaur and an image in the second virtual space. The user terminal 2 acquires the habitat of the dinosaur and image data of the dinosaur from the dinosaur DB 271 based on the dinosaur ID. The user terminal 2 acquires an image at a position corresponding to the dinosaur.

[0062] Specifically, the user terminal 2 acquires the longitude and latitude corresponding to the habitat of the acquired dinosaur. The user terminal 2 acquires the corresponding image data from the position image DB 272 based on the acquired longitude and latitude. As shown in the figure, image data including the sky and grassland is acquired from the position image DB 272. The user terminal 2 displays the acquired image data of the dinosaur and image data at the position corresponding to the dinosaur in the second virtual space display field 11c.

[0063] In addition, the image of the dinosaur and the image at the position corresponding to the dinosaur are changed in response to the movement of the globe in the first virtual space. Specifically, when the user terminal 2 detects a movement such as the rotation of the globe in the first virtual space or a change in the relative position between the user terminal 2 and the globe 1, the user terminal 2 acquires the rotation direction of the globe. The user terminal 2 rotates the image data of the dinosaur and the image data at the position corresponding to the dinosaur in accordance with the acquired rotation direction, and displays them in the second virtual space display field 11c.

[0064] 9 is a flowchart showing a processing procedure for displaying dinosaurs in a globe area as an augmented reality in a real space outside the globe area. The control unit 21 of the user terminal 2 executes a subroutine for processing to display a plurality of dinosaurs in a first virtual space (step S201). The subroutine for processing to display in the first virtual space will be described later. The control unit 21 accepts a selection of one of the plurality of dinosaurs via the input unit 24 (step S202).

[0065] Control unit 21 displays the selected dinosaur in the second virtual space by display unit 25 (step S203). Control unit 21 executes a subroutine for processing to display the dinosaur in the second virtual space as an augmented reality in the real space outside the globe area of ​​the first virtual space (step S204). The subroutine for processing to display in the real space outside the globe area will be described later. Control unit 21 ends the processing.

[0066] 10 is a flowchart showing the processing procedure of a subroutine for processing to display a plurality of dinosaurs in the first virtual space. When the user holds up the globe 1 in the real space, the control unit 21 of the user terminal 2 captures a captured image of the globe 1 having markers on its surface via the imaging unit 26 (step S01). The captured image includes the globe area and a real space area outside the globe area. The control unit 21 displays the captured captured image in the first virtual space on the display unit 25 (step S02).

[0067] The control unit 21 recognizes the AR marker information according to the globe area displayed in the captured image (step S03). The control unit 21 identifies the position of the globe area displayed in the captured image based on the recognized marker information (step S04). The control unit 21 refers to the identified position information and acquires information about the corresponding multiple dinosaurs (such as the scientific name, size, habitat, image or description information of the dinosaur) based on the habitat of each dinosaur from the dinosaur DB 271 of the mass storage unit 27 (step S05).

[0068] The control unit 21 acquires the habitat of each dinosaur from the acquired information about the multiple dinosaurs (step S06). The control unit 21 acquires the longitude and latitude corresponding to the acquired habitat of each dinosaur, for example, using the Google Maps service (step S07). The control unit 21 displays the globe of the first virtual space on the display unit 25 (step S08).

[0069] Control unit 21 displays the acquired multiple dinosaurs in the first virtual space (step S09). Specifically, control unit 21 displays the scientific name and image of each dinosaur at the longitude and latitude corresponding to the habitat of each dinosaur in the first virtual space on display unit 25. Control unit 21 ends the subroutine of the display process in the first virtual space and returns.

[0070] 11 is a flowchart showing a subroutine for displaying dinosaurs as augmented reality in the real space outside the globe region. The control unit 21 of the user terminal 2 receives a command for augmenting the dinosaurs in the second virtual space into the real space (such as a touch operation, a drag-and-drop operation, or a swipe operation on the dinosaurs in the second virtual space by the user) through the input unit 24 (step S11).

[0071] The control unit 21 calculates the initial display size of the dinosaur in the second virtual space (step S12). Specifically, the control unit 21 acquires the size of the dinosaur from the dinosaur DB 271 of the mass storage unit 27 based on the dinosaur ID. The control unit 21 calculates the initial display size based on the acquired dinosaur size at a predetermined ratio (for example, 10%).

[0072] Control unit 21 displays the dinosaur in the second virtual space as an augmented reality in the real space outside the globe area of ​​the first virtual space according to the calculated initial display size of the dinosaur in the second virtual space via display unit 25 (step S13). Control unit 21 ends the subroutine of the display process in the real space outside the globe area and returns.

[0073] Fig. 12 is a flowchart showing the processing procedure when enlarging or reducing a dinosaur displayed as an augmented reality. Note that the same reference numerals are used for the same contents as in Fig. 9, and the description thereof will be omitted. After executing the process of step S204, the control unit 21 of the user terminal 2 executes a subroutine for processing to enlarge or reduce the dinosaur as an augmented reality (step S205). Note that the subroutine for processing to enlarge or reduce the dinosaur will be described later.

[0074] Control unit 21 again displays the enlarged or reduced dinosaur as an augmented reality in the real space outside the globe area of ​​the first virtual space via display unit 25 (step S206). Control unit 21 ends the process.

[0075] 13 is a flowchart showing the procedure of a subroutine for enlarging or reducing a dinosaur. The control unit 21 of the user terminal 2 displays a slider (slider 11e in FIG. 6) for enlarging the dinosaur to life size as an augmented reality on the display unit 25 (step S21). The control unit 21 accepts an operation of the displayed slider through the input unit 24 (step S22).

[0076] Based on the received operation, the control unit 21 determines whether the operation on the dinosaur is an enlargement operation (step S23). Specifically, when the control unit 21 receives an operation of sliding the slider to the left, the control unit 21 determines that the operation on the dinosaur is a reduction operation. Alternatively, when the control unit 21 receives an operation of sliding the slider to the right, the control unit 21 determines that the operation on the dinosaur is an enlargement operation.

[0077] When the operation on the dinosaur is an enlargement operation (YES in step S23), the control unit 21 calculates an enlarged display size based on the position slid on the slider, the initial display size of the dinosaur (the display size of the dinosaur corresponding to the leftmost part of the slider), and the life-size display size of the dinosaur (the display size of the dinosaur corresponding to the rightmost part of the slider) (step S24). The control unit 21 performs a process of enlarging the dinosaur displayed as augmented reality according to the calculated enlarged display size (step S25). The control unit 21 ends the subroutine of the dinosaur enlargement / reduction process and returns.

[0078] If the operation on the dinosaur is not an enlargement operation (NO in step S23), the control unit 21 calculates a reduced display size based on the position slid on the slider, the initial display size of the dinosaur, and the life-size display size of the dinosaur (step S26). The control unit 21 performs a process of reducing the dinosaur displayed as augmented reality according to the calculated reduced display size (step S27). The control unit 21 ends the subroutine of the dinosaur enlargement / reduction process and returns.

[0079] Fig. 14 is a flowchart showing the processing procedure when switching from the second virtual space to the first virtual space. Note that the same reference numerals are used for the same contents as in Fig. 9, and the description thereof will be omitted. After executing the process of step S203, the control unit 21 of the user terminal 2 displays a switching button for switching from the second virtual space to the first virtual space on the display unit 25 (step S211).

[0080] The control unit 21 receives a touch operation of the switching button through the input unit 24 (step S212). The control unit 21 switches from the second virtual space to the first virtual space (step S213). The control unit 21 ends the process.

[0081] Fig. 15 is a flowchart showing the processing procedure for displaying dinosaurs and images in the second virtual space. Note that the same reference numerals are used for the contents that overlap with Fig. 9, and the description thereof will be omitted. After executing the process of step S202, the control unit 21 of the user terminal 2 acquires an image of a position corresponding to the selected dinosaur from the position image DB 272 of the mass storage unit 27 based on the longitude and latitude corresponding to the habitat of the dinosaur (step S221).

[0082] Control unit 21 displays image data of the selected dinosaur and image data at a position corresponding to the dinosaur in the second virtual space on display unit 25 (step S222). When control unit 21 detects a movement such as a rotation of the globe in the first virtual space or a change in the relative position between user terminal 2 and globe 1 (step S223), control unit 21 acquires the rotation direction of the globe (step S224).

[0083] Control unit 21 rotates the image data of the dinosaur and the image data at the position corresponding to the dinosaur in accordance with the acquired rotation direction and displays them in the second virtual space via display unit 25 (step S225). Control unit 21 returns to the process of step S223.

[0084] According to this embodiment, when a globe 1 in real space is held up, a globe in a first virtual space is displayed in response to the movement of globe 1, and multiple dinosaurs can be displayed at specified positions in the first virtual space.

[0085] According to this embodiment, it is possible to display a dinosaur selected in the first virtual space in the second virtual space.

[0086] According to this embodiment, it is possible to display dinosaurs in the second virtual space in the real space outside the globe area as augmented reality.

[0087] According to this embodiment, it is possible to enlarge the dinosaur displayed as augmented reality to life-size based on the size information of the dinosaur.

[0088] According to this embodiment, it is possible to display a dinosaur in a second virtual space together with an image at a position corresponding to the dinosaur in the second virtual space.

[0089] (Embodiment 2) The second embodiment relates to a form in which a second dinosaur different from the first dinosaur is displayed as an augmented reality in a real space outside the globe area together with a dinosaur (a first dinosaur). Note that a description of the contents that overlap with the first embodiment will be omitted.

[0090] Fig. 16 is an explanatory diagram showing an example of a dinosaur expansion screen in the second embodiment. Note that explanations of contents overlapping with Fig. 6 will be omitted. The screen includes a real space first dinosaur display field 11f (real space dinosaur display field 11f in Fig. 6) and a real space second dinosaur display field 11g. The real space second dinosaur display field 11g is a display field that displays the second dinosaur as an augmented reality in the real space outside the globe area.

[0091] When the user terminal 2 displays the first dinosaur as an augmented reality in the real space outside the globe area of ​​the first virtual space, the user terminal 2 accepts the selection of a second dinosaur, different from the first dinosaur, among the multiple dinosaurs displayed in a predetermined position (e.g., the dinosaur's habitat) in the first virtual space through a selection screen (FIG. 4) for multiple dinosaurs in the first virtual space. The user terminal 2 displays the selected second dinosaur in the second virtual space display field 11c.

[0092] When the user terminal 2 receives a command to extend the second dinosaur in the second virtual space displayed in the second virtual space display area 11c into real space (for example, a touch operation on the second dinosaur by the user), the user terminal 2 displays the second dinosaur in the second virtual space together with the first dinosaur as augmented reality in the real space outside the globe area.

[0093] As shown in the figure, a first dinosaur displayed in the real space first dinosaur display area 11f and a second dinosaur displayed in the real space second dinosaur display area 11g are displayed as augmented reality in the real space outside the globe area.

[0094] Also, by sliding the slider 11e left or right, the size of the second dinosaur can be changed in the same way as the first dinosaur. Note that the process of changing the size of the second dinosaur is the same as in the first embodiment, so a description thereof will be omitted.

[0095] 17 is a flowchart showing a processing procedure for displaying the second dinosaur together with the first dinosaur in the real space outside the globe area as an augmented reality. Note that the same reference numerals are used to designate the same contents as those in FIG. 9, and the description thereof will be omitted.

[0096] After executing the process of step S204, the control unit 21 of the user terminal 2 accepts a selection of a second dinosaur different from the first dinosaur from among the multiple dinosaurs through the input unit 24 (step S231). The control unit 21 displays the selected second dinosaur in the second virtual space through the display unit 25 (step S232). The control unit 21 executes a subroutine of a process of displaying the second dinosaur in the second virtual space as an augmented reality in the real space outside the globe area of ​​the first virtual space (step S233). The control unit 21 ends the process.

[0097] In the present embodiment, an example has been described in which the first dinosaur and the second dinosaur are displayed as augmented reality in the real space outside the globe area, but the present invention is not limited to this, and the number of dinosaurs is not particularly limited.

[0098] According to this embodiment, it is possible to display the second dinosaur in the second virtual space together with the first dinosaur in the real space outside the globe area as augmented reality.

[0099] (Embodiment 3) The third embodiment relates to a form in which, when a dinosaur selection is accepted, multiple boards on which information related to dinosaurs is written are displayed around the globe area. Note that a description of the contents that overlap with the first and second embodiments will be omitted.

[0100] 18 is an explanatory diagram showing an example of a board display screen. The screen includes a board display field 12a. The board display field 12a is a field in which information related to dinosaurs is written around the globe area. Note that multiple board display fields 12a may be provided according to the information related to dinosaurs.

[0101] The user terminal 2 displays the scientific name and image of each dinosaur in the corresponding dinosaur display field 11b at the longitude and latitude corresponding to the habitat of each dinosaur in the first virtual space. When the user terminal 2 receives a touch operation on the dinosaur display field 11b, it receives the selection of the corresponding dinosaur. Based on the selected dinosaur ID, the user terminal 2 obtains information about the dinosaur, including the habitat or explanatory information, from the dinosaur DB 271. The explanatory information includes food, habitat period, characteristics, etc.

[0102] The user terminal 2 generates a plurality of board display fields 12a in which the acquired information about the dinosaurs is described. The user terminal 2 displays the generated plurality of board display fields 12a around the globe area of ​​the first virtual space. As shown in the figure, the board display field 12a in which food is described, the board display field 12a in which habitat periods are described, and the board display field 12a in which characteristics are described are displayed around the globe area.

[0103] Furthermore, even if the globe in the first virtual space is rotated, the display positions of the board display area 12a describing food, the board display area 12a describing habitat season, and the board display area 12a describing characteristics can be maintained.

[0104] Fig. 19 is a flowchart showing the processing procedure when displaying multiple boards. Note that the same reference numerals are used for the contents that overlap with Fig. 9, and the description thereof will be omitted. After executing the process of step S202, the control unit 21 of the user terminal 2 acquires information about the dinosaur (habitat or explanatory information, etc.) from the dinosaur DB 271 of the mass storage unit 27 based on the selected dinosaur ID (step S241).

[0105] Control unit 21 generates a plurality of boards on which the acquired information about the dinosaurs is described (step S242). Control unit 21 displays the generated plurality of boards around the globe area of ​​the first virtual space via display unit 25 (step S243). When control unit 21 detects a movement such as a rotation of the globe in the first virtual space or a change in the relative position between user terminal 2 and globe 1 (step S245), control unit 21 performs a process for maintaining the display positions of the plurality of boards displayed around the globe area of ​​the first virtual space (step S246). Control unit 21 returns to the process of step S245.

[0106] According to this embodiment, when a selection of a dinosaur is accepted, it is possible to display a plurality of boards on which information relating to dinosaurs is written around the globe area.

[0107] According to this embodiment, by displaying multiple boards, it becomes possible to check information about dinosaurs at a glance.

[0108] The embodiments disclosed herein are illustrative in all respects and should not be considered as limiting. The scope of the present invention is defined by the claims, not by the above meaning, and is intended to include all modifications within the scope and meaning equivalent to the claims.

[0109] The matters described in each embodiment can be combined with each other. In addition, the independent claims and dependent claims described in the claims can be combined with each other in any combination regardless of the reference format. Furthermore, the claims use a format in which a claim references two or more other claims (multiple claim format), but this is not limited to this format. [Explanation of symbols]

[0110] 1. Globe 2. Information processing terminal (user terminal; computer) 21 Control section 22 Memory section 23 Communications Department 24 Input section 25 Display section 26 Photography Department 27 Mass storage 271 Dinosaur DB 272 Location Image DB 2P control program

Claims

1. when a globe in real space is held up, a globe in a first virtual space is displayed in response to a movement of the globe, and a plurality of objects are displayed at predetermined positions in the first virtual space; When a selection of any one of the objects is accepted, the selected object is displayed in a second virtual space different from the first virtual space within a globe area of ​​the first virtual space; When a command to extend an object in the second virtual space into a real space is received, the object in the second virtual space is displayed as an augmented reality in a real space outside a globe area. A program that causes a computer to carry out processing.

2. Obtain size information for the object; The object displayed as augmented reality is enlarged to life-size based on the acquired size information of the object. The program according to claim 1.

3. displaying a slider for enlarging the object displayed as augmented reality to life-size; Use the slider to enlarge the object to life size. The program according to claim 2.

4. When a selection of a second object different from the object is accepted, the selected second object is displayed in the second virtual space; When a command to extend a second object in the second virtual space into real space is received, the second object in the second virtual space is displayed as an augmented reality in real space outside the globe area together with the object.

3. The program according to claim 1 or 2.

5. The object in the second virtual space is displayed in the second virtual space together with an image of the object at a position corresponding to the object.

3. The program according to claim 1 or 2.

6. Changing the image and the object at the position corresponding to the object in response to the movement of the globe. The program according to claim 5.

7. displaying a button for switching from the second virtual space to the first virtual space when the second virtual space is being displayed; When an operation of the displayed button is accepted, the second virtual space is switched to the first virtual space.

3. The program according to claim 1 or 2.

8. When the selection of the object is accepted, a plurality of boards on which information related to the object is described are displayed around the globe area.

3. The program according to claim 1 or 2.

9. Maintaining display positions of the plurality of boards regardless of detection of movement of the globe The program according to claim 8.

10. when a globe in real space is held up, a globe in a first virtual space is displayed in response to a movement of the globe, and a plurality of objects are displayed at predetermined positions in the first virtual space; When a selection of any one of the objects is accepted, the selected object is displayed in a second virtual space different from the first virtual space within a globe area of ​​the first virtual space; When a command to extend an object in the second virtual space into a real space is received, the object in the second virtual space is displayed as an augmented reality in a real space outside a globe area. An information processing method in which processing is performed by a computer.

11. An information processing device including a control unit, The control unit is when a globe in real space is held up, a globe in a first virtual space is displayed in response to a movement of the globe, and a plurality of objects are displayed at predetermined positions in the first virtual space; When a selection of any one of the objects is accepted, the selected object is displayed in a second virtual space different from the first virtual space within a globe area of ​​the first virtual space; When a command to extend an object in the second virtual space into a real space is received, the object in the second virtual space is displayed as an augmented reality in a real space outside a globe area. Information processing device.

Images