Information processing device, information processing method, and program

Abstract

Control the display of content depending on the user's location. [Solution] An information processing device 1 has a memory unit 12 that associates and stores object identification information, which is information for identifying an object, and object attribute information that indicates the attributes of the object, a reception unit 131 that receives designation of content to be displayed from content owned by the user and object identification information from an information terminal used by the user, and a display control unit 132 that controls the content to be displayed to be superimposed on the information terminal 2 based on the content to be displayed received by the reception unit 131 and the object attribute information corresponding to the object identification information received by the reception unit 131.

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] There is known a technique for detecting a marker included in a captured image and displaying an AR (Augumented Reality) content corresponding to the marker (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2015-41126 A Summary of the Invention [Problem to be solved by the invention]

[0004] When displaying AR content outdoors, it is common for a marker to correspond to the content to be displayed. It is conceivable that any content can be displayed when a user has a terminal recognize a marker, but this can lead to a problem that content that is not suitable for the characteristics of the object to which the marker is attached or the location where the object to which the marker is attached is displayed.

[0005] Therefore, the present invention has been made in consideration of these points, and aims to make it possible to control the display of content according to the characteristics of a marked object and the location where the marked object is placed. [Means for solving the problem]

[0006] An information processing device of a first aspect of the present invention has a memory unit that associates and stores object identification information, which is information for identifying an object, and object attribute information that indicates attributes of the object, a reception unit that receives designation of content to be displayed from content owned by a user and the object identification information from an information terminal used by the user, and a display control unit that controls the content to be displayed to be superimposed on the information terminal based on the content to be displayed received by the reception unit and the object attribute information corresponding to the object identification information received by the reception unit.

[0007] The memory unit may further store content information associating content with object attribute information of an object capable of displaying the content, and the display control unit may display the content to be displayed when the content information indicates that the content to be displayed can be displayed for the object attribute information corresponding to the object identification information.

[0008] The display control unit may be configured to not display the content to be displayed when the content information indicates that the content to be displayed cannot be displayed, based on the object attribute information corresponding to the object identification information.

[0009] The reception unit may receive designation of multiple pieces of content to be displayed from among the content owned by the user, and the display control unit may determine which pieces of content to display from among the multiple pieces of content to be displayed based on object attribute information corresponding to the object identification information, and display the determined content on the user's information terminal.

[0010] The memory unit may further store content information associating a reference content, the object attribute information, and corresponding content which is content displayed in response to the reference content and the object attribute information, and the display control unit may (1) identify object attribute information corresponding to the object identification information, and (2) display, on the user's information terminal, the corresponding content associated in the content information with the specified content and the identified object attribute information.

[0011] The device may further include a generation unit that generates content to be displayed on the user's information terminal based on the specified content and the object attribute information corresponding to the object identification information received by the reception unit, and the display control unit may display the content generated by the generation unit on the user's information terminal.

[0012] The memory unit may further store a display mode table associating an external environment with a display mode of content corresponding to the external environment, and may further have an acquisition unit that acquires environmental information indicating the external environment around the information terminal used by the user, and the display control unit may determine a display mode of the content to be displayed on the user's information terminal based on the external environment indicated by the environmental information, and may display the content to be displayed in the determined display mode.

[0013] The information terminal may further include an acquisition unit that acquires environmental information indicating the external environment surrounding the information terminal used by the user, and the memory unit may further store content information that associates a reference content, an external environment, and corresponding content that is content that is displayed in response to the reference content and the external environment, and the display control unit may display, on the user's information terminal, the specified content and the corresponding content associated in the content information with the external environment indicated by the environmental information.

[0014] An information processing method of a second aspect of the present invention is executed by a computer and includes the steps of receiving, from an information terminal used by a user, designation of content to be displayed among content owned by the user and object identification information, which is information for identifying an object, and a step of referencing a memory unit that stores the object identification information in association with object attribute information indicating attributes of an object corresponding to the object identification information, and controlling the content to be displayed to be superimposed on the information terminal based on the content to be displayed received by the receiving unit and the object attribute information corresponding to the object identification information received by the receiving unit.

[0015] In a third aspect of the program of the present invention, a computer is caused to execute the steps of receiving, from an information terminal used by a user, designation of content to be displayed among content owned by the user and object identification information, which is information for identifying an object, and referring to a memory unit that stores the object identification information in association with object attribute information indicating attributes of an object corresponding to the object identification information, and controlling the content to be displayed to be superimposed and displayed on the information terminal based on the content to be displayed received by the receiving unit and the object attribute information corresponding to the object identification information received by the receiving unit. Effect of the Invention

[0016] According to the present invention, it is possible to control the display of content depending on the location of the user. [Brief description of the drawings]

[0017] [Figure 1] 1 is a diagram for explaining an overview of an information processing system S according to an embodiment. [Diagram 2] 1 is a block diagram showing a configuration of an information processing device 1. FIG. [Diagram 3] 4 is a diagram showing an example of a data structure of user information stored in a storage unit 12. FIG. [Figure 4] 3 is a diagram showing an example of a data structure of object information stored in a storage unit 12. FIG. [Diagram 5] 2 is a diagram showing an example of a data structure of content information stored in a storage unit 12. FIG. [Figure 6] 2 is a diagram showing an example of a data structure of content information stored in a storage unit 12. FIG. [Figure 7] 3 is a flowchart showing a process flow in the information processing device 1. [Figure 8] FIG. 13 is a block diagram showing a configuration of an information processing device 1B according to a modified example. [Figure 9] FIG. 13 is a block diagram showing a configuration of an information processing device 1A according to a modified example. [Figure 10] 2 is a diagram showing an example of a data structure of content information stored in a storage unit 12. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] [Outline of Information Processing System S] 1 is a diagram for explaining an overview of an information processing system S. The information processing system S is a system for displaying AR (Augmented Reality) content. The information processing system S includes an information processing device 1 and an information terminal 2.

[0019] The information processing device 1 is a device for displaying AR content. The information processing device 1 causes the information terminal 2 that recognizes the marker attached to the object OB to display AR content according to the attributes of the object OB. As an example, the object OB is a structure installed to indicate that the AR content can be displayed on the information terminal 2. A natural object that exists in a place where the AR content can be displayed and has a marker attached thereto may be used as the object OB. The object OB is placed at a scenic spot, a famous place, etc. An attribute is set for each object OB.

[0020] The attribute of the object OB is information for explaining the object OB, and may be information about the surroundings of the object OB or information about the object OB itself. Examples of the information about the surroundings of the object OB include the location where the object OB is installed, an event or incident that occurred in the vicinity of the location where the object OB is installed, or information related to the location where the object OB is installed. For example, when there is content such as music, drama, or anime set in the location, the information related to the location where the object OB is installed may include information indicating the content, characters appearing in the content, etc. The information indicating the content may be, for example, the title of the content or information about a creator or business that has sold, created, or distributed the content. Examples of the information about the object OB itself include information about the structure, type, material, color, size, or weight of the object OB.

[0021] The information terminal 2 is a terminal used by a user who uses a service provided in the information processing system S. The information terminal 2 is, for example, a smartphone, a tablet, or a personal computer. The information terminal 2 has a camera, captures an image of an object OB, recognizes a marker attached to the object OB included in the captured image, and identifies object identification information. For this purpose, the information terminal 2 stores information for recognizing a target marker and object identification information that identifies the object when the marker is recognized, in association with each other. The object identification information is information for identifying the object OB. The object identification information is ID (IDentification) assigned to the object. The information terminal 2 transmits the object identification information and information indicating display-compatible content to the information processing device 1, and displays the AR content acquired from the information processing device 1 on the display unit.

[0022] In the following, an example will be described in which the information terminal 2 recognizes the marker attached to the object OB, but the information terminal 2 may recognize the shape of the object OB to identify the object identifying information.

[0023] An overview of the processing in the information processing system S will be described. The information terminal 2 acquires object identification information ((1) in FIG. 1). More specifically, the information terminal 2 captures an image of the object OB with a camera, recognizes a marker included in the captured image, and identifies the object identification information corresponding to the recognized marker. The information terminal 2 accepts a selection of content to be displayed ((2) in FIG. 1). The information terminal 2 accepts a selection of a content ID indicating the content to be displayed from the user.

[0024] The information terminal 2 transmits a content ID indicating the selected content and the acquired object identification information to the information processing device 1 ((3) in FIG. 1). The information processing device 1 identifies the attributes of the object OB corresponding to the acquired object identification information ((4) in FIG. 1). The information processing device 1 determines whether or not to display the content based on the acquired content ID and the attributes of the object OB corresponding to the object identification information ((5) in FIG. 1).

[0025] The information processing device 1 stores object identification information and attributes of an object indicated by the object identification information in association with each other. The information processing device 1 also stores content information that associates content with attributes of an object capable of displaying the content.

[0026] For example, if the object OB is an aquarium-type ornament, the object identification information includes, for example, the attribute of "water" as the attribute of the object OB. In this case, when content having "water" as an attribute of a displayable object is selected, the information processing device 1 determines that the content is displayable.

[0027] When the information processing device 1 determines that the selected content can be displayed based on the attributes of the object indicated by the acquired object identification information, the information processing device 1 controls the information terminal 2 to display the selected content ((6) in FIG. 1). As an example, the content is AR content, and the information processing device 1 controls the information terminal 2 to display the AR content by superimposing it on an image captured by the information terminal 2.

[0028] By configuring the information processing system S to determine whether or not content can be displayed depending on the attributes of an object, it is possible to control the display of content depending on the object OB that the user has caused the information terminal 2 to recognize.

[0029] [Configuration of information processing device 1] 2 is a block diagram showing the configuration of the information processing device 1. The information processing device 1 includes a communication unit 11, a storage unit 12, and a control unit 13. The control unit 13 includes a reception unit 131 and a display control unit 132.

[0030] The communication unit 11 is a communication interface for transmitting and receiving data to and from other devices via a network. The storage unit 12 is a storage medium including a ROM (Read Only Memory), a RAM (Random Access Memory), an SSD (Solid State Drive), a hard disk drive, etc. The storage unit 12 stores in advance a program to be executed by the control unit 13.

[0031] The storage unit 12 stores user information. Fig. 3 is a diagram showing an example of a data structure of the user information stored in the storage unit 12. In the user information, a user ID for identifying a user is associated with content owned by the user.

[0032] The storage unit 12 stores object identification information, which is information for identifying an object, and object attribute information indicating attributes of the object, in association with each other. Fig. 4 is a diagram showing an example of a data structure of object information stored in the storage unit 12. In the object information, the object identification information is associated with attributes given to the object OB indicated by the object identification information. In the object information, multiple attributes may be associated with each other.

[0033] Fig. 5 is a diagram showing an example of the data structure of content information stored in storage unit 12. In the content information shown in Fig. 5, "content" and "displayable attributes" are associated with each other. "Content" indicates a content ID. "Displayable attributes" indicate attributes of an object OB that can display the corresponding content. The "displayable attributes" may have multiple attributes.

[0034] The control unit 13 is a processor such as a CPU (Central Processing Unit), etc. The control unit 13 executes a program stored in the storage unit 12, thereby functioning as a reception unit 131 and a display control unit 132.

[0035] The reception unit 131 receives designation of content to be displayed from among the content owned by the user, and object identification information from the information terminal 2 used by the user. As an example, the display control unit 132 refers to the user information stored in the storage unit 12 and identifies the content owned by the user. The display control unit 132 causes the information terminal 2 to display a screen for receiving selection of content to be displayed from among the content owned by the user. The user operates the information terminal 2 to select the content to be displayed. The information terminal 2 transmits the content ID of the content selected by the user to the information processing device 1, and the reception unit 131 acquires it.

[0036] The display control unit 132 controls the content to be displayed to be superimposed on the information terminal 2 based on the content to be displayed received by the receiving unit 131 and the object attribute information corresponding to the object identification information received by the receiving unit 131. As an example, the display control unit 132 refers to the object information stored in the storage unit 12 and identifies the object attribute information corresponding to the object identification information received by the receiving unit 131. The display control unit 132 refers to the content information stored in the storage unit 12 and determines whether the content indicated by the content ID received by the receiving unit 131 can be displayed for the object attribute information corresponding to the object identification information received by the receiving unit 131. If the content can be displayed, the display control unit 132 causes the information terminal 2 to display the content.

[0037] When the content information indicates that the content to be displayed can be displayed for the object attribute information corresponding to the object identification information, the display control unit 132 displays the content to be displayed. When the content can be displayed, the display control unit 132 controls the AR content indicated by the content ID accepted by the accepting unit 131 to be displayed superimposed on the display unit of the information terminal 2.

[0038] When the content information indicates that the content to be displayed cannot be displayed for the object attribute information corresponding to the object identification information, the display control unit 132 does not display the content to be displayed. When the content cannot be displayed, the display control unit 132, for example, causes the information terminal 2 to display a screen indicating that the content cannot be displayed. The display control unit 132 may also cause the information terminal 2 to display a screen indicating the content that can be displayed for the attribute of the object OB, among the contents owned by the user.

[0039] By configuring the information processing device 1 to control whether or not to display content depending on the attributes of an object, it is possible to control the display of content depending on the object OB that the user has caused the information terminal 2 to recognize.

[0040] The information processing device 1 may be configured to receive designation of a plurality of pieces of content and determine the content to be displayed based on the attributes of the object OB.

[0041] The reception unit 131 receives designation of multiple contents to be displayed from among the contents owned by the user. As an example, the display control unit 132 causes the information terminal 2 to display a screen configured so that the multiple contents to be displayed can be selected from the contents owned by the user. The reception unit 131 acquires content IDs of the multiple contents selected on the displayed screen.

[0042] The display control unit 132 determines which of the multiple display target contents is to be displayed based on the object attribute information corresponding to the object identification information, and displays the determined content on the user's information terminal 2. The display control unit 132 refers to the content information and determines whether each of the multiple content IDs acquired by the reception unit 131 is displayable for the attribute of the object corresponding to the object identification information. The display control unit 132 displays on the information terminal 2 the content corresponding to the content ID determined to be displayable after the multiple content IDs acquired by the reception unit 131.

[0043] By configuring the information processing device 1 to display, among a plurality of contents, contents that can be displayed according to the attributes of an object, it becomes possible to control the contents to be displayed according to the location where the object OB is placed. As a result, it is possible to motivate the user to move the content to a location where it can be displayed.

[0044] [Processing flow in information processing device 1] Fig. 6 is a flowchart showing the flow of processing in the information processing device 1. The flowchart shown in Fig. 6 starts at the point when preparations for accepting content designation are complete.

[0045] The receiving unit 131 receives object identification information and content to be displayed (S01). The display control unit 132 refers to the object information stored in the storage unit 12, and identifies attributes of the object OB corresponding to the acquired object identification information (S02).

[0046] The display control unit 132 refers to the content information stored in the storage unit 12, and determines whether or not the content can be displayed based on the content to be displayed and the attribute of the identified object OB (S03).

[0047] If the content can be displayed (YES in S03), the display control unit 132 causes the information terminal 2 to display the content (S04). Then, the information processing device 1 ends the process.

[0048] If the content cannot be displayed (NO in S03), the display control unit 132 causes the information terminal 2 to display a screen indicating that the content cannot be displayed (S05). Then, the information processing device 1 ends the process.

[0049] <Variation 1> In the above description, an example has been described in which it is determined whether or not content can be displayed based on the attributes of an object. However, the information processing device 1 may be configured to determine the display mode of content depending on the attributes of an object.

[0050] The storage unit 12 further stores content information that associates a reference content, object attribute information, and corresponding content that is content displayed in response to the reference content and the object attribute information. FIG. 7 is a diagram showing an example of a data structure of content information stored in the storage unit 12. In the content information shown in FIG. 7, "content", "attribute", and "corresponding content" are associated. In this case, "content" indicates the content ID of the reference content. "Corresponding content" indicates the display mode of the content that is displayed when the object attribute information matches the corresponding "attribute". As an example, if the "content" is an anime character, the "corresponding content" is content that depicts the character represented by the content in different costumes and poses for each corresponding attribute. In the "corresponding content", multiple corresponding contents may be associated with one piece of content.

[0051] The display control unit 132 identifies object attribute information corresponding to the object identifying information. The display control unit 132 refers to the object information stored in the storage unit 12, and identifies object attribute information corresponding to the object identifying information accepted by the accepting unit 131.

[0052] The display control unit 132 displays the designated content and the corresponding content associated in the content information with the identified object attribute information on the user's information terminal 2. The display control unit 132 refers to the content information stored in the storage unit 12, and identifies the content ID accepted by the acceptance unit 131 and the corresponding content corresponding to the identified object attribute information. The display control unit 132 displays the identified corresponding content on the information terminal 2.

[0053] By configuring the information processing device 1 in this way, it becomes possible to change the display mode of the content according to the attributes of the object, which can motivate the user to move to the locations where various objects OB are placed. <Variation 2>

[0054] The information processing device 1 may be configured to display content generated based on the selected content and the attribute of the object OB on the information terminal 2. Fig. 8 is a block diagram showing the configuration of an information processing device 1A according to Modification 2. The information processing device 1A according to Modification 2 further includes a generation unit 133.

[0055] In this case, the storage unit 12 stores a trained model. The trained model is trained to output content obtained by modifying the specified content based on the specified content and a prompt written in a natural language.

[0056] The generating unit 133 generates content to be displayed on the information terminal 2 of the user based on the specified content and the object attribute information corresponding to the object identification information received by the receiving unit 131. The generating unit 133 generates a prompt to be input to the trained model based on the specified content and the specified object attribute information. As an example, when the specified object attribute information is "mountain", the generating unit 133 generates a prompt stating "Please modify the specified content to match "mountain". The generating unit 133 inputs the generated prompt and the content specified by the user to the trained model, and causes the trained model to output the modified content. The display control unit 132 causes the content generated by the generating unit 133 to be displayed on the information terminal 2 of the user.

[0057] The information processing device 1A according to the modified example is configured in this way, so that it becomes possible to cause the information terminal 2 to display content in a flexible manner according to the attributes of the object OB.

[0058] <Variation 3> The information processing device 1 may be configured to display content selected according to the external environment. The configuration of an information processing device 1B according to Modification 3 is shown in Fig. 9. In the information processing device 1B according to Modification 3, the control unit 13 further has an acquisition unit 134. The control unit 13 functions as the acquisition unit 134 by executing a program stored in the storage unit 12.

[0059] In this case, the storage unit 12 stores a display mode corresponding to the external environment (for example, the background of the content and the pose of the character which is the content). More specifically, the storage unit 12 further stores content information that associates the reference content, the external environment, and the corresponding content which is the content displayed corresponding to the reference content and the external environment. FIG. 10 is a diagram showing an example of the data structure of the content information stored in the storage unit 12 in this case. The storage unit 12 further stores a display mode table that associates the external environment with the display mode of the content corresponding to the external environment.

[0060] It further includes an acquisition unit 134 that acquires environmental information indicating the external environment around the information terminal 2 used by the user. The environmental information indicates, for example, the weather, temperature, humidity, or the like around the location of the information processing device 1. As an example, the environmental information acquisition unit 136 acquires location information indicating the location of the information terminal 2. The environmental information acquisition unit 136 transmits the location information of the information terminal 2 to an external server (not shown) and acquires the environmental information corresponding to the location of the information terminal 2. The display control unit 132 determines the display mode of the content to be displayed on the user's information terminal 2 based on the external environment indicated by the environmental information, and displays the content to be displayed in the determined display mode. When the display control unit 132 determines that the content can be displayed based on the object attribute information, it specifies the display mode corresponding to the external environment indicated by the environmental information. The display control unit 132 displays the content on the information terminal 2 in the specified display mode.

[0061] More specifically, the display control unit 132 causes the information terminal 2 of the user to display the designated content and the corresponding content associated in the content information with the external environment indicated by the environmental information.

[0062] Since the information processing device 1B in the modified example is configured to display content corresponding to the external environment on the information terminal 2, it becomes possible to change the content according to the external environment, thereby motivating the user to visit the same place multiple times.

[0063] Furthermore, this invention will make it possible to contribute to Goal 9 of the United Nations' Sustainable Development Goals (SDGs), which is to "build resilient infrastructure, promote industry, innovation and infrastructure."

[0064] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. For example, all or part of the device can be configured by distributing or integrating functionally or physically in any unit. In addition, new embodiments resulting from any combination of multiple embodiments are also included in the embodiments of the present invention. The effect of the new embodiment resulting from the combination combines the effect of the original embodiment. [Explanation of symbols]

[0065] 1. Information processing device 2. Information terminal 11 Communications Department 12 Storage section 13 Control section 131 Reception 132 Display control section 133 Generation part 134 Acquisition Department

Claims

1. a storage unit that stores object identification information, which is information given to an object arranged in real space in order to identify the object, and object attribute information, which indicates attributes of the object and is information for describing the object or a location where the object is located, in association with each other, and also stores content information that associates content with object attribute information of an object that can display the content; a reception unit that receives designation of content to be displayed from among content owned by a user and the object identification information from an information terminal used by the user; a display control unit that controls the information terminal so as to display the content to be displayed superimposed on an image of the object captured by the information terminal when the object attribute information corresponding to the object identification information received by the receiving unit matches the displayable object attribute information associated with the content to be displayed in the content information; An information processing device having the above configuration.

2. the display control unit does not display the content to be displayed when object attribute information corresponding to the object identification information does not match the displayable object attribute information associated with the content to be displayed in the content information. The information processing device according to claim 1 .

3. The receiving unit receives designation of a plurality of contents to be displayed from among the contents owned by the user; the display control unit determines a content to be displayed from among a plurality of contents to be displayed based on object attribute information corresponding to the object identification information, and causes the determined content to be displayed on the information terminal of the user. The information processing device according to claim 1 .

4. the storage unit further stores content information associating a reference content, the object attribute information, and corresponding content that is content displayed in response to the reference content and the object attribute information; The display control unit (1) identifies object attribute information corresponding to the object identification information, and (2) displays, on the information terminal of the user, a designated content and the corresponding content associated in the content information with the identified object attribute information. The information processing device according to claim 1 .

5. a generation unit that generates content to be displayed on the information terminal of the user based on the designated content and the object attribute information corresponding to the object identification information received by the reception unit, The display control unit causes the content generated by the generation unit to be displayed on the information terminal of the user. The information processing device according to claim 1 .

6. the storage unit further stores a display mode table that associates an external environment with a display mode of the content corresponding to the external environment; An acquisition unit that acquires environmental information indicating an external environment around the information terminal used by the user, the display control unit determines a display mode of the content to be displayed on the information terminal of the user based on the external environment indicated by the environmental information, and displays the content to be displayed in the determined display mode. The information processing device according to claim 1 .

7. An acquisition unit that acquires environmental information indicating an external environment around the information terminal used by the user, the storage unit further stores content information associating a reference content, an external environment, and a corresponding content that is a content displayed in response to the reference content and the external environment; the display control unit causes the information terminal of the user to display the designated content and the corresponding content associated in the content information with the external environment indicated by the environmental information. The information processing device according to claim 1 .

8. The computer executes receiving, from an information terminal used by the user, designation of content to be displayed among the content owned by the user and object identification information which is information given to an object in order to identify the object disposed in real space; a step of referring to a storage unit that stores the object identification information in association with object attribute information indicating attributes of an object corresponding to the object identification information, the object attribute information being information for describing the object or a location where the object is located, and that stores content information in association with object attribute information of an object capable of displaying the content, and when the object attribute information corresponding to the received object identification information is indicated in the content information as the displayable object attribute information associated with the content to be displayed, controlling the information terminal to display the content to be displayed superimposed on an image captured by the information terminal of the object; An information processing method comprising the steps of:

9. On the computer, receiving, from an information terminal used by the user, object identification information that is information given to an object in order to specify a content to be displayed among the contents owned by the user and an object arranged in real space; a step of referring to a storage unit that stores the object identification information in association with object attribute information indicating attributes of an object corresponding to the object identification information, the object attribute information being information for describing the object or a location where the object is located, and that stores content information in association with object attribute information of an object capable of displaying the content, and when the object attribute information corresponding to the received object identification information is indicated in the content information as the displayable object attribute information associated with the content to be displayed, controlling the information terminal to display the content to be displayed superimposed on an image captured by the information terminal of the object; A program that executes the following.

Images