Information processing apparatus, information processing method, and program

Abstract

To control display of content according to a place where a user is present.SOLUTION: An information processing device 1 includes a storage unit 12 that stores object identification information which is information for identifying an object and object attribute information indicating an attribute of the object in association with each other, a reception unit 131 that receives designation of display target content among pieces of content possessed by a user and the object identification information from an information terminal used by the user, and a display control unit 132 that performs control such that the display target content is superimposed and displayed on the information terminal 2 based on the display target content received by the reception unit 131 and the object attribute information corresponding to the object identification information received by the reception unit 131.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] There is known a technique for detecting a marker included in a captured image and displaying an AR (Augumented Reality) content corresponding to the marker (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-41126 Summary of the Invention [Problem to be solved by the invention]

[0004] When displaying AR content outdoors, it is common for a marker to correspond to the content to be displayed. While it is conceivable that any content can be displayed when a user has a device recognize a marker, there is a possibility that content that is not suitable for the characteristics of the object to which the marker is attached or the location where the object to which the marker is attached may be displayed.

[0005] Therefore, the present invention has been made in consideration of these points, and aims to make it possible to control the display of content according to the characteristics of the marked object and the location where the marked object is placed. [Means for solving the problem]

[0006] An information processing device of a first aspect of the present invention includes a memory unit that stores object identification information, which is information for identifying an object, and object attribute information that indicates the attributes of the object, in association with each other; a reception unit that receives designation of content to be displayed from content owned by a user and the object identification information from an information terminal used by the user; and a display control unit that controls the content to be displayed to be superimposed on the information terminal based on the content to be displayed received by the reception unit and the object attribute information corresponding to the object identification information received by the reception unit.

[0007] The storage unit may further store content information that associates content with object attribute information of an object that can display the content, and the display control unit may display the content to be displayed when the content information indicates that the content to be displayed can be displayed for the object attribute information corresponding to the object identification information.

[0008] The display control unit may be configured to not display the content to be displayed when the content information indicates that the content to be displayed cannot be displayed, relative to the object attribute information corresponding to the object identification information.

[0009] The reception unit may receive a specification of multiple pieces of content to be displayed from among the content owned by the user, and the display control unit may determine the content to be displayed from among the multiple pieces of content to be displayed based on object attribute information corresponding to the object identification information, and display the determined content on the user's information terminal.

[0010] The storage unit may further store content information that associates a reference content, the object attribute information, and corresponding content that is content that is displayed in response to the reference content and the object attribute information, and the display control unit may (1) identify the object attribute information that corresponds to the object identification information, and (2) display the corresponding content that is associated in the content information with the specified content and the identified object attribute information on the user's information terminal.

[0011] The device may further include a generation unit that generates content to be displayed on the user's information terminal based on the specified content and the object attribute information corresponding to the object identification information received by the reception unit, and the display control unit may display the content generated by the generation unit on the user's information terminal.

[0012] The storage unit may further store a display mode table that associates an external environment with a display mode of content corresponding to the external environment, and may further have an acquisition unit that acquires environmental information that indicates the external environment around the information terminal used by the user, and the display control unit may determine the display mode of the content to be displayed on the user's information terminal based on the external environment indicated by the environmental information, and display the content to be displayed in the determined display mode.

[0013] The information terminal may further include an acquisition unit that acquires environmental information indicating the external environment surrounding the information terminal used by the user, and the storage unit may further store content information that associates a reference content, an external environment, and corresponding content that is content that is displayed in correspondence with the reference content and the external environment, and the display control unit may display, on the user's information terminal, the specified content and the corresponding content associated in the content information with the external environment indicated by the environmental information.

[0014] An information processing method of a second aspect of the present invention includes the steps of receiving, executed by a computer, from an information terminal used by the user, object identification information, which is information for specifying content to be displayed among content owned by the user and for identifying an object; and referring to a memory unit that stores the object identification information in association with object attribute information indicating attributes of the object corresponding to the object identification information, and controlling the content to be displayed to be superimposed on the information terminal based on the content to be displayed received by the receiving unit and the object attribute information corresponding to the object identification information received by the receiving unit.

[0015] In a third aspect of the program of the present invention, a computer is caused to execute the steps of: receiving, from an information terminal used by the user, designation of content to be displayed among content owned by the user and object identification information, which is information for identifying an object; and referencing a memory unit that stores the object identification information in association with object attribute information indicating the attributes of the object corresponding to the object identification information, and controlling the content to be displayed to be superimposed on the information terminal based on the content to be displayed received by the receiving unit and the object attribute information corresponding to the object identification information received by the receiving unit. [Effects of the Invention]

[0016] According to the present invention, it is possible to control the display of content depending on the location of the user. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a diagram for explaining an overview of an information processing system S according to an embodiment. [Figure 2] 1 is a block diagram showing a configuration of an information processing device 1. FIG. [Figure 3] 3 is a diagram showing an example of a data structure of user information stored in a storage unit 12. FIG. [Figure 4] 3 is a diagram showing an example of a data structure of object information stored in a storage unit 12. FIG. [Figure 5] 3 is a diagram showing an example of a data structure of content information stored in a storage unit 12. FIG. [Figure 6] 3 is a diagram showing an example of a data structure of content information stored in a storage unit 12. FIG. [Figure 7] 3 is a flowchart showing the flow of processing in the information processing device 1. [Figure 8] FIG. 10 is a block diagram showing a configuration of an information processing device 1B according to a modified example. [Figure 9] FIG. 10 is a block diagram showing a configuration of an information processing device 1A according to a modified example. [Figure 10] 3 is a diagram showing an example of a data structure of content information stored in a storage unit 12. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0018] [Outline of Information Processing System S] 1 is a diagram illustrating an overview of an information processing system S. The information processing system S is a system for displaying AR (Augmented Reality) content. The information processing system S includes an information processing device 1 and an information terminal 2.

[0019] The information processing device 1 is a device for displaying AR content. The information processing device 1 causes an information terminal 2 that recognizes a marker attached to an object OB to display AR content according to the attributes of the object OB. As an example, the object OB is a structure installed to indicate that the information terminal 2 can display AR content. The object OB may also be a natural object with a marker attached that is present in a location where the AR content can be displayed. The object OB is placed at a scenic spot, a famous place, etc. Each object OB has an attribute set thereto.

[0020] The attributes of an object OB are information for describing the object OB, and may be information about the surroundings of the object OB or information about the object OB itself. Examples of information about the surroundings of the object OB include the location where the object OB is installed, events or incidents that occurred in the vicinity of the location where the object OB is installed, or information related to the location where the object OB is installed. For example, if there is content such as music, drama, or anime set in the location, the information related to the location where the object OB is installed may include information indicating the content, characters appearing in the content, etc. Information indicating the content may, for example, be the title of the content or information about the creator or business that sold, created, distributed, etc. the content. Examples of information about the object OB itself include information about the structure, type, material, color, size, or weight of the object OB.

[0021] The information terminal 2 is a terminal used by a user who uses a service provided by the information processing system S. The information terminal 2 is, for example, a smartphone, a tablet, or a personal computer. The information terminal 2 has a camera, captures an image of an object OB, recognizes a marker attached to the object OB included in the captured image, and identifies object identification information. To this end, the information terminal 2 stores information for recognizing a target marker and object identification information that identifies the object when the marker is recognized, in association with each other. The object identification information is information for identifying the object OB. The object identification information is an ID (ID) assigned to the object. The information terminal 2 transmits the object identification information and information indicating display-compatible content to the information processing device 1, and displays the AR content acquired from the information processing device 1 on a display unit.

[0022] In the following, an example will be described in which the information terminal 2 recognizes a marker attached to the object OB, but the information terminal 2 may also recognize the shape of the object OB and identify the object-identifying information.

[0023] An overview of the processing in the information processing system S will be described. The information terminal 2 acquires object identification information ((1) in FIG. 1). More specifically, the information terminal 2 captures an image of the object OB using a camera, recognizes markers included in the captured image, and identifies object identification information corresponding to the recognized marker. The information terminal 2 accepts a selection of content to be displayed ((2) in FIG. 1). The information terminal 2 accepts, from the user, a selection of a content ID indicating the content to be displayed.

[0024] The information terminal 2 transmits a content ID indicating the selected content and the acquired object identification information to the information processing device 1 ((3) in FIG. 1). The information processing device 1 identifies the attributes of the object OB corresponding to the acquired object identification information ((4) in FIG. 1). The information processing device 1 determines whether or not to display the content based on the acquired content ID and the attributes of the object OB corresponding to the object identification information ((5) in FIG. 1).

[0025] The information processing device 1 stores object identification information and attributes of the object indicated by the object identification information in association with each other. The information processing device 1 also stores content information that associates content with attributes of the object that can display the content.

[0026] For example, if the object OB is an aquarium-shaped ornament, the object identification information includes, for example, the attribute of "water" as an attribute of the object OB. In this case, when content having "water" as an attribute of a displayable object is selected, the information processing device 1 determines that the content can be displayed.

[0027] When the information processing device 1 determines that the selected content can be displayed based on the attributes of the object indicated by the acquired object identification information, the information processing device 1 controls the information terminal 2 to display the selected content ((6) in FIG. 1). As an example, the content is AR content, and the information processing device 1 controls the information terminal 2 to display the AR content superimposed on an image captured by the information terminal 2.

[0028] By configuring the information processing system S to determine whether or not content can be displayed depending on the attributes of an object, it is possible to control the display of content depending on the object OB that the user has caused the information terminal 2 to recognize.

[0029] [Configuration of information processing device 1] 2 is a block diagram showing the configuration of the information processing device 1. The information processing device 1 has a communication unit 11, a storage unit 12, and a control unit 13. The control unit 13 has a reception unit 131 and a display control unit 132.

[0030] The communication unit 11 is a communication interface for transmitting and receiving data to and from other devices via a network. The storage unit 12 is a storage medium including a ROM (Read Only Memory), a RAM (Random Access Memory), an SSD (Solid State Drive), a hard disk drive, etc. The storage unit 12 stores in advance a program to be executed by the control unit 13.

[0031] The storage unit 12 stores user information. Fig. 3 is a diagram showing an example of the data structure of the user information stored in the storage unit 12. In the user information, a user ID that identifies a user is associated with content owned by the user.

[0032] The storage unit 12 stores object identification information, which is information for identifying an object, and object attribute information, which indicates the attributes of the object, in association with each other. Fig. 4 is a diagram showing an example of the data structure of the object information stored in the storage unit 12. In the object information, the object identification information is associated with attributes given to the object OB indicated by the object identification information. In the object information, multiple attributes may be associated.

[0033] Fig. 5 is a diagram showing an example of the data structure of content information stored in storage unit 12. In the content information shown in Fig. 5, "content" and "displayable attributes" are associated with each other. "Content" indicates a content ID. "Displayable attributes" indicate attributes of an object OB that can display the corresponding content. "Displayable attributes" may have multiple attributes.

[0034] The control unit 13 is a processor such as a CPU (Central Processing Unit), etc. The control unit 13 executes a program stored in the storage unit 12, thereby functioning as a reception unit 131 and a display control unit 132.

[0035] The reception unit 131 receives, from the information terminal 2 used by the user, designation of content to be displayed from among the content owned by the user and object identification information. As an example, the display control unit 132 refers to the user information stored in the storage unit 12 and identifies the content owned by the user. The display control unit 132 causes the information terminal 2 to display a screen for receiving selection of content to be displayed from among the content owned by the user. The user operates the information terminal 2 to select the content to be displayed. The information terminal 2 transmits the content ID of the content selected by the user to the information processing device 1, and the reception unit 131 acquires this.

[0036] The display control unit 132 controls the content to be displayed to be superimposed on the information terminal 2, based on the content to be displayed received by the receiving unit 131 and the object attribute information corresponding to the object identification information received by the receiving unit 131. As an example, the display control unit 132 refers to the object information stored in the storage unit 12 and identifies the object attribute information corresponding to the object identification information received by the receiving unit 131. The display control unit 132 refers to the content information stored in the storage unit 12 and determines whether the content indicated by the content ID received by the receiving unit 131 can be displayed for the object attribute information corresponding to the object identification information received by the receiving unit 131. If the content can be displayed, the display control unit 132 causes the information terminal 2 to display the content.

[0037] When the content information indicates that the content to be displayed can be displayed for the object attribute information corresponding to the object identification information, the display control unit 132 displays the content to be displayed. When the content can be displayed, the display control unit 132 controls the AR content indicated by the content ID accepted by the accepting unit 131 to be superimposed on the display unit of the information terminal 2.

[0038] When the content information indicates that the content to be displayed cannot be displayed for the object attribute information corresponding to the object identification information, the display control unit 132 does not display the content to be displayed. When the content cannot be displayed, the display control unit 132, for example, causes the information terminal 2 to display a screen indicating that the content cannot be displayed. The display control unit 132 may also cause the information terminal 2 to display a screen indicating, from among the content owned by the user, content that can be displayed for the attribute of the object OB.

[0039] By configuring the information processing device 1 to control whether or not to display content depending on the attributes of an object, it is possible to control the display of content depending on the object OB that the user has caused the information terminal 2 to recognize.

[0040] The information processing device 1 may be configured to receive the specification of a plurality of pieces of content and determine the content to be displayed based on the attributes of the object OB.

[0041] The receiving unit 131 receives designation of multiple pieces of content to be displayed from among the content owned by the user. As an example, the display control unit 132 causes the information terminal 2 to display a screen configured so that multiple pieces of content to be displayed can be selected from the content owned by the user. The receiving unit 131 acquires content IDs of the multiple pieces of content selected on the displayed screen.

[0042] The display control unit 132 determines which content to display from among the plurality of pieces of content to be displayed, based on the object attribute information corresponding to the object identification information, and displays the determined content on the user's information terminal 2. The display control unit 132 refers to the content information and determines whether each of the plurality of content IDs acquired by the reception unit 131 can be displayed for the attribute of the object corresponding to the object identification information. The display control unit 132 displays, on the information terminal 2, the content corresponding to the content ID determined to be displayable among the plurality of content IDs acquired by the reception unit 131.

[0043] By configuring the information processing device 1 to display, among multiple contents, content that can be displayed according to the attributes of the object, it becomes possible to control the content that is displayed according to the location where the object OB is placed, which can motivate the user to move the content to a location where it can be displayed.

[0044] [Processing flow in information processing device 1] Fig. 6 is a flowchart showing the flow of processing in the information processing device 1. The flowchart shown in Fig. 6 starts when preparations are made to accept content designation.

[0045] The receiving unit 131 receives object identification information and content to be displayed (S01). The display control unit 132 refers to the object information stored in the storage unit 12, and identifies the attribute of the object OB corresponding to the acquired object identification information (S02).

[0046] The display control unit 132 refers to the content information stored in the storage unit 12, and determines whether or not the content can be displayed based on the content to be displayed and the attribute of the identified object OB (S03).

[0047] If the content can be displayed (YES in S03), the display control unit 132 causes the information terminal 2 to display the content (S04). Then, the information processing device 1 ends the process.

[0048] If the content cannot be displayed (NO in S03), the display control unit 132 causes the information terminal 2 to display a screen indicating that the content cannot be displayed (S05). Then, the information processing device 1 ends the process.

[0049] <Variation 1> In the above description, an example has been described in which it is determined whether or not content can be displayed based on the attributes of an object, but the information processing device 1 may be configured to determine the display mode of content depending on the attributes of an object.

[0050] The storage unit 12 further stores content information that associates a reference content, object attribute information, and corresponding content, which is content displayed in accordance with the reference content and the object attribute information. FIG. 7 is a diagram showing an example of the data structure of content information stored in the storage unit 12. In the content information shown in FIG. 7, "content," "attribute," and "corresponding content" are associated with each other. In this case, "content" indicates the content ID of the reference content. "Corresponding content" indicates the display mode of content that is displayed when the object attribute information matches the corresponding "attribute." As an example, if the "content" is an anime character, the "corresponding content" is content in which the character represented by the content is depicted in different costumes and poses for each corresponding attribute. In terms of "corresponding content," one piece of content may be associated with multiple corresponding pieces of content.

[0051] The display control unit 132 identifies object attribute information corresponding to the object identification information. The display control unit 132 refers to the object information stored in the storage unit 12, and identifies object attribute information corresponding to the object identification information accepted by the acceptance unit 131.

[0052] The display control unit 132 displays corresponding content associated in the content information with the specified content and the identified object attribute information on the user's information terminal 2. The display control unit 132 refers to the content information stored in the storage unit 12, and identifies the content ID accepted by the acceptance unit 131 and the corresponding content that corresponds to the identified object attribute information. The display control unit 132 displays the identified corresponding content on the information terminal 2.

[0053] By configuring the information processing device 1 in this way, it becomes possible to change the display mode of the content depending on the attributes of the object, which can motivate the user to move to locations where various objects OB are placed. <Variation 2>

[0054] The information processing device 1 may be configured to display content generated based on the selected content and the attribute of the object OB on the information terminal 2. Fig. 8 is a block diagram showing the configuration of an information processing device 1A according to Modification 2. The information processing device 1A according to Modification 2 further includes a generation unit 133.

[0055] In this case, the storage unit 12 stores a trained model that has been trained to output content obtained by modifying the specified content based on the specified content and a prompt written in a natural language.

[0056] The generation unit 133 generates content to be displayed on the user's information terminal 2 based on the specified content and object attribute information corresponding to the object identification information received by the reception unit 131. The generation unit 133 generates a prompt to be input to the trained model based on the specified content and the identified object attribute information. As an example, if the identified object attribute information is "mountain," the generation unit 133 generates a prompt stating, "Please modify the specified content to match 'mountain.'" The generation unit 133 inputs the generated prompt and the content specified by the user to the trained model and causes the trained model to output the modified content. The display control unit 132 causes the content generated by the generation unit 133 to be displayed on the user's information terminal 2.

[0057] By configuring the information processing device 1A according to the modified example in this way, it becomes possible to display content on the information terminal 2 in a flexible manner according to the attribute of the object OB.

[0058] <Variation 3> The information processing device 1 may be configured to display content selected in accordance with the external environment. The configuration of an information processing device 1B according to Modification 3 is shown in FIG. 9. In the information processing device 1B according to Modification 3, the control unit 13 further includes an acquisition unit 134. The control unit 13 functions as the acquisition unit 134 by executing a program stored in the storage unit 12.

[0059] In this case, the storage unit 12 stores display modes corresponding to the external environment (for example, the background of the content, the pose of the character that is the content, etc.). More specifically, the storage unit 12 further stores content information that associates a reference content, an external environment, and corresponding content that is content that is displayed corresponding to the reference content and the external environment. FIG. 10 is a diagram showing an example of the data structure of content information stored in the storage unit 12 in this case. The storage unit 12 further stores a display mode table that associates an external environment with a display mode of content that corresponds to the external environment.

[0060] The information processing device 1 further includes an acquisition unit 134 that acquires environmental information indicating the external environment around the information terminal 2 used by the user. The environmental information indicates, for example, the weather, temperature, humidity, etc. around the location of the information processing device 1. As an example, the environmental information acquisition unit 136 acquires location information indicating the location of the information terminal 2. The environmental information acquisition unit 136 transmits the location information of the information terminal 2 to an external server (not shown) and acquires environmental information corresponding to the location of the information terminal 2. The display control unit 132 determines the display mode of the content to be displayed on the user's information terminal 2 based on the external environment indicated by the environmental information, and displays the content to be displayed in the determined display mode. When the display control unit 132 determines that the content can be displayed based on the object attribute information, it specifies the display mode corresponding to the external environment indicated by the environmental information. The display control unit 132 displays the content on the information terminal 2 in the specified display mode.

[0061] More specifically, the display control unit 132 causes the information terminal 2 of the user to display the specified content and the corresponding content associated in the content information with the external environment indicated by the environmental information.

[0062] The information processing device 1B according to the modified example is configured to display content corresponding to the external environment on the information terminal 2, making it possible to change the content according to the external environment, thereby motivating users to visit the same location multiple times.

[0063] Furthermore, this invention will make it possible to contribute to Goal 9 of the United Nations' Sustainable Development Goals (SDGs), which is "Build resilient infrastructure, promote inclusive and sustainable industrialization, and promote innovation and resilience."

[0064] The present invention has been described above using embodiments, but the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. For example, all or part of the device can be configured by functionally or physically distributing or integrating any unit. Furthermore, new embodiments resulting from any combination of multiple embodiments are also included in the embodiments of the present invention. The effects of the new embodiments resulting from the combination also have the effects of the original embodiments. [Explanation of symbols]

[0065] 1. Information processing equipment 2. Information terminal 11 Communications Department 12 Storage section 13 Control Unit 131 Reception 132 Display control unit 133 Generation part 134 Acquisition Department

Claims

1. a storage unit that stores object identification information, which is information for identifying an object, and object attribute information, which indicates attributes of the object, in association with each other; a receiving unit that receives, from an information terminal used by the user, designation of content to be displayed from among content owned by the user and the object identification information; a display control unit that controls the content to be displayed to be superimposed on the information terminal based on the content to be displayed that has been received by the receiving unit and the object attribute information that corresponds to the object identification information that has been received by the receiving unit; and An information processing device having the above.

2. the storage unit further stores content information that associates content with object attribute information of an object that can display the content; the display control unit displays the content to be displayed when the content information indicates that the content to be displayed can be displayed, with respect to the object attribute information corresponding to the object identification information; The information processing device according to claim 1 .

3. the display control unit does not display the content to be displayed when the content information indicates that the content to be displayed cannot be displayed, with respect to the object attribute information corresponding to the object identification information. The information processing device according to claim 2 .

4. the receiving unit receives designation of a plurality of contents to be displayed from among the contents owned by the user; the display control unit determines a content to be displayed from among a plurality of contents to be displayed based on object attribute information corresponding to the object identification information, and causes the determined content to be displayed on the information terminal of the user. The information processing device according to claim 1 .

5. the storage unit further stores content information that associates a reference content, the object attribute information, and corresponding content that is content displayed in accordance with the reference content and the object attribute information; The display control unit (1) identifies object attribute information corresponding to the object identification information, and (2) displays, on the user's information terminal, the designated content and the corresponding content associated in the content information with the identified object attribute information. The information processing device according to claim 1 .

6. a generation unit that generates content to be displayed on the information terminal of the user based on the designated content and the object attribute information corresponding to the object identification information received by the reception unit; the display control unit causes the content generated by the generation unit to be displayed on the information terminal of the user; The information processing device according to claim 1 .

7. the storage unit further stores a display mode table that associates an external environment with a display mode of content corresponding to the external environment; an acquisition unit that acquires environmental information indicating an external environment around the information terminal used by the user; the display control unit determines a display mode of the content to be displayed on the information terminal of the user based on the external environment indicated by the environmental information, and displays the content to be displayed in the determined display mode. The information processing device according to claim 1 .

8. an acquisition unit that acquires environmental information indicating an external environment around the information terminal used by the user; the storage unit further stores content information associating a reference content, an external environment, and a corresponding content that is content displayed in accordance with the reference content and the external environment; the display control unit causes the information terminal of the user to display the designated content and the corresponding content associated in the content information with the external environment indicated by the environmental information. The information processing device according to claim 1 .

9. The computer executes receiving, from an information terminal used by the user, designation of content to be displayed from among content owned by the user and object identification information that is information for identifying an object; a step of referring to a storage unit that stores the object identification information and object attribute information indicating attributes of the object corresponding to the object identification information in association with each other, and controlling the content to be displayed to be superimposed on the information terminal based on the content to be displayed that has been received by the receiving unit and the object attribute information that corresponds to the object identification information that has been received by the receiving unit; An information processing method comprising:

10. On the computer, receiving, from an information terminal used by the user, designation of content to be displayed from among content owned by the user and object identification information that is information for identifying an object; a step of referring to a storage unit that stores the object identification information and object attribute information indicating attributes of the object corresponding to the object identification information in association with each other, and controlling the content to be displayed to be superimposed on the information terminal based on the content to be displayed that has been received by the receiving unit and the object attribute information that corresponds to the object identification information that has been received by the receiving unit; A program that executes the following.

Images