Augmented reality system

Abstract

To provide an advertisement in an augmented reality space.SOLUTION: An augmented reality system includes an object management section for disposing a content object in an augmented reality space, an advertisement management section for disposing an advertisement object displaying an advertisement in the augmented reality space, an image generation section for generating an image including the content object and advertisement object, and a posting section for posting an image on a social network.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an augmented reality system.

Background Art

[0002] There has been an attempt to provide guide information in an augmented reality form (see Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When using guide information in an augmented reality form, it is not possible to display an advertisement.

[0005] The present invention has been made in view of such a background, and an object thereof is to provide a technology capable of providing an advertisement in an augmented reality space.

Means for Solving the Problems

[0006] The main invention of the present invention for solving the above problems is an augmented reality system, comprising: an object management unit that arranges content objects in an augmented reality space; an advertisement management unit that arranges advertisement objects for displaying advertisements in the augmented reality space; an image generation unit that generates an image including the content objects and the advertisement objects; and a posting unit that posts the image on a social network.

[0007] Regarding other problems disclosed in the present application and methods for solving them, they will be clarified by the embodiments of the invention and the drawings.

Effects of the Invention

[0008] According to the present invention, advertisements can be provided in augmented reality space. [Brief explanation of the drawing]

[0009] [Figure 1] This is an overall configuration diagram of the augmented reality system 1 according to this embodiment. [Figure 2] This is a system configuration diagram of the augmented reality system 1 according to this embodiment. [Figure 3] This diagram shows the flow of providing augmented reality content in the augmented reality system of this embodiment. [Figure 4] This is a diagram illustrating the display of augmented reality content. [Figure 5] This diagram illustrates the process of posting to social media. [Figure 6] This is a diagram illustrating the posting of augmented reality content to social media. [Figure 7] This figure shows an example configuration of an augmented reality system according to the second embodiment. [Figure 8] This is a diagram illustrating the operation of the augmented reality system according to the second embodiment. [Modes for carrying out the invention]

[0010] <Summary of the Invention> The embodiments of the present invention will be described by listing them. The present invention has, for example, the following configuration. [Item 1] An object management unit that places content objects in an augmented reality space, An advertising management unit that places advertising objects to display advertisements in the augmented reality space, An image generation unit that generates an image including the content object and the advertisement object, A posting unit that posts the aforementioned image to a social network, An augmented reality system characterized by having the following features. [Item 2] The augmented reality system described in item 1, The content object is a character, The image generation unit synthesizes the image including the character and the advertisement object with the image of the real space photographed by the user, An augmented reality system characterized by the above. [Item 3] The augmented reality system according to Item 1, Managing right information indicating that the user has the right to post the image including the content and the advertisement object in the blockchain network, An augmented reality system characterized by the above.

[0011] A specific example of the augmented reality system 1 according to an embodiment of the present invention will be described below with reference to the drawings. Note that the present invention is not limited to these examples, and is shown by the claims, and is intended to include all modifications within the meaning and scope equivalent to the claims. In the following description, the same elements in the description of the drawings are denoted by the same reference numerals, and duplicate descriptions are omitted.

[0012] <First Embodiment> FIG. 1 is an overall configuration diagram of the augmented reality system 1 according to the present embodiment. FIG. 2 is a system configuration diagram of the augmented reality system 1 according to the present embodiment. The augmented reality system 1 includes a server 2, and the server 2 is connected to a user terminal 3 via the Internet 4 so as to be capable of two-way communication. An SNS server 5 is also communicably connected to the Internet 4.

[0013] The augmented reality system 1 of the present embodiment is intended to provide content (hereinafter referred to as augmented reality content) for display as so-called augmented reality (AR; Augumented Reality).

[0014] Server 2 is a computer that provides augmented reality content to user terminal 3. Server 2 may be a general-purpose computer such as a workstation or a personal computer, or may be logically realized by cloud computing.

[0015] Server 2 is also connected to blockchain network 6. Blockchain network 6 is a distributed ledger system constructed by a plurality of node computers. With blockchain network 6, the ledger can be managed with difficulty of tampering.

[0016] User terminal 3 is a computer operated by a user who attempts to view augmented reality content. User terminal 3 can be an electronic terminal such as a smartphone, a tablet, or a notebook personal computer.

[0017] SNS server 5 is a computer that provides a social network service. For example, it can receive posts of texts and images from users and share them with other users. Server 2 and user terminal 3 can communicate with SNS server 5 with each other, and the user can post texts, images, etc. to SNS server 5 via user terminal 3, for example.

[0018] <Server 2> Server 2 mainly includes a server communication unit 21, a storage unit 22, a providing unit 23, and a rights management unit 24. Communication unit 21, providing unit 23, and rights management unit 24 can be realized by, for example, a CPU included in server 2 executing a program stored in a storage. Also, storage unit 22 is realized as a part of the storage area provided by the storage included in server 2. The storage can be, for example, a hard disk, a solid state disk, a flash memory, a RAM, a ROM, etc.

[0019] Communication unit 21 communicates with external devices such as customer's user terminal 3. Communication unit 21 can be, for example, wireless communication means.

[0020] The memory unit 22 stores databases and programs containing various types of information. For example, the memory unit 22 includes an advertising DB 22a that stores advertising information used in augmented reality. Advertising used in augmented reality may include advertisements for goods and services. The form of the advertisement may include visual information, such as still images or videos, audio information, smell information, taste information, and tactile information. Advertising information related to visual information may be in the form of still images or videos for displaying the advertisement, or as text data. Advertising information related to audio information may be, for example, audio data encoded in a format such as PCM or MP3, or as text data for generating audio information using synthesized speech. Advertising information also includes advertising attribute information directly related to the advertisement and advertising-related information indirectly related to the advertisement. In the case of product advertisements, advertising attribute information may include, for example, information such as the type, shape, size, features, price, form, use, and effect of the product, while in the case of service advertisements, it may include, for example, information such as the type, features, price, form, and effect of the service. Furthermore, advertising-related information includes, for example, information such as the sales location of a product in the case of a product advertisement, and information such as the location where the service can be received in the case of a service advertisement.

[0021] The provisioning unit 23 has the function of extracting advertising information from the advertising database 22a and sending it to the user terminal 3. In addition, the provisioning unit 23 can acquire various information from the user terminal 3 and has the function of extracting advertisements related to the acquired information.

[0022] The rights management unit 24 manages the conditions under which users are permitted to post augmented reality content to social networks (SNS). In this embodiment, for each object, the conditions under which users are permitted to post augmented reality content containing that object to SNS are managed in the distributed ledger of the blockchain network 6. In this embodiment, objects such as characters are permitted to be posted to SNS on the condition that they contain one or more advertising objects, but other conditions may also be registered.

[0023] <User Terminal 3> The user terminal 3 mainly comprises a communication unit 31, a camera 32, an input unit 33, a display unit 34, an object storage unit 35a, an object management unit 35b, an advertisement receiving unit 36, an advertisement management unit 37, an information output unit 38, and a location identification unit 39. The advertisement management unit 36, the advertisement handling unit 37, the information output unit 38, and the location identification unit 39 can be implemented by the CPU of the user terminal 3 reading and executing a program stored in storage. The object storage unit 35a can be implemented, for example, as part of the storage area provided by the storage. The storage can be, for example, a hard disk, a solid-state disk, flash memory, RAM, ROM, etc.

[0024] The communication unit 31 is a part that can communicate with external devices, such as wireless communication means.

[0025] Camera 32 captures images of the real world. Camera 32 is the component that performs shooting operations and includes an image sensor such as a CCD or CMOS element that converts optical images into electrical signals, as well as a shutter. It can capture not only still images but also moving images.

[0026] The input unit 33 can receive input through user operation, such as a touch panel or keyboard. Furthermore, the input unit 33 may accept input not only through manual operation but also, for example, through voice input.

[0027] The display unit 34 is, for example, a liquid crystal monitor and is capable of displaying various types of information. Alternatively, the display unit 34 may be integrated with the input unit 33.

[0028] The object storage unit 35a stores model information necessary to generate virtual objects, such as virtual characters or everyday items. The model information can be in data format such as USDZ or glTF. The model information also includes, for example, attribute information directly related to the virtual object, and advertising-related information corresponding to advertising attribute information and advertising-related information. The attribute information includes, for example, information such as appearance, type, form, and component configuration.

[0029] The object management unit 35b creates augmented reality content. Specifically, the object management unit 35b creates augmented reality images to be superimposed on images of the real world (images captured by the camera 32). The object management unit 35b can create virtual objects based on model information stored in the object storage unit 35a and place them in the augmented reality space. The object management unit 35b can also place advertising objects created by the advertising management unit 36, which will be described later, in the augmented reality space.

[0030] The advertising management unit 37 applies the received advertising information to an advertising object suitable for that advertising information. The advertising object is a virtual object created by the object management unit 35b. The advertising object may be a virtual object originally intended to display content on its surface, such as a picture frame or a television; it may be a virtual object representing an item placed around a virtual object that is the main content, such as a character, such as a wall, a desk, or a door; or it may be another virtual object possessed by the virtual object that is the main content, such as a character, such as a T-shirt or a cup. The advertising management unit 37 can display an advertisement on the appearance of an advertising object by, for example, attaching advertising information of a still image or a moving image as a texture to the surface of the advertising object. Also, for example, if the advertising information is audio information, it can attach the audio information to the advertising object so that the audio information is also played when the advertising object is played. For example, if the advertising management unit 37 receives advertising information about new clothing, it can attach the design of the new clothing to the surface of an advertising object that is based on clothing. The advertising management unit 37 can attach the advertising information to advertising objects with attributes that correspond to the attribute information of the advertising information. For example, when the advertising management unit 37 receives advertising information for a new song, since the advertising attribute information is music, it can attach it to an advertising object modeled after a boombox that contains advertising information corresponding to the music.

[0031] The information output unit 38 superimposes the augmented reality image created by the object management unit 35b onto the image captured by the camera 32 in the real space and outputs it. The information output unit 38 can also output visual information such as moving images, audio information, smell information, tactile information, taste information, etc. The information output unit can output visual information such as video via the display unit 34.

[0032] The location determination unit 39 has the function of acquiring location information of the user terminal 3, for example, by using satellite positioning such as GPS or beacons that use short-range wireless technology for positioning. The location determination unit 39 can also transmit the acquired location information to the server 2.

[0033] <Flowchart> Figure 3 is a diagram illustrating the flow of providing augmented reality content in the augmented reality system of this embodiment. Figure 4 is a diagram illustrating the display of augmented reality content.

[0034] First, in step S1, the user activates augmented reality mode on user terminal 3. Augmented reality mode is a state in which the camera 32 (shooting unit) of user terminal 3 is activated, and the surrounding real-space image P (information of the real space) captured by the camera 32, along with a virtual character O and a virtual drink D (virtual object) based on a drink, generated by the object management unit 35b, are displayed on the display unit 34.

[0035] Next, in step S2, the user terminal 3 transmits location information to the server 2.

[0036] Next, in step S3, the server 2's provision unit 23 extracts location-related advertisements from the advertisement DB 22a and sends them to the user terminal 3. In this embodiment, based on the received location information, advertisements for drinks sold at cafes within a predetermined range from the user terminal 3's location are extracted.

[0037] Next, in step S4, the advertising management unit 37 generates an advertising object C by associating the received advertisement with a suitable virtual drink D.

[0038] Next, in step S5, the object management unit 35b controls the virtual character O to react to the advertising object C. In this embodiment, the virtual character O is lifting the advertising object C, but is not limited to this. For example, the virtual character may touch or hit the advertising object, or apply other external force to the advertising object. The virtual character may also make comments about the advertising object. For example, when an advertisement is received or an advertising object is created, the virtual character may make an audio output to that effect. The virtual character may also be used according to its purpose. For example, if the advertising object is a drink, the virtual character may make a drinking gesture towards the advertising object.

[0039] Next, in step S6, the user views or plays the advertisement content.

[0040] Thus, the augmented reality system 1 in this embodiment can output advertisements to the user along with augmented reality content.

[0041] In the augmented reality system of this embodiment, the augmented reality content displayed on the user terminal 3 can be posted to a social network (SNS). Figure 5 is a diagram illustrating the flow of the posting process to the SNS.

[0042] When a user requests a post to SNS from user terminal 3, user terminal 3 sends the request to server 2 (S41). Server 2 checks whether the user has the right to post augmented reality content to SNS in response to the request. In this embodiment, the rights of users regarding augmented reality content are managed in the blockchain network 6, and server 2 can refer to the distributed ledger managed by the blockchain network 6 to determine whether the user is permitted to post (S42). Here, server 2 can determine whether posting to SNS is permitted, for example, based on the characters and advertisements included in the augmented reality content. For example, if the augmented reality content does not contain advertising content, server 2 can deny permission to post to SNS. Alternatively, the user terminal 3 may send objects included in the augmented reality content to server 2 in response to the request, or the user terminal 3 may send a snapshot image of the augmented reality content to server 2, and server 2 may analyze the snapshot image to determine whether or not advertisements are displayed.

[0043] Server 2 responds to user terminal 3 with the determination result (S43). If posting is permitted, user terminal 3 can send the augmented reality content to SNS server 5 and post it (S44). User terminal 3 can, for example, post a snapshot image of the augmented reality content to SNS. User terminal 3 may also accept and add text or other input when posting. SNS server 5 processes the posted content to share it with other users (S45).

[0044] As described above, with the augmented reality system 1 of this embodiment, users can lawfully post augmented reality content they are viewing on their user terminal 3 to social networking services (SNS). In this embodiment, posting of augmented reality content to SNS is permitted when certain conditions are met, such as when the augmented reality content includes advertisements, depending on the conditions managed by the blockchain network 6. Here, advertising fees from advertisers can also be returned to the object providers. Figure 6 illustrates the posting of augmented reality content to social media. The user terminal 3 captures a snapshot of all or part of the augmented reality content displayed on the display unit 34, and this snapshot image 35 is posted to social media. The user can also point the camera 32 of the user terminal 3 towards themselves so that user U and character O appear in the snapshot image 35. This allows for the posting of a virtually photographic image of character O and user U to social media. Text 36 may also be included in the post. The snapshot image 35 also includes the advertising object, drink D, enabling the dissemination of the advertisement to social media. In exchange for this advertisement, the user is granted the right to post to social media, and the provider of character O can also receive monetary compensation.

[0045] <Second Embodiment> Figure 7 shows an example configuration of an augmented reality system according to the second embodiment. In the second embodiment, the user terminal 3 omits the advertisement receiving unit 36 ​​and the advertisement management unit 37. In addition, the storage unit 22 of the server 2 includes an object DB 22b in addition to the advertisement DB 22a.

[0046] Object DB22b stores model information (hereinafter also simply referred to as virtual objects) for virtual objects placed in augmented reality space. Model information can be in data format such as USDZ or glTF.

[0047] In the second embodiment, the providing unit 23 provides augmented reality content. The providing unit 23 can provide augmented reality content, for example, in response to a request from a user terminal 3. The request specifies a virtual object (hereinafter referred to as a content object) that the user wishes to view, such as a character. The providing unit 23 reads the content object specified in the request from the storage unit 22 and responds to the user terminal 3 with augmented reality content including the content object and scene information for placing the content object in augmented reality space. At this time, the providing unit 23 can include a virtual object for displaying an advertisement (hereinafter referred to as an advertisement object) in the augmented reality content when responding to the user terminal 3. The providing unit 23 can, for example, paste an advertisement image as the texture of the advertisement object.

[0048] Figure 8 is a diagram illustrating the operation of the augmented reality system according to the second embodiment.

[0049] In step S2, user terminal 3 sends a request for augmented reality content to server 2. The request includes specifying the virtual object, such as a character, that the user wants to view.

[0050] In step S3, Server 2 creates augmented reality content in response to a request. For example, Server 2 can read model information corresponding to virtual objects such as characters specified in the request (hereinafter referred to as content objects) from object DB22b and create augmented reality content. Server 2 can also generate objects not specified by the user, such as background sets and small objects, as advertising objects, and include advertising information in the augmented reality content after attaching it to the texture of the advertising objects. Furthermore, Server 2 can include scene information that specifies the placement and movement of content objects and advertising objects in the augmented reality content.

[0051] In step S4, the user terminal 3 receives augmented reality content and can create an augmented reality scene based on the received augmented reality content.

[0052] As described above, the augmented reality system according to the second embodiment can provide advertising objects in conjunction with content objects that the user intends to view. Therefore, advertisements can be placed in the same augmented reality space as the content objects that the user is interested in.

[0053] In the second embodiment as well, the user terminal 3 can post augmented reality content to the social networking service (SNS). In this case, the server 2 may create a snapshot image.

[0054] This concludes the description of embodiments of the present invention, but the embodiments of the present invention are not limited to these embodiments.

[0055] <Variation> In the above embodiment, the user plays the advertisement by operating the input unit 33, but this is not limited to this. For example, the advertisement management unit 36 ​​may have a function to detect keywords from audio information or text information, and when the advertisement management unit 36 ​​detects keywords related to the advertisement information during an audio or text conversation between a virtual character and the user, it may have a function to display or play the advertisement.

[0056] Furthermore, while the above embodiment associates advertising information with advertising objects, it is not limited to this. For example, the object management unit 35 of the user terminal 3 may have a function to directly convert advertising information into advertising objects (virtual objects). For example, the object management unit may create an advertising object from the object storage unit that includes attribute information that is identical, similar, or corresponding to the advertising attribute information of the received advertising information, based on the advertising attribute information and advertising-related information of the received advertising information.

[0057] Furthermore, in the above embodiment, the augmented reality mode is defined as a state in which the surrounding real-space image P (information of the real space) captured by the camera 32 and the virtual character O (virtual object) generated by the object management unit 35b are displayed on the display unit 34, but is not limited to this. For example, it may be an augmented reality route guidance service (for example, Google Maps Live View) that recognizes the surrounding real space (buildings, roads, objects, etc.) captured by the camera based on the location information of the user terminal and the direction the user terminal is facing.

[0058] Furthermore, in the above embodiment, the user terminal 3 receives advertisements when it activates the augmented reality mode, but this is not limited to this. For example, the user terminal 3 may automatically receive (update) advertisements related to the surrounding environment when the surrounding environment changes.

[0059] For example, the user terminal 3 may be equipped with an ad update unit (not shown) that updates ads in response to changes in the user's location. More specifically, the ad update unit has a function that detects when the user moves or otherwise changes location and causes the ad management unit 36 ​​to acquire new ads.

[0060] In the above embodiment, the advertising information is displayed as video, but it is not limited to video; it may also be audio information.

[0061] Furthermore, the advertisement and the environment in the real space in the above embodiment may also be scent information. For example, the information output unit 38 of the user terminal 3 contains at least one scent element, and the information output unit 38 can release one or more scent elements. The scent element may be a single chemical substance such as furaneol (strawberry scent) or 1-hexanol (forest scent), or a combination thereof. Alternatively, it may be a scent with a unit equivalent to the scent of jasmine, regardless of the composition of the chemical substances. That is, the information output unit 38 may, for example, output the scent of strawberry milk as an output method by combining multiple chemical substances to produce an equivalent scent, or it may output a combination of a scent with a unit of strawberry scent and a scent with a unit of milk scent.

[0062] Furthermore, the advertisement and the environment in the real space in the above embodiment may also be taste information. For example, the information output unit 38 of the user terminal 3 contains at least one taste component, and the information output unit 38 can release one or more taste components. The taste component may be a single chemical substance such as inosinic acid or glutamic acid, or a combination thereof. Alternatively, it may be a taste with a unit equivalent to strawberry flavor, regardless of the composition of the chemical substances. That is, the information output unit 38 may, for example, output a taste equivalent to strawberry chocolate by combining multiple chemical substances, or it may output a combination of a taste with strawberry flavor as one unit and a taste with chocolate flavor as one unit.

[0063] Furthermore, the advertisement and the real-world environment in the above embodiment may also be tactile information.

[0064] Furthermore, while this embodiment assumes a virtual space using augmented reality (AR), the advertising system of this embodiment is not limited to this and can be applied as is to virtual reality (VR) spaces or mixed reality (MR) spaces. [Explanation of symbols]

[0065] 1. Advertising System 2 servers 3. User terminals 4. Communication Network 5 SOS Servers 6 Blockchain Network 21 Server Communication Section 22 Memory section 22a Advertising Database 23 Providing Department 31 Terminal Communication Unit 32 cameras 33 Input section 34 Display section 35a Object Storage Unit 35b Object Management Department 36 Advertising Receiving Section 37 Advertising Management Department 38 Information Output Unit 39 Location identification part

Claims

1. An advertising system including a server, An object management unit that places character objects in a virtual space, as well as ad objects that display advertisements, An image generation unit that generates an image including the character object and the advertisement object, A rights management unit that, by referring to a distributed ledger of a blockchain network that manages rights information indicating that a user has the right to post the image, including the character object and the advertising object, to a social network, permits the posting of the image to the social network if the advertising object is included in the image. A posting unit that posts the image to the social network if permission to post the image is granted, Equipped with, The object management unit, the image generation unit, the rights management unit, and the posting unit are implemented by the server's computer executing a program. An advertising system characterized by the following:

2. The advertising system according to claim 1, The object management unit places the character object specified by the user into the virtual space. An advertising system characterized by the following:

Images