Augmented reality system

Abstract

To provide an advertisement in an augmented reality space.SOLUTION: An augmented reality system includes an object management section for disposing a content object in an augmented reality space, an advertisement management section for disposing an advertisement object displaying an advertisement in the augmented reality space, an image generation section for generating an image including the content object and advertisement object, and a posting section for posting an image on a social network.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to augmented reality systems. [Background technology]

[0002] Guide information in the form of augmented reality has been provided (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6100496 specification Summary of the Invention [Problem to be solved by the invention]

[0004] Advertisements cannot be displayed when using the augmented reality form of guide information.

[0005] The present invention has been made in consideration of the above background, and aims to provide a technique that can provide advertisements in an augmented reality space. [Means for solving the problem]

[0006] The main invention of the present invention for solving the above problem is an augmented reality system comprising an object management unit that places content objects in an augmented reality space, an advertising management unit that places advertising objects that display advertisements in the augmented reality space, an image generation unit that generates an image including the content objects and the advertising objects, and a posting unit that posts the image to a social network.

[0007] Other problems and solutions disclosed in this application will be made clear in the description of the preferred embodiments of the invention and the drawings. Effect of the Invention

[0008] According to the present invention, advertisements can be provided in an augmented reality space. [Brief description of the drawings]

[0009] [Figure 1] 1 is an overall configuration diagram of an augmented reality system 1 according to an embodiment of the present invention. [Diagram 2] 1 is a system configuration diagram of an augmented reality system 1 according to an embodiment of the present invention. [Diagram 3] FIG. 2 is a diagram showing a flow of providing augmented reality content in the augmented reality system of the present embodiment. [Figure 4] FIG. 1 is a diagram illustrating the display of augmented reality content. [Diagram 5] FIG. 11 is a diagram illustrating the flow of a posting process to an SNS. [Figure 6] FIG. 1 is a diagram illustrating posting of augmented reality content to a social networking site. [Figure 7] FIG. 11 is a diagram illustrating an example of the configuration of an augmented reality system according to a second embodiment. [Figure 8] FIG. 11 is a diagram illustrating the operation of the augmented reality system according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] <Summary of the Invention> The present invention will be described below with reference to the preferred embodiments thereof. [Item 1] an object management unit that arranges content objects in an augmented reality space; an advertisement management unit that arranges an advertisement object that displays an advertisement in the augmented reality space; an image generating unit that generates an image including the content object and the advertisement object; a posting unit for posting the image on a social network; An augmented reality system comprising: [Item 2] Item 1, an augmented reality system comprising: the content object is a character; The image generation unit synthesizes the image including the character and the advertisement object with an image of real space captured by a user; An augmented reality system featuring: [Item 3] Item 1, an augmented reality system comprising: Managing rights information indicating that a user has the right to post the image including the content and the advertisement object in a blockchain network; An augmented reality system featuring:

[0011] A specific example of an augmented reality system 1 according to an embodiment of the present invention will be described below with reference to the drawings. Note that the present invention is not limited to these examples, but is intended to include all modifications within the scope of the claims and meaning equivalent to the claims. In the following description, the same elements in the description of the drawings are given the same reference numerals, and duplicated descriptions will be omitted.

[0012] <First embodiment> Fig. 1 is an overall configuration diagram of an augmented reality system 1 according to this embodiment. Fig. 2 is a system configuration diagram of the augmented reality system 1 according to this embodiment. The augmented reality system 1 includes a server 2, which is connected to a user terminal 3 via the Internet 4 so as to be able to communicate bidirectionally. An SNS server 5 is also connected to the Internet 4 so as to be able to communicate.

[0013] The augmented reality system 1 of the present embodiment is intended to provide content (hereinafter, referred to as augmented reality content) to be displayed as so-called augmented reality (AR).

[0014] The server 2 is a computer that provides augmented reality content to the user terminal 3. The server 2 may be, for example, a general-purpose computer such as a workstation or a personal computer, or may be logically realized by cloud computing.

[0015] The server 2 is also connected to a blockchain network 6. The blockchain network 6 is a distributed ledger system constructed by a plurality of node computers. The blockchain network 6 makes it possible to manage the ledger in a manner that makes it difficult to tamper with.

[0016] The user terminal 3 is a computer operated by a user who wishes to view augmented reality content. The user terminal 3 may be, for example, an electronic terminal such as a smartphone, a tablet, or a notebook computer.

[0017] The SNS server 5 is a computer that provides a social network service and can, for example, accept text and image posts from users and share them with other users. The server 2 and the user terminal 3 can communicate with the SNS server 5, and users can post text, images, and the like to the SOS server 5 via the user terminal 3, for example.

[0018] <Server 2> The server 2 mainly comprises a server communication unit 21, a storage unit 22, a providing unit 23, and a rights management unit 24. The communication unit 21, the providing unit 23, and the rights management unit 24 can be realized, for example, by a CPU included in the server 2 executing a program stored in a storage. The storage unit 22 is realized as a part of a storage area provided by the storage included in the server 2. The storage can be, for example, a hard disk, a solid state disk, a flash memory, a RAM, a ROM, or the like.

[0019] The communication unit 21 communicates with external devices such as the customer's user terminal 3. The communication unit 21 can be, for example, a wireless communication means.

[0020] The storage unit 22 stores databases and programs in which various information is stored. For example, the storage unit 22 includes an advertisement DB 22a in which advertisement information used in augmented reality is stored. The advertisements used in augmented reality may include advertisements related to products and services. In addition, the forms of advertisements include visual information including still images or moving images, audio information, smell information, taste information, tactile information, and the like. The advertisement information related to visual information may be a still image or a moving image for displaying an advertisement, or may be text data. The advertisement information related to audio information may be audio data encoded in a format such as PCM or MP3, or may be text data for generating audio information by synthetic voice. In addition, the advertisement information includes advertisement attribute information directly related to the advertisement, and advertisement-related information indirectly related to the advertisement. In the advertisement attribute information, in the case of a product advertisement, for example, information such as the type, shape, size, features, price, mode, use, and effect of the product is included, and in the case of a service advertisement, for example, information such as the type, features, price, mode, and effect of the service is included. The advertisement-related information includes, for example, in the case of a product advertisement, information such as the place where the product is sold, and in the case of a service, information such as the place where the service can be received.

[0021] The providing unit 23 has a function of extracting advertisement information from the advertisement DB 22a and transmitting it to the user terminal 3. The providing unit 23 also has a function of acquiring various information from the user terminal 3 and extracting advertisements related to the acquired information.

[0022] The rights management unit 24 manages the conditions under which a user is permitted to post augmented reality content to a social network (SNS). In this embodiment, the conditions under which a user is permitted to post augmented reality content including the object to an SNS are managed in the distributed ledger of the blockchain network 6 for each object. In this embodiment, objects such as characters are permitted to be posted to an SNS on the condition that one or more advertisement objects are included, but other conditions may be registered.

[0023] <User terminal 3> The user terminal 3 mainly includes a communication unit 31, a camera 32, an input unit 33, a display unit 34, an object storage unit 35a, an object management unit 35b, an advertisement receiving unit 36, an advertisement management unit 37, an information output unit 38, and a position identification unit 39. The advertisement management unit 36, the advertisement response unit 37, the information output unit 38, and the position identification unit 39 can be realized by a CPU included in the user terminal 3 reading and executing a program stored in the storage. The object storage unit 35a can be realized, for example, as a part of the storage area provided by the storage. The storage can be, for example, a hard disk, a solid state disk, a flash memory, a RAM, a ROM, or the like.

[0024] The communication unit 31 is a part capable of communicating with an external device, and is, for example, a wireless communication means.

[0025] The camera 32 captures images of the real world. The camera 32 is a part that performs the shooting operation, and has an image sensor such as a CCD or CMOS device that converts an optical image into an electrical signal, and a shutter. The camera 32 can capture not only still images, but also moving images.

[0026] The input unit 33 can input data through a user's operation, such as a touch panel, a keyboard, etc. The input unit 33 can also input data not only by manual operation, but also by voice, for example.

[0027] The display unit 34 is, for example, a liquid crystal monitor, and is capable of displaying various types of information. The display unit 34 may be integrated with the input unit 33.

[0028] The object storage unit 35a stores model information necessary for generating virtual objects based on, for example, virtual characters, everyday items, and the like. The model information can be data in a format such as USDZ or glTF. The model information also includes, for example, attribute information directly related to the virtual object, and advertisement-related information corresponding to advertisement attribute information and advertisement-related information. The attribute information includes, for example, information on appearance, type, mode, component configuration, and the like.

[0029] The object management unit 35b creates augmented reality content. Specifically, the object management unit 35b creates an augmented reality image to be superimposed on an image of real space (an image captured by the camera 32). The object management unit 35b can create a virtual object based on model information stored in the object storage unit 35a and place it in the augmented reality space. The object management unit 35b can also place an advertisement object created by the advertisement management unit 36 ​​(described later) in the augmented reality space.

[0030] The advertisement management unit 37 applies the received advertisement information to an advertisement object suitable for the advertisement information. The advertisement object is a virtual object created by the object management unit 35b. The advertisement object may be, for example, a picture frame or a television, which is a virtual object originally used to display content on its surface, or may be a virtual object representing an item arranged around a virtual object that is a main content such as a character, such as a wall, a desk, or a door, or may be another virtual object that is owned by a virtual object that is a main content such as a character, such as a T-shirt or a cup. The advertisement management unit 37 can display an advertisement on the exterior of the advertisement object by, for example, attaching advertisement information of a still image or a moving image as a texture to the surface of the advertisement object. In addition, for example, when the advertisement information is audio information, the advertisement object can be attached with the audio information, and the audio information can also be played when the advertisement object is played. For example, when the advertisement management unit 37 receives advertisement information related to new clothes, the advertisement management unit 37 can attach the design of the new clothes to the surface of an advertisement object that has a motif of clothes. The advertisement management unit 37 can attach the advertisement information to an advertisement object with an attribute corresponding to the attribute information of the advertisement information. For example, when the advertising management unit 37 receives advertising information for a new song, since the advertising attribute information is music, it can be attached to an advertising object with a boombox motif that contains advertising information corresponding to the music.

[0031] The information output unit 38 outputs the augmented reality image created by the object management unit 35b by superimposing it on the image captured by the camera 32 of the real space. The information output unit 38 can also output visual information such as moving images, audio information, smell information, tactile information, taste information, etc. The information output unit can output visual information such as video via the display unit 34.

[0032] The position identification unit 39 has a function of acquiring the position information of the user terminal 3 by, for example, satellite positioning such as GPS or a beacon that performs positioning using a short-range wireless technology. The position identification unit 39 can also transmit the acquired position information to the server 2.

[0033] <Flowchart> Fig. 3 is a diagram showing a flow of providing augmented reality content in the augmented reality system of this embodiment Fig. 4 is a diagram for explaining the display of the augmented reality content.

[0034] First, in step S1, the user activates the augmented reality mode on the user terminal 3. The augmented reality mode is a state in which the camera 32 (image capture unit) of the user terminal 3 is activated, and a surrounding real space image P (real space information) captured by the camera 32, a virtual character O generated by the object management unit 35b, and a virtual drink D (virtual object) based on a drink motif are displayed on the display unit 34.

[0035] Next, in step S2, the user terminal 3 transmits the location information to the server 2.

[0036] Next, in step S3, the providing unit 23 of the server 2 extracts advertisements related to the location information from the advertisement DB 22a, and transmits them to the user terminal 3. In this embodiment, advertisements related to drinks sold at cafes within a predetermined range from the location of the user terminal 3 are extracted based on the received location information.

[0037] Next, in step S4, the advertisement management unit 37 generates an advertisement object C in association with an appropriate virtual drink D for the received advertisement.

[0038] Next, in step S5, the object management unit 35b controls the virtual character O to react to the advertisement object C. In this embodiment, the virtual character O holds up the advertisement object C, but this is not limited thereto. For example, the virtual character may apply an external force to the advertisement object, such as touching or hitting the advertisement object. The virtual character may also mention the advertisement object. For example, when an advertisement is received or an advertisement object is generated, the virtual character may output a voice to that effect. The virtual character may also be used according to its purpose. For example, if the advertisement object is a drink, the virtual character may make a gesture of drinking the advertisement object.

[0039] Next, in step S6, the user displays or plays back the content of the advertisement.

[0040] In this manner, the augmented reality system 1 in this embodiment can output advertisements to the user along with augmented reality content.

[0041] In the augmented reality system of this embodiment, the state in which the augmented reality content is displayed on the user terminal 3 can be posted to a social network (SNS). Fig. 5 is a diagram illustrating the flow of a posting process to an SNS.

[0042] When a user makes a request to the user terminal 3 to post to the SNS, the user terminal 3 transmits the request to post to the SNS to the server 2 (S41). In response to the request, the server 2 checks whether the user has the right to post augmented reality content to the SNS. In this embodiment, the rights regarding the user's augmented reality content are manually managed in the blockchain network 6, and the server 2 can determine whether the user is permitted to post by referring to the distributed ledger managed by the blockchain network 6 (S42). Here, the server 2 can determine whether or not the augmented reality content can be posted to the SNS, for example, according to the characters and advertisements included in the augmented reality content. For example, if the augmented reality content does not include advertisement content, the server 2 can disallow the posting to the SNS. In response to a request from the user terminal 3, an object included in the augmented reality content may be transmitted to the server 2, or a snapshot image of the augmented reality content may be transmitted from the user terminal 3 to the server 2, and the server 2 may analyze the snapshot image to determine whether or not an advertisement is displayed.

[0043] The server 2 responds with the determination result to the user terminal 3 (S43), and if posting is permitted, the user terminal 3 can transmit the augmented reality content to the SNS server 5 and post it (S44). The user terminal 3 can post, for example, a snapshot image of the augmented reality content to the SNS. Furthermore, the user terminal 3 may accept input of text or the like and attach it when posting. The SNS server 5 performs a process of sharing the posted content with other users (S45).

[0044] As described above, according to the augmented reality system 1 of the present embodiment, the user can legally post the augmented reality content being viewed on the user terminal 3 to the SNS. In the present embodiment, the posting of the augmented reality content to the SNS is permitted when certain conditions are met, such as when the augmented reality content contains an advertisement, according to the conditions managed by the blockchain network 6. Here, advertising fees from the advertiser can also be returned to the provider of the object. FIG. 6 is a diagram for explaining the posting of augmented reality content to an SNS. The user terminal 3 takes a snapshot of all or part of the augmented reality content displayed on the display unit 34, and posts the snapshot image 35 to the SNS. Here, the user can also aim the camera 32 of the user terminal 3 at himself so that the user U appears in the snapshot image 35 together with the character O. This allows a virtual photo-like image of the character O and the user U to be posted to the SNS. At this time, text 36 may be posted. The snapshot image 35 also includes a drink D, which is an advertising object, and this makes it possible to spread the advertisement on the SNS. In consideration of this advertisement, the user is given the right to post to the SNS, and it is also possible to return money to the provider of the character O.

[0045] <Second embodiment> 7 is a diagram showing a configuration example of an augmented reality system according to the second embodiment. In the second embodiment, the user terminal 3 omits the advertisement receiving unit 36 ​​and the advertisement management unit 37. The storage unit 22 of the server 2 includes an object DB 22b in addition to the advertisement DB 22a.

[0046] The object DB 22b stores model information of virtual objects to be placed in the augmented reality space (hereinafter, simply referred to as virtual objects). The model information may be data in a format such as USDZ or glTF.

[0047] In the second embodiment, the providing unit 23 provides augmented reality content. The providing unit 23 can provide the augmented reality content in response to a request from the user terminal 3, for example. A virtual object (hereinafter referred to as a content object) that the user wishes to view, such as a character, is specified in the request. The providing unit 23 reads out the content object specified in the request from the storage unit 22, and responds to the user terminal 3 with augmented reality content including the content object and scene information for arranging the content object in an augmented reality space. At this time, the providing unit 23 can respond to the user terminal 3 by including a virtual object (hereinafter referred to as an advertisement object) for displaying an advertisement in the augmented reality content. The providing unit 23 can paste an advertisement image, for example, as a texture for the advertisement object.

[0048] FIG. 8 is a diagram illustrating the operation of the augmented reality system according to the second embodiment.

[0049] In step S2, the user terminal 3 transmits a request for augmented reality content to the server 2. The request includes a specification of a virtual object, such as a character, that the user wishes to view.

[0050] In step S3, the server 2 creates augmented reality content in response to the request. The server 2 can create the augmented reality content by, for example, reading model information corresponding to a virtual object (hereinafter referred to as a content object) such as a character specified in the request from the object DB 22b. The server 2 can also generate an object not specified by the user, such as a background set or a small item, as an advertisement object, and include the advertisement information in the texture of the advertisement object before including it in the augmented reality content. The server 2 can also include scene information specifying the placement positions and movements of the content object and advertisement object in the augmented reality content.

[0051] In step S4, the user terminal 3 receives the augmented reality content and is able to create an augmented reality scene based on the received augmented reality content.

[0052] As described above, the augmented reality system according to the second embodiment can provide an advertisement object in association with a content object that the user is about to view, and therefore can place an advertisement in the same augmented reality space as the content object in which the user is interested.

[0053] In the second embodiment, the user terminal 3 can also post the augmented reality content to the SNS. In this case, in the second embodiment, a snapshot image may be created in the server 2.

[0054] Although the embodiment of the present invention has been described above, the aspects of the present invention are not limited to this embodiment.

[0055] <Modification> In the above embodiment, the user plays back the advertisement by operating the input unit 33, but this is not limited to the above. For example, the advertisement management unit 36 ​​may have a function of detecting keywords from voice information or text information, and may have a function of displaying or playing the advertisement when a keyword related to advertisement information is detected during a voice or text conversation between a virtual character and a user.

[0056] In the above embodiment, the advertisement information and the advertisement object are associated with each other, but this is not limited thereto. For example, the object management unit 35 of the user terminal 3 may have a function of directly converting the advertisement information into an advertisement object (virtual object). For example, the object management unit may create an advertisement object including attribute information that is the same as, similar to, or corresponds to the advertisement attribute information from the object storage unit based on the advertisement attribute information and advertisement-related information of the received advertisement information.

[0057] In the above embodiment, the augmented reality mode is a state in which the surrounding real space image P (information on the real space) captured by the camera 32 and the virtual character O (virtual object) generated by the object management unit 35b are displayed on the display unit 34, but is not limited to this. For example, it may be a route plan service (for example, Live View, which is a Google map service) using augmented reality that recognizes the surrounding real space (buildings, roads, objects, etc.) captured by the camera based on the position information of the user terminal and the direction in which the user terminal is facing.

[0058] In the above embodiment, the user terminal 3 receives an advertisement when the augmented reality mode is activated, but this is not limited to the above. For example, when the surrounding environment changes, the user terminal 3 may automatically receive (update) an advertisement related to the surrounding environment.

[0059] For example, the user terminal 3 may be provided with an advertisement update unit (not shown) that updates advertisements in response to changes in the user's location. In detail, the advertisement update unit has a function of detecting a change in the user's location due to, for example, moving, and causing the advertisement management unit 36 ​​to acquire a new advertisement.

[0060] In the above embodiment, the advertisement information is displayed as video, but it is not limited to this and may be audio information.

[0061] Moreover, the advertisement and the real space environment in the above embodiment may be odor information. For example, the information output unit 38 of the user terminal 3 includes at least one odor element, and the information output unit 38 can emit one or more odor elements. The odor element may be, for example, a single chemical substance such as furaneol (strawberry odor) or 1-hexanol (forest odor), or a combination of these. Also, for example, an odor having an equivalent odor to the odor of jasmine as one unit may be used, regardless of the composition of the chemical substance. That is, the information output unit 38 may output, for example, an equivalent odor by combining a plurality of chemical substances using the odor of strawberry milk as an output method, or may output a combination of an odor having an odor of strawberry as one unit and an odor having an odor of milk as one unit.

[0062] Moreover, the advertisement and the real-space environment in the above embodiment may be taste information. For example, the information output unit 38 of the user terminal 3 includes at least one taste element, and the information output unit 38 can release one or more taste elements. The taste element may be, for example, a single chemical substance such as inosinic acid or glutamic acid, or a combination of these. Also, for example, a taste equivalent to strawberry flavor may be used as one unit, regardless of the composition of the chemical substance. That is, the information output unit 38 may output, for example, strawberry chocolate taste information by combining a plurality of chemical substances to output an equivalent taste, or may output a combination of a taste with strawberry flavor as one unit and a taste with chocolate flavor as one unit.

[0063] Moreover, the advertisement and the real space environment in the above embodiment may be tactile information.

[0064] In addition, in this embodiment, a virtual space using augmented reality (AR) is assumed, but this is not limited to this, and the advertising system of this embodiment can be applied as is to a virtual reality (VR) space or a mixed reality (MR) space. [Explanation of symbols]

[0065] 1. Advertising System 2 Server 3. User terminal 4. Communication Network 5. SOS Server 6. Blockchain Network 21 Server Communication Department 22 Memory section 22a Advertising DB 23 Providing Department 31 Terminal communication unit 32 Camera 33 Input section 34 Display section 35a Object Storage 35b Object Management Section 36 Advertising Receiving Unit 37 Advertising Management Department 38 Information output section 39 Location identification part

Claims

1. an object management unit that arranges content objects in an augmented reality space; an advertisement management unit that arranges an advertisement object that displays an advertisement in the augmented reality space; an image generating unit that generates an image including the content object and the advertisement object; a posting unit for posting the image on a social network; An augmented reality system comprising:

2. 2. The augmented reality system of claim 1, the content object is a character; The image generation unit synthesizes the image including the character and the advertisement object with an image of real space captured by a user; An augmented reality system that features:

3. 2. The augmented reality system of claim 1, Managing rights information indicating that a user has the right to post the content and the image including the advertisement object in a blockchain network; An augmented reality system that features:

Images