Augmented reality system

Abstract

To provide an advertisement in an augmented reality system.SOLUTION: An augmented reality system includes an advertisement acquisition section for acquiring advertisement information according to a user's attribute viewing an object disposed in an augmented reality space, and an object management section of disposing the object in an augmented space and also dispose the advertisement information therein.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to augmented reality systems. [Background technology]

[0002] Guide information in the form of augmented reality has been provided (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6100496 specification Summary of the Invention [Problem to be solved by the invention]

[0004] Advertisements cannot be displayed when using the augmented reality form of guide information.

[0005] The present invention has been made in consideration of the above background, and aims to provide a technique that can provide advertisements in an augmented reality space. [Means for solving the problem]

[0006] The main invention of the present invention for solving the above problem is an augmented reality system, characterized in that it comprises an advertising acquisition unit that acquires advertising information corresponding to user attributes of a user viewing an object placed in an augmented reality space, and an object management unit that places the object in the augmented space and also places the advertising information.

[0007] Other problems and solutions disclosed in this application will be made clear in the description of the preferred embodiments of the invention and the drawings. Effect of the Invention

[0008] According to the present invention, advertisements can be provided in an augmented reality space. [Brief description of the drawings]

[0009] [Figure 1] 1 is an overall configuration diagram of an augmented reality system 1 according to an embodiment of the present invention. [Diagram 2] 1 is a system configuration diagram of an augmented reality system 1 according to an embodiment of the present invention. [Diagram 3] 13 is a flowchart showing a process in which the user terminal 3 receives an advertisement. [Figure 4] FIG. 2 is an explanatory diagram relating to the display of augmented reality content in the augmented reality system according to the present embodiment. [Diagram 5] FIG. 2 is an explanatory diagram regarding interaction between a virtual object and an advertisement in the augmented reality system according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] <Summary of the Invention> The present invention will be described below with reference to the preferred embodiments thereof. [Item 1] an advertisement acquisition unit that acquires advertisement information according to attributes of a user who views an object arranged in an augmented reality space; an object management unit that arranges the object in the extended space and arranges the advertisement information; An augmented reality system comprising: [Item 2] Item 1, an augmented reality system comprising: a purchase processing unit that purchases a product or service in the augmented reality space, the advertisement acquisition unit acquires the advertisement information according to a purchase history of the product or service; An augmented reality system featuring: [Item 3] Item 1, an augmented reality system comprising: A behavior history acquisition unit that acquires behavior history information of the user in the augmented reality space, the advertisement acquisition unit acquires the advertisement information according to the behavior history; An augmented reality system featuring: [Item 4] Item 1, an augmented reality system comprising: the object management unit arranges a plurality of the objects in the augmented reality space; A browsing detection unit that identifies the object that the user has browsed, the advertisement acquisition unit acquires the advertisement information according to a browsing history of the object; An augmented reality system featuring: [Item 5] Item 1, an augmented reality system comprising: a viewing detection unit that identifies a location in the augmented reality space viewed by the user; the advertisement acquisition unit acquires the advertisement information according to a browsing history of the location; An augmented reality system featuring:

[0011] <System Overview> Specific examples of an augmented reality system according to an embodiment of the present invention will be described below with reference to the drawings. Note that the present invention is not limited to these examples, but is intended to include all modifications within the scope of the claims and meaning equivalent thereto. In the following description, the same elements in the description of the drawings will be given the same reference numerals, and duplicated descriptions will be omitted.

[0012] <First embodiment> Fig. 1 is an overall configuration diagram of an augmented reality system 1 according to this embodiment. Fig. 2 is a system configuration diagram of the augmented reality system 1 according to this embodiment. The augmented reality system 1 includes a server 2, and the server 2 is connected to a user terminal 3 via the Internet 4 so as to be able to communicate bidirectionally.

[0013] The augmented reality content providing system 1 of this embodiment is intended to provide content (hereinafter, referred to as augmented reality content) to be displayed as so-called augmented reality (AR).

[0014] The server 2 is a computer that provides augmented reality content to the user terminal 3. The server 2 may be, for example, a general-purpose computer such as a workstation or a personal computer, or may be logically realized by cloud computing.

[0015] The user terminal 3 is a computer operated by a user who wishes to view augmented reality content. The user terminal 3 may be, for example, an electronic terminal such as a smartphone, a tablet, or a notebook computer.

[0016] <Server 2> The server 2 mainly includes a communication unit 21, a storage unit 22, a provision unit 23, a behavioral history acquisition unit 24, and a user attribute determination unit 25. The communication unit 21, the provision unit 23, the behavioral history acquisition unit 24, and the user attribute determination unit 25 can be realized, for example, by a CPU included in the server 2 executing a program stored in the storage. The storage unit 22 is realized as a part of the storage area provided by the storage included in the server 2. The storage can be, for example, a hard disk, a solid state disk, a flash memory, a RAM, a ROM, or the like.

[0017] The communication unit 21 communicates with external devices such as the customer's user terminal 3. The communication unit 21 can be, for example, a wireless communication means.

[0018] The storage unit 22 stores databases in which various information is stored, programs, and the like.

[0019] For example, the storage unit 22 includes an advertisement DB 22a in which advertisement information used in the augmented reality is stored. The advertisements used in the augmented reality may include advertisements related to products and services. In addition, the forms of advertisements include visual information including still images or moving images, audio information, smell information, taste information, tactile information, and the like. The advertisement information related to visual information may be a still image or a moving image for displaying an advertisement, or may be text data. The advertisement information related to audio information may be audio data encoded in a format such as PCM or MP3, or may be text data for generating audio information by synthetic voice. In addition, the advertisement information includes advertisement attribute information directly related to the advertisement, and advertisement-related information indirectly related to the advertisement. In the advertisement attribute information, in the case of a product advertisement, for example, information such as the type, shape, size, features, price, mode, use, and effect of the product is included, and in the case of a service advertisement, for example, information such as the type, features, price, mode, and effect of the service is included. The advertisement-related information includes, for example, in the case of a product advertisement, information such as the place where the product is sold, and in the case of a service, information such as the place where the service can be received.

[0020] Further, for example, the storage unit 22 includes a behavior history DB22b that stores personal information of the user, behavior history information of the user, and purchase history information of products or services. The personal information of the user stored in the behavior history DB22b includes, for example, information such as the user's gender, age, preferences, and hobbies. The behavior history information of the user stored in the behavior history DB22b includes, for example, information such as the user's Internet browsing record, the number of visits, location history, and participating social networking services (SNS). The purchase history information stored in the behavior history DB22b records products and services purchased by the user through online shopping or the like.

[0021] The behavior history acquisition unit 24 collects the user's personal information, behavior history information, and purchase history information, and stores or updates them in the behavior history DB 22b. Specifically, the information collection unit may collect the information from a terminal storage unit or a web browser cookie in the user terminal 3 (described later), or from a location identification unit (described later), and may acquire the information by accessing the terminal storage unit or a web browser cookie in the user terminal 3 (described later). However, any general method may be adopted, and detailed description thereof will be omitted here.

[0022] The user attribute determination unit 25 determines a segment to which the user belongs. The user attribute determination unit 25 can determine the tendency of advertisements of interest to the user (advertising range) from at least one of the user's personal information and behavioral history information (user attributes) stored in the storage unit 22. For example, if the user is a woman in her twenties (personal information) and the behavioral history information includes a history of accessing a cafe site multiple times and a history of purchasing coffee beans multiple times, the user attribute determination unit can determine that the user is interested in advertisements related to coffee, cafe latte, and cafe facilities (advertising range). In addition, the user attribute determination unit can learn about advertisements of interest to the user by machine learning or the like.

[0023] The providing unit 23 extracts advertisement information belonging to the segment from the advertisement DB 22a based on the segment determined by the user attribute determining unit 25, and transmits the advertisement information to the user terminal 3. Note that the providing unit 23 may transmit not only a single advertisement, but also a plurality of advertisements to the user terminal 3.

[0024] <User terminal 3> The user terminal 3 mainly includes a communication unit 31, a camera 32, an input unit 33, a display unit 34, an object management unit 35, an advertisement management unit 36, an advertisement response unit 37, an information output unit 38, and a position identification unit 39.

[0025] The terminal communication unit 31 is a part capable of communicating with an external device, and is, for example, a wireless communication means.

[0026] The camera 32 is a part that performs shooting operations, and has an image pickup element such as a CCD or CMOS element that converts an optical image into an electrical signal, and a shutter. In addition, the camera 32 can take not only still images but also moving images.

[0027] The input unit 33 can input data through a user's operation, such as a touch panel, a keyboard, etc. The input unit 33 can also input data not only by manual operation, but also by voice, for example.

[0028] The display unit 34 is, for example, a liquid crystal monitor, and is capable of displaying various types of information. The display unit 34 may be integrated with the input unit 33.

[0029] The object storage unit 35a stores model information necessary for generating virtual objects based on, for example, virtual characters or everyday items. The model information can be data in a format such as USDZ or glTF. The model information also includes, for example, attribute information directly related to the virtual object, related advertisement attribute information corresponding to the attribute information, and advertisement-related information corresponding to the advertisement-related information. The attribute information can include, for example, information on the appearance, type, mode, and component configuration.

[0030] The object management unit 35b creates augmented reality content. Specifically, the object management unit 35b creates an augmented reality image to be superimposed on an image of real space (an image captured by the camera 32). The object management unit 35b can create a virtual object based on model information stored in the object storage unit 35a and place it in the augmented reality space. The object management unit 35b can also place an advertisement object created by the advertisement management unit 36 ​​(described later) in the augmented reality space.

[0031] The advertisement receiving unit 36 ​​receives advertisement information from the server 2 .

[0032] The advertisement management unit 37 applies the received advertisement information to an advertisement object suitable for the advertisement information. The advertisement object is a virtual object created by the object management unit 35b. The advertisement object is, for example, an object (peripheral object) that is delivered together with an object designated by the user, such as a character. The advertisement object may be, for example, a picture frame or a television, which is a virtual object originally for displaying content on its surface, or may be a virtual object representing an item arranged around a virtual object that is a main content such as a character, such as a wall, a desk, or a door, or may be another virtual object that is owned by a virtual object that is a main content such as a character, such as a T-shirt or a cup. The advertisement management unit 37 can display an advertisement on the exterior of the advertisement object, for example, by attaching advertisement information of a still image or a moving image as a texture to the surface of the advertisement object. In addition, for example, when the advertisement information is audio information, the advertisement object can be attached with the audio information, and the audio information can also be reproduced when the advertisement object is reproduced. For example, when the advertisement management unit 37 receives advertisement information related to new clothes, the advertisement management unit 37 can attach the design of the new clothes to the surface of the advertisement object that has a clothing motif. The advertisement management unit 37 can attach the advertisement information to an advertisement object having an attribute corresponding to the attribute information of the advertisement information. For example, when the advertisement management unit 37 receives advertisement information for a new song, since the advertisement attribute information is music, the advertisement management unit 37 can attach the advertisement information to an advertisement object with a boombox motif that includes advertisement-related information corresponding to music.

[0033] The terminal storage unit has a function of storing the user's personal information, internet browsing history, product or service purchase history (described later), etc. The terminal storage unit may also include cookies of the web browser.

[0034] The purchase processing unit has a function for allowing a user to purchase products or services over the Internet, and also has a function for storing a purchase history of the products or services in the terminal storage unit.

[0035] The information output unit 38 outputs the augmented reality image created by the object management unit 35b by superimposing it on the image captured by the camera 32 of the real space. The information output unit 38 can also output visual information such as moving images, audio information, smell information, tactile information, taste information, etc. The information output unit can output visual information such as video via the display unit 34.

[0036] The position identification unit 39 has a function of acquiring and storing position information of the user terminal 3, for example, by a beacon that performs positioning using satellite positioning such as GPS or a short-range wireless technology. The position identification unit 39 can also transmit the acquired position information to the server 2.

[0037] Fig. 3 is a flowchart showing a process in which the user terminal 3 receives an advertisement. Fig. 4 is an explanatory diagram regarding the display of an augmented reality content in the augmented reality system according to the present embodiment. Fig. 5 is an explanatory diagram regarding an interaction between a virtual object and an advertisement in the augmented reality system according to the present embodiment.

[0038] First, in step S1, the user activates the augmented reality mode on the user terminal 3. The augmented reality mode is a state in which the camera 32 of the user terminal 3 is activated, and a surrounding real space image P (real space information) captured by the camera 32, a virtual character O generated by the object management unit 37, and a virtual drink D (virtual object) based on a drink motif are displayed on the display unit 34.

[0039] Next, in step S2, the information collection unit of the server acquires the user's personal information, behavior history information, purchase history information, etc. from the user terminal and stores or updates them in the storage unit. Then, the server determines the advertising range to be provided to the user based on the personal information, behavior history information, and purchase history information stored in the storage unit.

[0040] Next, in step S3, the server 2 extracts advertisements that fall within the determined advertising range from the advertisement DB and transmits them to the user terminal 3.

[0041] Next, in step S4, the advertisement management unit 37 generates an advertisement object C by associating the received advertisement with a suitable virtual drink D.

[0042] Next, in step S5, the object management unit 35b controls the virtual character O to react to the advertisement object C. In this embodiment, the virtual character O holds up the advertisement object C, but this is not limited thereto. For example, the virtual character may apply an external force to the advertisement object, such as touching or hitting the advertisement object. The virtual character may also mention the advertisement object. For example, when an advertisement is received or an advertisement object is generated, the virtual character may output a voice to that effect. The virtual character may also be used according to its purpose. For example, if the advertisement object is a drink, the virtual character may make a gesture of drinking the advertisement object.

[0043] Next, in step S6, the user displays or plays back the contents of the advertisement, and the process in this embodiment ends here.

[0044] In this manner, the augmented reality system 1 in this embodiment can output advertisements to the user along with augmented reality content.

[0045] <Second embodiment> In the second embodiment, the user terminal 3 omits the advertisement receiving unit 36 ​​and the advertisement management unit 37, and the storage unit 22 of the server 2 includes an object DB 22b in addition to the advertisement DB 22a.

[0046] The object DB 22b stores model information of virtual objects to be placed in the augmented reality space (hereinafter, simply referred to as virtual objects). The model information may be data in a format such as USDZ or glTF.

[0047] In the second embodiment, the providing unit 23 provides augmented reality content. The providing unit 23 can provide the augmented reality content in response to a request from the user terminal 3, for example. A virtual object (hereinafter referred to as a content object) that the user wishes to view, such as a character, is specified in the request. The providing unit 23 reads out the content object specified in the request from the storage unit 22, and responds to the user terminal 3 with augmented reality content including the content object and scene information for arranging the content object in an augmented reality space. At this time, the providing unit 23 can respond to the user terminal 3 by including a virtual object (hereinafter referred to as an advertisement object) for displaying an advertisement in the augmented reality content. The providing unit 23 can paste an advertisement image, for example, as a texture for the advertisement object.

[0048] In the second embodiment, prior to step S2, the user terminal 3 transmits a request for the augmented reality content to the server 2. The request includes a specification of a virtual object, such as a character, that the user wishes to view.

[0049] In addition, in step S3, the server 2 creates augmented reality content in response to the request. For example, the server 2 can read model information corresponding to a virtual object (hereinafter referred to as a content object) such as a character specified in the request from the object DB 22b to create the augmented reality content. The server 2 can also generate an object not specified by the user, such as a background set or a small item, as an advertisement object, and include the advertisement information in the texture of the advertisement object before attaching it to the augmented reality content. The server 2 can also include scene information specifying the placement positions and movements of the content object and advertisement object in the augmented reality content.

[0050] In step S4, the user terminal 3 receives the augmented reality content and is able to create an augmented reality scene based on the received augmented reality content.

[0051] In the second embodiment, steps S5 and S6 can be omitted.

[0052] As described above, the augmented reality content providing system according to the second embodiment can provide an advertisement object in association with a content object that the user is about to view. Therefore, the advertisement can be placed in the same augmented reality space as the content object in which the user is interested.

[0053] Although the embodiment of the present invention has been described above, the aspects of the present invention are not limited to this embodiment.

[0054] In the above embodiment, a segment is specified according to the attributes of the user, but the method is not limited to the above-mentioned method as long as the segmentation is for targeted advertising. For example, the segmentation may be performed based on a purchase history or a behavior history. The segmentation may be performed based on a behavior history limited to behavior in the augmented reality space. In addition, where the user was looking in the augmented reality space can be specified by, for example, where the focus point of the camera 32 was, and the segmentation may be performed according to the history of the place where the user was looking. In addition, it may be specified which object placed in the augmented reality space the user was looking at, and the segmentation may be performed according to the object the user was looking at.

[0055] In the above embodiment, the user plays the advertisement by operating the input unit 33, but this is not limited to the above. For example, the advertisement management unit 36 ​​may have a function of detecting keywords, and the advertisement management unit 36 ​​may have a function of displaying or playing the advertisement when it detects a keyword related to the advertisement during a voice or text conversation between the virtual character and the user. This has the effect of making the user more interested in and feel closer to the advertisement. (Effect of claim 5)

[0056] In the above embodiment, the advertisement provision unit 23 of the server 2 extracts advertisements related to the location information and transmits them to the user terminal 3, but this is not limited to the above. For example, the advertisement provision unit 23 may have a function of acquiring attribute information of virtual objects from the object management unit 35, and may extract advertisements related to the acquired attribute information. In this way, the server 2 can provide advertisements that are of high interest to the user. (Effect of claim 6)

[0057] In the above embodiment, the advertisement object is generated by associating the advertisement with the virtual object, but this is not limited to the above. For example, the object management unit 35 of the user terminal 3 may have a function of directly converting the advertisement into an advertisement object (virtual object). (Effect of claim 3)

[0058] In the above embodiment, the augmented reality mode is a state in which the surrounding real space image P (information about the real space) captured by the camera 32 and the virtual character O (virtual object) generated by the object management unit 37 are displayed on the display unit 34, but is not limited to this. For example, it may be a route plan service (for example, Live View, which is a Google map service) using augmented reality that recognizes the surrounding real space (buildings, roads, objects, etc.) captured by the camera based on the position information of the user terminal and the direction in which the user terminal is facing.

[0059] The object management unit 35 may also have a function of converting all or a part of the surrounding real space (buildings, roads, objects, etc.) recognized in the augmented reality mode into a virtual object (real space object). (Effect of Claim 4)

[0060] Furthermore, the advertisement provision unit 23 of the server 2 may extract advertisements related to the location information and the information related to the real space object, and transmit the advertisements to the user terminal 3. (Effect of Claim 7)

[0061] In the above embodiment, the user terminal 3 receives advertisements when the augmented reality mode is activated, but this is not limited to the above. For example, the user terminal 3 may automatically receive (update) advertisements related to the surrounding environment when the surrounding environment changes. (Effect of Claim 11)

[0062] For example, the user terminal 3 may be provided with an advertisement update unit (not shown) that updates advertisements in response to changes in the user's location. In detail, the advertisement update unit has a function of detecting a change in the user's location due to, for example, moving, and causing the advertisement management unit 36 ​​to acquire a new advertisement. (Effect of claim 12)

[0063] Furthermore, the condition for updating advertisements by the advertisement update unit is not limited to a change in the user's location, but may also be a change in time period. For example, the advertisement update unit may acquire advertisements during a specific time period, or at a predetermined time interval. Furthermore, the advertisement update unit may acquire updated advertisements every time advertisements related to search results are updated. Furthermore, the advertisement update unit may be integrated with the advertisement management unit 36. (Effect of claim 13)

[0064] In the above embodiment, the advertisement is displayed as a video, but the advertisement may be displayed as audio information.

[0065] Moreover, the advertisement and the real space environment in the above embodiment may be odor information. For example, the information output unit 38 of the user terminal 3 includes at least one odor element, and the information output unit 38 can emit one or more odor elements. The odor element may be, for example, a single chemical substance such as furaneol (strawberry odor) or 1-hexanol (forest odor), or a combination of these. Also, for example, an odor having an equivalent odor to the odor of jasmine as one unit may be used, regardless of the composition of the chemical substance. That is, the information output unit 38 may output, for example, an equivalent odor by combining a plurality of chemical substances using the odor of strawberry milk as an output method, or may output a combination of an odor having an odor of strawberry as one unit and an odor having an odor of milk as one unit.

[0066] Moreover, the advertisement and the real-space environment in the above embodiment may be taste information. For example, the information output unit 38 of the user terminal 3 includes at least one taste element, and the information output unit 38 can release one or more taste elements. The taste element may be, for example, a single chemical substance such as inosinic acid or glutamic acid, or a combination of these. Also, for example, a taste equivalent to strawberry flavor may be used as one unit, regardless of the composition of the chemical substance. That is, the information output unit 38 may output, for example, strawberry chocolate taste information by combining a plurality of chemical substances to output an equivalent taste, or may output a combination of a taste with strawberry flavor as one unit and a taste with chocolate flavor as one unit.

[0067] In addition, the advertisement and the real-space environment in the above embodiment may be tactile information. (Effect of Claim 14)

[0068] In addition, in this embodiment, a virtual space using augmented reality (AR) is assumed, but this is not limited to this, and the advertising system of this embodiment can be applied as is to a virtual reality (VR) space or a mixed reality (MR) space. [Explanation of symbols]

[0069] 1. Advertising System 2 Server 3. User terminal 21 Server Communication Department 22 Memory section 22a Advertising DB 23 Advertising Department 31 Terminal communication unit 32 Camera 33 Input section 34 Display section 35 Object Management Department 36 Advertising Management Department 37 Advertising Department 38 Information output section 39 Location identification part

Claims

1. an advertisement acquisition unit that acquires advertisement information according to attributes of a user who views an object arranged in an augmented reality space; an object management unit that arranges the object in the extended space and arranges the advertisement information; An augmented reality system comprising:

2. 2. The augmented reality system of claim 1, a purchase processing unit that purchases a product or service in the augmented reality space, the advertisement acquisition unit acquires the advertisement information according to a purchase history of the product or service; An augmented reality system featuring:

3. 2. The augmented reality system of claim 1, A behavior history acquisition unit that acquires behavior history information of the user in the augmented reality space, the advertisement acquisition unit acquires the advertisement information according to the behavior history; An augmented reality system featuring:

4. 2. The augmented reality system of claim 1, the object management unit arranges a plurality of the objects in the augmented reality space; A browsing detection unit that identifies the object that the user has browsed, the advertisement acquisition unit acquires the advertisement information according to a browsing history of the object; An augmented reality system featuring:

5. 2. The augmented reality system of claim 1, a viewing detection unit that identifies a location in the augmented reality space viewed by the user; the advertisement acquisition unit acquires the advertisement information according to a browsing history of the location; An augmented reality system featuring:

Images