Content display system, information terminal, information processing apparatus, and program

Abstract

To provide a technique that can easily change the details of display of additional information.SOLUTION: A content display system according to an aspect of the present disclosure comprises: a real-world object that indicates superimposing virtual content; and an information terminal that acquires a first image including the real-world object, and displays, on a display, a second image in which the virtual content according to identification information associated with the real-world object is superimposed on the first image.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a content display system, an information terminal, an information processing device, and a program. [Background technology]

[0002] 2. Description of the Related Art A technology called Augmented Reality (AR) is known that blends the real world with the virtual world by adding additional virtual information to a real landscape.

[0003] For example, Patent Document 1 discloses a technique for determining a superimposition area for additional information (eg, advertising information) on a video, and superimposing the additional information on the determined superimposition area. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2012-10117 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, in many cases, advertising as a sales promotion measure in stores is limited to simply listing information about products and services on craft tools made of paper or polypropylene (so-called POP (point of purchase advertising)). Therefore, it is difficult to make changes to such information once it has been printed and processed.

[0006] In view of the above, an object of the present disclosure is to provide a technique that enables easy change of the display content of additional information. [Means for solving the problem]

[0007] A content display system according to one embodiment of the present disclosure includes a real-world object indicating that virtual content is to be overlaid, and an information terminal that acquires a first image including the real-world object and displays on a display unit a second image in which virtual content corresponding to identification information associated with the real-world object is overlaid on the first image. Effect of the Invention

[0008] According to one aspect of the present disclosure, virtual content corresponding to identification information associated with a real-world object is superimposed on an image including the real-world object, thereby making it possible to easily change the display content of additional information according to the identification information. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram illustrating a schematic configuration example of an advertisement delivery system according to an embodiment of the present disclosure. [Diagram 2] FIG. 1 is a block diagram illustrating an example of a hardware configuration of a user device according to an embodiment of the present disclosure. [Diagram 3] FIG. 1 is a block diagram illustrating an example of a functional configuration of a user device according to an embodiment of the present disclosure. [Figure 4] FIG. 1 is a block diagram illustrating an example of a hardware configuration of an advertisement distribution device according to an embodiment of the present disclosure. [Diagram 5] FIG. 1 is a block diagram illustrating an example of a functional configuration of an advertisement distribution device according to an embodiment of the present disclosure. [Figure 6] FIG. 1 is an external perspective view showing an example of a fixture according to an embodiment of the present disclosure. [Figure 7] FIG. 1 is a diagram for explaining an example of access by a user device to an advertisement distribution device according to an embodiment of the present disclosure. [Figure 8] FIG. 1 is a diagram for explaining an example of display of display data on a user device according to an embodiment of the present disclosure. [Figure 9] FIG. 1 is a diagram showing a data table that is an example of association information between AR content and identification information associated with fixtures according to an embodiment of the present disclosure. [Figure 10A] 1 is a flowchart illustrating an example of the operation of a user device according to an embodiment of the present disclosure. [Figure 10B] 1 is a flowchart illustrating an example of the operation of a user device according to an embodiment of the present disclosure. [Figure 10C] 1 is a flowchart illustrating an example of the operation of a user device according to an embodiment of the present disclosure. [Figure 10D] 1 is a flowchart illustrating an example of the operation of a user device according to an embodiment of the present disclosure. [Figure 11A] 1 is a flowchart illustrating an example of an operation of an advertisement distribution device according to an embodiment of the present disclosure. [Figure 11B] 1 is a flowchart illustrating an example of an operation of an advertisement distribution device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. However, the scope of the invention is not limited to the examples shown in the drawings. In the following description, the same reference numerals are used to designate components having the same functions and configurations, and the description thereof will be omitted.

[0011] (Embodiment) <Summary> According to an embodiment of the present disclosure, a content display system (e.g., advertisement distribution system 1 described later) includes a real-world object (e.g., fixture 30 described later or an advertising surface 31 that is a part of fixture 30) indicating that virtual content (e.g., AR content) is to be superimposed, and an information terminal (e.g., user device 10 described later). The information terminal acquires a first image including the real-world object, for example, via an imaging device (e.g., imaging device 15 described later) included in the information terminal. The information terminal displays, on a display unit, a second image in which virtual content corresponding to identification information associated with the real-world object (e.g., information related to the location where fixture 30 is located) is superimposed on the first image.

[0012] The content display system further includes an information processing device (for example, an advertisement distribution device 20 described later). The information processing device generates a second image in response to recognition of a real-world object in a first image transmitted from the information terminal to the information processing device, and transmits the second image to the information terminal.

[0013] In this way, the content display system displays virtual content corresponding to identification information associated with a real-world object by superimposing it on an image including the real-world object, thereby making it possible to easily change the display content of additional information according to the identification information.

[0014] <Advertisement distribution system configuration> FIG. 1 is a block diagram illustrating a schematic configuration example of an advertisement delivery system according to an embodiment of the present disclosure.

[0015] As shown in Fig. 1, the advertisement delivery system 1 includes a user device 10, an advertisement delivery device 20, a fixture 30, and a network 40. The user device 10 and the advertisement delivery device 20 are communicably connected to each other via the network 40. The network 40 may be a local area network (LAN), the Internet, a virtual private network (VPN), a mobile communication network, or a combination of these. The advertisement delivery system 1 is an example of a content display system according to the present disclosure.

[0016] The user device 10 has a configuration capable of connecting to the advertisement distribution device 20 via, for example, a web browser or a dedicated application installed on the user device 10. For example, the following processing of the user device 10 may be performed via the web browser or the dedicated application.

[0017] The user device 10 acquires image data (also simply referred to as an image) generated by a camera (for example, an imaging device 15 described below) included in the user device 10, and transmits the image data to the advertisement distribution device 20 via the network 40. Here, such an image may include a part or all of the fixtures 30. The part or all of the fixtures 30 are examples of real-world objects that indicate the superimposition of virtual content (hereinafter also referred to as AR content, AR content data, or AR object), as described below.

[0018] The user device 10 receives an image (hereinafter also referred to as display data) output (transmitted) by the advertisement distribution device 20 via the network 40. Here, such an image may be an image in which AR content is superimposed by the advertisement distribution device 20 on the image transmitted by the user device 10, or may be an image transmitted by the user device 10 to the advertisement distribution device 20. The user device 10 displays the received image.

[0019] 1 shows one user device 10, there may be multiple user devices 10. The user device 10 may be called an information terminal, a user terminal, a terminal, a client, a client device, or the like.

[0020] The advertisement distribution device 20 is installed in a cloud environment to host an advertisement distribution web service. The advertisement distribution device 20 is accessed by the user device 10 when the user device 10 specifies a uniform resource locator (URL) that hosts the advertisement distribution web service via a web browser or a dedicated application. As a result, the advertisement distribution web service is provided to the user device 10.

[0021] The advertisement distribution device 20 receives the image transmitted by the user device 10 via the network 40. As described above, such an image may include a part or all of the fixture 30.

[0022] When the advertisement distribution device 20 recognizes a target for superimposing AR content (also called an AR target or a recognition target) in an image, that is, a real-world object indicating that virtual content is to be superimposed, the advertisement distribution device 20 performs the following in response to this recognition: The advertisement distribution device 20 generates display data (image) in which AR content associated with the AR target in the image is superimposed, for example, at the AR target position in the image, a position within the AR target, or a predetermined position a predetermined distance away from the AR target. At this time, if the advertisement distribution device 20 has acquired identification information associated with the fixture 30, it generates display data in which AR content in accordance with the identification information is superimposed. The identification information associated with the fixture 30 may be, for example, information on the position where the fixture 30 is installed (for example, latitude and longitude information, etc.), may be a store code of the store where the fixture 30 is installed, or more generally, may be unique information that identifies the fixture 30. Alternatively, the identification information associated with the fixtures 30 may be the same when the fixtures 30 are installed in the same area (for example, Tokyo), and may be different when the fixtures 30 are installed in different areas (for example, Tokyo and Osaka). The identification information associated with the fixtures 30 may be transmitted to the advertisement distribution device 20 by the user device 10 when the user device 10 accesses the advertisement distribution device 20.

[0023] On the other hand, if the advertisement distribution device 20 does not recognize the AR target in the image, it determines to use the received image as display data.

[0024] The advertisement distribution device 20 outputs (transmits) the generated display data to the user device 10 via the network 40.

[0025] Although one advertisement distribution device 20 is shown in FIG. 1, the functions of the advertisement distribution device 20 may be distributed among a plurality of devices, for example, for load balancing.

[0026] In this embodiment, the fixture 30 is a product shelf for sales promotion on which a product (e.g., cosmetics) is placed. A two-dimensional code (e.g., a QR code (registered trademark)) including advertising information (illustrations, photographs, characters, etc.) of the product and information for accessing the advertisement distribution device 20 (the above-mentioned URL) is written on the fixture 30. The two-dimensional code may include identification information associated with the fixture 30. The user device 10 can access (connect to) the advertisement distribution device 20 by scanning the two-dimensional code, and can transmit the identification information to the advertisement distribution device 20 if the two-dimensional code includes the identification information.

[0027] <User device configuration> 2 is a block diagram showing an example of a hardware configuration of a user device according to an embodiment of the present disclosure. The user device 10 may be realized by any computer or information processing device, such as a tablet, a smartphone, a notebook PC (Personal Computer), or a wearable device such as smart glasses or a head-mounted display.

[0028] 2, the user device 10 may include a storage device 11, a processing device 12, a user interface (UI) device 13, a communication device 14, an imaging device (photographing device) 15, and a bus 16. The storage device 11, the processing device 12, the UI device 13, the communication device 14, and the imaging device 15 may be connected to each other via the bus 16.

[0029] The programs or instructions for implementing the functions and processes of the user device 10 described above and below may be downloaded from some external device (e.g., a server) via a network, etc. Also, such programs or instructions may be provided from a removable storage medium such as a CD-ROM (Compact Disc Read Only Memory) or a flash memory.

[0030] Storage device 11 is realized by a random access memory (RAM), a flash memory, a hard disk drive, etc., and stores installed programs or instructions as well as files, data, etc. used in executing the programs or instructions. Storage device 11 may include a non-transitory storage medium.

[0031] The processing device 12 may be realized, for example, by a general-purpose processor or controller (circuit), or may be realized by a dedicated processor or controller (circuit). When realized by a general-purpose processor, the processing device 12 may be realized by one or more central processing units (CPUs), graphics processing units (GPUs), processing circuitry, etc., which may be composed of one or more processor cores. In this case, the processing device 12 executes the functions and processes of the user device 10 described above and below according to a program or instruction stored in the storage device 11, data such as parameters used to execute the program or instruction, etc. Such a program may be a program that causes the computer or processing device 12 to function as the user device 10 according to the present disclosure when executed by the computer or processing device 12. In addition, such a program may be a program that causes the computer or processing device 12 to execute a method related to advertisement delivery according to the present disclosure when executed by the computer or processing device 12.

[0032] The UI device 13 may be composed of input devices such as a keyboard, a mouse, a camera, a microphone, etc., output devices such as a display, a speaker, a headset, a printer, etc., and input / output devices such as a haptic device, a touch panel, etc., and realizes an interface between a user and the user device 10. For example, a user operates the user device 10 by operating a graphical user interface (GUI) displayed on a display or a touch panel using a keyboard, a mouse, etc.

[0033] The communication device 14 may be realized by various communication circuits that execute communication processes with external devices, the Internet, a LAN, a VPN, or other communication networks.

[0034] The imaging device 15 may be realized by including a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor. The imaging device 15 captures an image of a scene around the user device 10 to generate image data. The imaging device 15 outputs the generated image data to a communication control unit 101 and an image acquisition unit 102 described later, and stores the image data in the storage device 11 (an image storage unit 111 described later). The imaging device 15 generates image data including the above-mentioned two-dimensional code, a part or all of the fixture 30, etc.

[0035] The above-described hardware configuration of the user device 10 is merely an example, and the user device 10 according to the present disclosure may be realized by any other appropriate hardware configuration.

[0036] FIG. 3 is a block diagram illustrating an example of a functional configuration of a user device according to an embodiment of the present disclosure.

[0037] 3, the user device 10 includes a communication control unit 101, an image acquisition unit 102, a display data acquisition unit 103, a display control unit 104, an image storage unit 111, and a display data storage unit 112. The communication control unit 101, the image acquisition unit 102, the display data acquisition unit 103, and the display control unit 104 may be realized by the processing device 12 as hardware, as software (e.g., an application executed on a web browser or a dedicated application), or as a combination of hardware and software. The image storage unit 111 and the display data storage unit 112 may be realized by the storage device 11.

[0038] The communication control unit 101 acquires as input an image including a two-dimensional code output by the imaging device 15, and connects to the advertisement distribution device 20 by accessing a URL included in the two-dimensional code via the communication device 14 (if the user has permitted access). If the two-dimensional code includes identification information, the communication control unit 101 transmits the identification information to the advertisement distribution device 20 when accessing the URL. Note that this process of the communication control unit 101 does not necessarily need to be realized via a web browser or a dedicated application, and may be realized via a general-purpose function included in the user device 10.

[0039] The communication control unit 101 controls communication with the advertisement distribution device 20 by the image acquisition unit 102 and the display data acquisition unit 103 .

[0040] When the user inputs to the user device 10 via the UI device 13 to end image capture (photography) by the imaging device 15 in order to end advertisement display, the communication control unit 101 transmits image capture end information to the advertisement distribution device 20.

[0041] When a connection with the advertisement distribution device 20 is established by the communication control unit 101, the image acquisition unit 102 acquires, as an input, an image output by the imaging device 15. For example, the image acquisition unit 102 acquires a first image including a real-world object indicating that virtual content is to be superimposed. The image acquisition unit 102 transmits the acquired image to the advertisement distribution device 20 via the communication device 14. Note that the image acquisition unit 102 may transmit, together with the image, coordinate information when the image is displayed on the display device of the user device 10 to the advertisement distribution device 20.

[0042] The display data acquisition unit 103 acquires (receives) display data in which the AR content is or is not superimposed on the transmitted image from the advertisement distribution device 20 via the communication device 14. The display data acquisition unit 103 outputs the acquired display data to the display control unit 104 and stores it in the display data storage unit 112.

[0043] The display control unit 104 receives as input the display data output by the display data acquisition unit 103, generates a screen including the received display data, and displays the screen on a display device (display unit) included in the UI device 13. For example, the display control unit 104 displays on the display unit a second image in which virtual content corresponding to identification information associated with a real-world object is superimposed on a first image.

[0044] The image storage unit 111 stores image data generated by the imaging device 15 .

[0045] The display data storage unit 112 stores the display data acquired by the display data acquisition unit 103 .

[0046] <Configuration of the ad distribution device> 4 is a block diagram showing an example of a hardware configuration of an advertisement distribution device according to an embodiment of the present disclosure. The advertisement distribution device 20 may be realized by a calculation device (calculator, computer) such as a server or an information processing device.

[0047] 4, the advertisement distribution device 20 may include a storage device 21, a processing device 22, a UI device 23, a communication device 24, and a bus 25. The storage device 21, the processing device 22, the UI device 23, and the communication device 24 may be connected to each other via the bus 25.

[0048] The programs or instructions for implementing the functions and processes of the advertisement distribution device 20 described above and below may be downloaded from some external device (e.g., a server) via a network, etc. Also, such programs or instructions may be provided from a removable storage medium such as a CD-ROM or a flash memory.

[0049] Storage device 21 may be implemented by RAM, flash memory, a hard disk drive, etc., and stores installed programs or instructions as well as files, data, etc. used in executing the programs or instructions. Storage device 11 may include a non-transitory storage medium.

[0050] The processing device 22 may be realized by, for example, a general-purpose processor or controller (circuit), or may be realized by a dedicated processor or controller (circuit). When realized by a general-purpose processor, the processing device 22 may be realized by one or more CPUs, GPUs, processing circuits, etc., which may be composed of one or more processor cores. In this case, the processing device 22 executes the functions and processes of the advertisement distribution device 20 described above and below according to a program or instruction stored in the storage device 21, data such as parameters used to execute the program or instruction, etc. Such a program may be a program that causes the computer or processing device 22 to function as the advertisement distribution device 20 according to the present disclosure when executed by the computer or processing device 22. In addition, such a program may be a program that causes the computer or processing device 22 to execute a method related to advertisement distribution according to the present disclosure when executed by the computer or processing device 22.

[0051] The UI device 23 may be composed of input devices such as a keyboard, a mouse, a camera, a microphone, etc., output devices such as a display, a speaker, a headset, a printer, etc., and input / output devices such as a haptic device, a touch panel, etc., and realizes an interface between a user and the advertisement distribution device 20. For example, a user operates the advertisement distribution device 20 by operating a GUI displayed on a display or a touch panel using a keyboard, a mouse, etc.

[0052] The communication device 24 may be realized by various communication circuits that execute communication processes with external devices, the Internet, a LAN, a VPN, or other communication networks.

[0053] The above-described hardware configuration of the advertisement distribution device 20 is merely an example, and the advertisement distribution device 20 according to the present disclosure may be realized by another appropriate hardware configuration.

[0054] FIG. 5 is a block diagram illustrating an example of a functional configuration of an advertisement distribution device according to an embodiment of the present disclosure.

[0055] 5, the advertisement distribution device 20 includes a communication control unit 201, an image acquisition unit 202, a display data generation unit 203, a display data output unit 204, a content data storage unit 211, an image storage unit 212, and a display data storage unit 213. The communication control unit 201, the image acquisition unit 202, the display data generation unit 203, and the display data output unit 204 may be realized as hardware, software, or a combination of hardware and software by the processing device 12. The content data storage unit 211, the image storage unit 212, and the display data storage unit 213 may be realized by the storage device 21.

[0056] The communication control unit 201 establishes a connection with the user device 10 that has accessed the URL (connects to the user device 10) via the communication device 24. The communication control unit 201 may also receive identification information transmitted by the user device 10 (communication control unit 101) via the communication device 24. The communication control unit 201 outputs the identification information to the display data generation unit 203.

[0057] The communication control unit 201 controls communication with the user device 10 by the image acquisition unit 202 and the display data output unit 204 .

[0058] The communication control unit 201 receives, via the communication unit 24, the shooting end information transmitted by the user device 10 (communication control unit 101).

[0059] The image acquisition unit 202 acquires (receives) an image transmitted by the user device 10 (image acquisition unit 202) via the communication device 24. For example, the image acquisition unit 202 acquires a first image including a real-world object indicating that virtual content is to be superimposed from an information terminal. The image acquisition unit 202 outputs the acquired image to the display data generation unit 203 and stores it in the image storage unit 212.

[0060] The display data generating unit 203 receives the image output by the image acquiring unit 202 as an input.

[0061] When the display data generating unit 203 recognizes (or identifies or detects) an AR target in an image, the display data generating unit 203 reads out, from the content data storage unit 211, AR content data associated with the recognized AR target.

[0062] The display data generating unit 203 calculates (local) feature amounts of the image, and compares the calculated feature amounts with feature amounts of AR targets stored in advance, thereby recognizing the AR target within the image.

[0063] For example, the AR target may be a flat surface such as an advertising surface on which an advertising photo or the like is attached, a curved surface (such as the side of a cone or cylinder), or a floor surface. The AR target may exist on a flat surface, a curved surface, or a floor surface as an advertising photo or the like. The AR target may also be the sky.

[0064] The display data generating unit 203 superimposes the read AR content on the received image data at a designated or predetermined position such as a recognition position where the AR target is recognized, a position near the recognition position, or a position at a predetermined distance from the recognition position, by referring to the association information described later, and generates the image data on which the AR content is superimposed as display data. When the display data generating unit 203 receives the identification information output by the communication control unit 201 as an input, the display data generating unit 203 generates display data in which the AR content corresponding to the identification information is superimposed on the image when generating the display data. For example, in response to recognizing a real-world object in the first image, the display data generating unit 203 generates a second image in which virtual content corresponding to the identification information associated with the real-world object is superimposed on the first image.

[0065] On the other hand, if the display data generating unit 203 does not recognize the AR target in the image, it determines to use the received image data as display data.

[0066] The display data generating unit 203 outputs the display data (image data on which the AR content is superimposed or accepted image data) to the display data output unit 204, and also causes the display data storage unit 213 to store the same.

[0067] The display data output unit 204 receives the display data output by the display data generation unit 203 as an input, and transmits (outputs) the display data to the user device 10 via the communication device 24. For example, the display data output unit 204 transmits a second image generated by the display data generation unit 203 to the user device 10.

[0068] The content data storage unit 211 stores AR content data. The AR content data may be any content data that can be displayed via a display device, such as a still image, a moving image, an animation effect, a three-dimensional (3D) object, a 3D animation, or text.

[0069] The content data storage unit 211 also stores association information (for example, in a table format) that associates the AR target, the AR content, and the identification information (for example, see FIG. 9). Thus, when the display data generation unit 203 recognizes an AR target in an image, the display data generation unit 203 can read out, from the content data storage unit 211, the AR content that is associated with the recognized AR target and conforms to the identification information by referring to the association information.

[0070] The image storage unit 212 stores the image data acquired by the image acquisition unit 202 .

[0071] The display data storage unit 213 stores the display data generated by the display data generation unit 203 and image data accepted by the display data generation unit 203 and used as the display data.

[0072] <Furniture composition> 6 is an external perspective view showing an example of a fixture according to an embodiment of the present disclosure. As shown in FIG. 6, the fixture 30 includes an advertising surface 31, an advertising area 32, an advertising distribution device access information area 33, and a product display area 34.

[0073] The advertising surface 31 is a surface on which various advertising information (illustrations, photos, text, etc.) of the product is written. In this embodiment, the advertising surface 31 is a flat surface. However, the advertising surface 31 may be a curved surface.

[0074] Advertisement information is written in the advertisement area 32. There may be a plurality of advertisement areas 32.

[0075] In the advertisement distribution device access information area 33, a two-dimensional code including a URL that hosts an advertisement distribution web service, which is information for accessing the advertisement distribution device 20, is described. The two-dimensional code may optionally include identification information.

[0076] In the product display area 34, products (lipstick in FIG. 6) are displayed.

[0077] In this embodiment, the advertisement surface 31 is the AR target. However, some or all of one or more advertisement areas 32 may be the AR target, or the entire fixture 30 may be the AR target. The fixture 30, the advertisement surface 31, and the advertisement areas 32 are examples of real-world objects.

[0078] <Access to ad distribution device by user device> FIG. 7 is a diagram illustrating an example of access by a user device to an advertisement distribution device according to an embodiment of the present disclosure.

[0079] As shown in FIG. 7, for example, when a user turns on the camera function of the user device 10 and takes a picture of the advertisement distribution device access information area 33 of the fixture 30, the user device 10 can access the URL included in the two-dimensional code and connect to the advertisement distribution device 20.

[0080] <Displaying data for display on user device> Fig. 8 is a diagram for explaining an example of display of display data on a user device according to an embodiment of the present disclosure, and Fig. 9 is a diagram showing a data table that is an example of association information between AR content and identification information associated with a fixture according to an embodiment of the present disclosure. Although not shown, the data table 900 shown in Fig. 9 is associated with the advertisement surface 31 as an AR target.

[0081] As shown in FIG. 9, a data table 900 has a “content ID” field 901 , an “identification information” field 902 , and a “content description” field 903 .

[0082] In the “content ID” field 901, information for identifying the AR content to be superimposed on the image transmitted from the user device 10 is stored.

[0083] The “identification information” field 902 stores identification information associated with the fixture 30.

[0084] The "content description" field 903 stores a description of the corresponding content.

[0085] As a premise for explanation with reference to FIG. 8, it is assumed that the user device 10 is connected to the advertisement distribution device 20, and f1 is transmitted from the user device 10 to the advertisement distribution device 20 as identification information associated with the fixture 30 (hence, as identification information associated with the advertisement surface 31 and the advertisement area 32). Images generated by the imaging device 15 of the user device 10 are sequentially transmitted from the user device 10 to the advertisement distribution device 20. Then, images (display data) with or without AR content superimposed on the images generated by the imaging device 15 are sequentially transmitted from the advertisement distribution device 20 to the user device 10, and displayed on the user device 10. At this time, since the advertisement surface 31 is an AR target, the advertisement distribution device 20 recognizes the advertisement surface 31 in the image as the AR target, and reads out from the content data storage unit 211 a video for f1 having a content ID "c1" corresponding to the identification information "f1" associated with the fixture 30 (the advertisement surface 31 of the fixture 30), thereby superimposing the AR content. As a result, the user device 10 displays the AR content corresponding to the identification information associated with the fixture 30 (the advertisement surface 31 of the fixture 30) in a superimposed manner on the advertisement surface 31.

[0086] Depending on the fixture 30, a two-dimensional code that does not include identification information may be attached. In this case, the advertisement distribution device 20 reads out from the content data storage unit 211 a general video having a content ID "cN" corresponding to the identification information "-" shown in the bottom row of the data table 900, thereby superimposing the AR content.

[0087] Furthermore, multiple AR contents (AR targets) may exist for one piece of fixture 30. In this case, another field associated with the AR content (for example, a field in which information identifying the AR target is stored when multiple AR targets exist) may exist in one data table 900. Alternatively, multiple data tables 900 each corresponding to one AR content may exist.

[0088] By having the above-mentioned configuration, the user device 10 acquires a first image including a real-world object indicating that virtual content is to be overlaid, and displays on the display unit a second image in which virtual content corresponding to identification information associated with the real-world object is overlaid on the first image.

[0089] By having the above-mentioned configuration, the advertisement distribution device 20 receives a first image from the user device 10, and in response to recognizing a real-world object in the first image, generates a second image and transmits the second image to the user device 10.

[0090] <Operation of the ad distribution system> Next, an example of the operation of the advertisement delivery system 1 will be described with reference to FIGS. 10A to 11B.

[0091] 10A is a flowchart illustrating an example of an operation of a user device according to an embodiment of the present disclosure. The example of the operation illustrated in FIG. 10A relates to access to the advertisement distribution device 20 by the user device 10.

[0092] In step S101, the user device 10 (communication control unit 101) connects to the advertisement distribution device 20 by accessing a URL included in a two-dimensional code in an image of the fixture 30 generated by the imaging device 15. If the two-dimensional code includes identification information, in step S101, the user device 10 (communication control unit 101) transmits the identification information to the advertisement distribution device 20. Then, the flow ends.

[0093] Fig. 10B is a flowchart showing an example of an operation of a user device according to an embodiment of the present disclosure. The example of the operation shown in Fig. 10B is related to the transmission of an image by the user device 10 to the advertisement distribution device 20. In the example of the operation shown in Fig. 10B, it is assumed that the user device 10 is connected to the advertisement distribution device 20.

[0094] In step S121, the user device 10 (image acquisition unit 102) acquires an image of the fixture 30 or the like from the imaging device 15.

[0095] In step S122, the user device 10 (image acquisition unit 102) transmits the captured image of the fixture 30 or the like to the advertisement distribution device 20. Then, the flow ends.

[0096] The process shown in FIG. 10B is repeatedly executed until the user inputs an instruction to end image capture to the user device 10 (see FIG. 10D).

[0097] 10C is a flowchart illustrating an example of an operation of a user device according to an embodiment of the present disclosure. The example of the operation illustrated in FIG. 10C relates to reception and display of display data from the advertisement distribution device 20 by the user device 10. In the example of the operation illustrated in FIG. 10C, it is assumed that the user device 10 is connected to the advertisement distribution device 20.

[0098] In step S141, the user device 10 (the display data acquisition unit 103) receives display data from the advertisement distribution device 20. The display data is an image in which AR content is superimposed on an image transmitted from the user device 10 to the advertisement distribution device 20, or an image transmitted from the user device 10 to the advertisement distribution device 20.

[0099] In step S142, the user device 10 (display control unit 104) generates a screen including the display data, and displays the screen including the display data on a display device (display unit) included in the user device 10. Then, the flow ends.

[0100] The process shown in FIG. 10C is repeatedly executed until the user inputs an instruction to end image capture to the user device 10 (see FIG. 10D).

[0101] 10D is a flowchart showing an example of an operation of a user device according to an embodiment of the present disclosure. The example of the operation shown in FIG. 10D relates to the end of the method related to advertisement distribution according to the present disclosure. In the example of the operation shown in FIG. 10D, it is assumed that the user device 10 is connected to the advertisement distribution device 20.

[0102] In step S161, the user device 10 (communication control unit 101) transmits shooting end information to the advertisement distribution device 20 in response to a user's input to end shooting to the user device 10. Then, the flow ends. This ends the method related to advertisement distribution according to the present disclosure.

[0103] 11A is a flowchart illustrating an example of an operation of the advertisement distribution device according to an embodiment of the present disclosure. The example of the operation illustrated in FIG 11A corresponds to the example of the operation illustrated in FIG 10A, and relates to a connection between the advertisement distribution device 20 and the user device 10.

[0104] In step S201, the advertisement distribution device 20 (communication control unit 201) connects to the user device 10 in response to access to a URL by the user device 10. Note that in step S201, the advertisement distribution device 20 (communication control unit 201) may also receive identification information from the user device 10. Then, the flow ends.

[0105] Fig. 11B is a flowchart showing an example of an operation of the advertisement delivery device according to an embodiment of the present disclosure. The example of operation shown in Fig. 11B corresponds to the example of operation shown in Figs. 10B to 10D, and relates to the reception of an image from the user device 10 by the advertisement delivery device 20, the transmission of display data to the user device 10, the end of the method for advertisement delivery according to the present disclosure, etc. In the example of operation shown in Fig. 11B, it is assumed that the advertisement delivery device 20 is connected to the user device 10.

[0106] In step S251, the advertisement distribution apparatus 20 (the communication control unit 201) determines whether or not image-taking by the user device 10 has ended by determining whether or not image-taking end information has been received.

[0107] If the shooting has not ended (Step S251; NO), the advertisement distribution device 20 (image acquisition unit 202) acquires the image transmitted by the user device 10 in Step S252.

[0108] In step S253, the advertisement distribution device 20 (the display data generating unit 203) determines whether or not a recognizable AR target is present in the image.

[0109] If there is no recognizable AR target in the image, that is, if the AR target is not recognized (step S253; NO), in step S254, the advertisement distribution device 20 (display data generation unit 203) determines whether or not AR content exists in the previous display data.

[0110] If the AR content does not exist (step S254; NO), in step S255, the advertisement distribution device 20 (display data generation unit 203) determines to use the image as display data without superimposing the AR content. Then, the flow proceeds to step S260. Note that step S255 is also executed when the image acquired in step S252 is the image initially transmitted by the user device 10.

[0111] On the other hand, if the AR content exists (step S254; YES), in step S256, the advertisement distribution device 20 (display data generation unit 203) decides to clear the AR content and use the image as display data. Then, the flow proceeds to step S260.

[0112] In step S253, if there is a recognizable AR target in the image, that is, if the AR target is recognized (step S253; YES), in step S257, the advertisement distribution device 20 (display data generation unit 203) determines whether the AR target is associated with pre-specified identification information. Then, the flow proceeds to step S260.

[0113] If the AR target is not associated with the previously specified identification information (step S257; NO), in step S258, the advertisement distribution device 20 (display data generation unit 203) refers to the association information and generates display data in which the AR content that is associated with the AR target but not associated with any identification information is superimposed on the image. Then, the flow proceeds to step S260.

[0114] On the other hand, if the AR target is associated with the previously specified identification information (step S257; YES), in step S259, the advertisement distribution device 20 (display data generation unit 203) refers to the association information and generates display data in which the AR content corresponding to the previously specified identification information and corresponding to the AR target is superimposed on the image. Then, the flow proceeds to step S260.

[0115] In step S260, the advertisement-distribution device 20 (display data output unit 204) outputs (transmits) the display data to the advertisement-distribution device 20. Then, the flow returns to step S251.

[0116] If the shooting is completed (step S251; YES), in step S261, the advertisement distribution device 20 (image acquisition unit 202, etc.) clears all processing (for example, deletes the image from the image storage unit 212, etc.), and the flow ends.

[0117] <Modification> Modifications of the above embodiment will now be described.

[0118] [Variation 1] In the above embodiment, an example has been described in which the advertisement surface 31 is the AR target and AR content is superimposed on the advertisement surface 31, but the present disclosure is not limited to this example. For example, the AR target may be the entire fixture 30, and the display data generating unit 203 may generate display data in which AR content is superimposed on the advertisement surface 31, which is a part of the AR target (fixture 30).

[0119] [Variation 2] In the above embodiment, an example of recognizing an AR target in an image by so-called markerless AR has been described, but the present disclosure is not limited to this example. For example, the AR target may be recognized by the display data generating unit 203 using an AR marker, which is a pattern image that serves as a marker for specifying a position where the AR content is to be displayed.

[0120] [Variation 3] In the above embodiment, the user device 10 accesses the advertisement distribution device 20 by reading a URL included in a two-dimensional code in image data generated by the imaging device 15. However, the present disclosure is not limited to this example. For example, if the user device 10 supports RFID (radio frequency identifier) ​​such as NFC (near field communication), an RF tag storing a URL and, optionally, identification information may be provided on the fixture 30, and the user device 10 may access the advertisement distribution device 20 (URL) by reading the RF tag using the RFID, and may optionally transmit the identification information to the advertisement distribution device 20. In addition, for example, the user device 10 may access the advertisement distribution device 20 (URL) in response to a user inputting a URL into the user device 10.

[0121] [Variation 4] In the above embodiment, an example was described in which the user device 10 reads identification information contained in a two-dimensional code in image data generated by the imaging device 15 and transmits it to the advertising distribution device 20, but the present disclosure is not limited to this example.

[0122] For example, the identification information may be information (e.g., SSID (service set identifier)) regarding the wireless LAN (e.g., WiFi (registered trademark)) of the store in which the fixture 30 is installed, and the information acquired by the user device 10 may be transmitted from the user device 10 to the advertising distribution device 20 at some timing (e.g., when a URL is accessed).

[0123] Also, for example, the identification information may be a beacon-specific ID transmitted by a beacon of the store in which the fixture 30 is installed, and the information acquired by the user device 10 may be transmitted from the user device 10 to the advertising distribution device 20 at some timing (for example, when accessing a URL).

[0124] Also, for example, the identification information may be location information acquired by a positioning function such as a GPS function included in the user device 10, and the information acquired by the user device 10 may be transmitted from the user device 10 to the advertisement distribution device 20 at some timing (for example, when accessing a URL). Note that the location information may be absolute location information or relative location information.

[0125] Furthermore, for example, the identification information may be location information derived from an image acquired by the user device 10. The information may be derived by the user device 10 and transmitted from the user device 10 to the advertisement distribution device 20 at some timing (for example, when a URL is accessed). Alternatively, the information may be derived by the advertisement distribution device 20.

[0126] [Variation 5] In the above embodiment, an example has been described in which the advertisement distribution device 20 generates display data in which virtual content is superimposed on an image including a real-world object, but the present disclosure is not limited to this example. For example, the user device 10 may have a function (the same or similar functional unit as described above) provided in the advertisement distribution device 20 in the form of an application. That is, the user device 10 may generate and display display data in which virtual content corresponding to identification information associated with a real-world object is superimposed on an image including the real-world object.

[0127] [Other variations] The names of the above-mentioned devices (user device 10, advertisement distribution device 20, etc.) and the functional units constituting the devices (communication control unit 101, image acquisition unit 102, etc.) are merely examples and may be changed as appropriate. Furthermore, two or more of the functional units constituting the devices may be combined and integrated, or one functional unit may be divided into functional units. Furthermore, the transmission function and / or reception function possessed by each functional unit may be summarized as a notification unit, a notified unit, a communication unit, a transmission / reception unit, a transmission unit, a reception unit, etc.

[0128] The order of steps in the flowcharts shown in the figures and described above is merely an example, and may be rearranged as appropriate, and some steps may be performed in parallel.

[0129] As described above, according to one aspect of the present disclosure, virtual content corresponding to identification information associated with a real-world object is superimposed on an image including the real-world object and displayed. This makes it possible to easily change the display content of additional information according to the identification information. For example, when the identification information is location information, virtual content corresponding to the location where the real-world object exists can be displayed.

[0130] Although the embodiments have been described above with reference to the drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art can come up with various modified or amended examples within the scope of the claims. It is understood that such modified or amended examples also belong to the technical scope of the present disclosure. In addition, each component or each item in the embodiment (including modified or amended examples) may be arbitrarily combined within the scope of the present disclosure. [Explanation of symbols]

[0131] 1. Advertisement distribution system 10. User Devices 101 Communication control unit 102 Image acquisition unit 103 Display data acquisition unit 104 Display control unit 111 Image storage unit 112 Display data storage unit 20 Advertisement distribution device 201 Communication control section 202 Image acquisition unit 203 Display data generation unit 204 Display data output section 211 Content data storage unit 212 Image storage unit 213 Display data storage unit 30 Furniture 31 Advertising space 32 Advertising Area 33 Ad distribution device access information area 34 Product display area

Claims

1. a real-world object indicating overlay of virtual content; an information terminal that acquires a first image including the real-world object, and displays, on a display unit, a second image in which virtual content corresponding to identification information associated with the real-world object is superimposed on the first image; A content display system comprising:

2. The content display system further includes an information processing device, The information terminal transmits the first image to the information processing device, the information processing device receives the first image from the information terminal, and in response to recognizing the real-world object in the first image, generates the second image and transmits the second image to the information terminal. The content display system according to claim 1 .

3. The information terminal transmits the identification information to the information processing device, the information processing device generates the second image by using the virtual content corresponding to the received identification information. The content display system according to claim 2 .

4. The identification information is information regarding a location where the real-world object exists. A content display system according to any one of claims 1 to 3.

5. the information terminal generates the second image in response to recognizing the real-world object in the first image. The content display system according to claim 1 .

6. an image capture unit that captures a first image including a real-world object indicative of an overlay of virtual content; a display control unit that causes a display unit to display a second image in which virtual content corresponding to identification information associated with the real-world object is superimposed on the first image; An information terminal comprising:

7. an image acquisition unit that acquires, from an information terminal, a first image including a real-world object indicating that virtual content is to be overlaid; a display data generating unit that generates a second image in which virtual content corresponding to identification information associated with the real-world object is superimposed on the first image in response to the recognition of the real-world object in the first image; a display data output unit that transmits the second image to the information terminal; An information processing device comprising:

8. acquiring a first image including a real-world object indicative of overlaying virtual content; displaying, on a display unit, a second image in which virtual content corresponding to identification information associated with the real-world object is superimposed on the first image; A program for causing an information terminal to execute processing including the above.

9. acquiring a first image from an information terminal, the first image including a real-world object indicating that virtual content is to be overlaid; generating a second image in response to recognizing the real-world object in the first image, the second image having virtual content corresponding to identification information associated with the real-world object superimposed on the first image; transmitting the second image to the information terminal; A program for causing an information processing device to execute a process including the steps of:

Images