Content display system, information terminal, information processing device, and program

Abstract

To provide a technique capable of increasing appeal power for a customer.SOLUTION: A content display system according to an embodiment comprises a device including a first object indicative of superposition of virtual content and a second object where a prescribed operation can be performed, and an information terminal which acquires a first image including the first object and displays a second image where the virtual content associated with the first object is superposed on the first image in a display unit. The device causes the second object to perform the prescribed operation according to a touch for the first object displayed in the display unit or the virtual content.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a content display system, an information terminal, an information processing device, and a program. [Background technology]

[0002] 2. Description of the Related Art A technology called Augmented Reality (AR) is known that blends the real world with the virtual world by adding additional virtual information to a real landscape.

[0003] For example, Patent Document 1 discloses a technique for determining a superimposition area for additional information (eg, advertising information) on a video, and superimposing the additional information on the determined superimposition area. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2012-10117 A Summary of the Invention [Problem to be solved by the invention]

[0005] Incidentally, in-store advertising as a sales promotion measure is often limited to listing information about products or services on craft tools made of paper or polypropylene (so-called POP (point of purchase advertising)). There is room for improvement in terms of providing content to attract customers.

[0006] In view of the above, the present disclosure aims to provide a technology that can increase appeal to customers. [Means for solving the problem]

[0007] A content display system according to one embodiment of the present disclosure includes a device having a first object indicating that virtual content is to be overlaid and a second object capable of a predetermined operation, and an information terminal that acquires a first image including the first object and displays on a display unit a second image in which virtual content associated with the first object is overlaid on the first image, and the device causes the second object to perform the predetermined operation in response to a touch on the first object or the virtual content displayed on the display unit. Effect of the Invention

[0008] According to one aspect of the present disclosure, a virtual content is displayed superimposed on an image including a first object included in a device, and a second object of the device is caused to perform a predetermined action in response to a touch on the displayed first object or the virtual content. This allows the second object to perform a predetermined action in response to a touch by a user (customer) of the information terminal, thereby increasing the appeal to customers through interactive communication with the customers. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram illustrating a schematic configuration example of an advertisement delivery system according to an embodiment of the present disclosure. [Diagram 2] FIG. 1 is a block diagram illustrating an example of a hardware configuration of a user device according to an embodiment of the present disclosure. [Diagram 3] FIG. 1 is a block diagram illustrating an example of a functional configuration of a user device according to an embodiment of the present disclosure. [Figure 4] FIG. 1 is a block diagram illustrating an example of a hardware configuration of an advertisement distribution device according to an embodiment of the present disclosure. [Diagram 5] FIG. 1 is a block diagram illustrating an example of a functional configuration of an advertisement distribution device according to an embodiment of the present disclosure. [Figure 6] FIG. 1 is an external perspective view showing an example of a fixture according to an embodiment of the present disclosure. [Figure 7]FIG. 1 is a block diagram showing an example of the hardware configuration of an information processing device included in a fixture according to an embodiment of the present disclosure. [Figure 8] A block diagram showing an example of the functional configuration of an information processing device included in a fixture according to an embodiment of the present disclosure. [Figure 9A] FIG. 1 is a diagram for explaining a display example on a user device and an example of access by the user device to an advertisement distribution device according to an embodiment of the present disclosure. [Figure 9B] FIG. 1 is a diagram illustrating an example of a display on a user device according to an embodiment of the present disclosure. [Figure 9C] FIG. 1 is a diagram illustrating an example of a display on a user device according to an embodiment of the present disclosure. [Figure 10A] FIG. 1 is a diagram illustrating a data table that is an example of association information between AR content and AR targets according to an embodiment of the present disclosure. [Figure 10B] FIG. 1 is a diagram illustrating a data table that is an example of association information between AR content and an event that occurs in response to a touch on the AR content according to an embodiment of the present disclosure. [Figure 10C] FIG. 1 is a diagram illustrating a data table that is an example of association information between an AR target and an event that occurs in response to a touch on the AR target according to an embodiment of the present disclosure. [Figure 10D] FIG. 1 illustrates an exemplary data table showing event content according to one embodiment of the present disclosure. [Figure 11A] 1 is a flowchart illustrating an example of the operation of a user device according to an embodiment of the present disclosure. [Figure 11B] 1 is a flowchart illustrating an example of the operation of a user device according to an embodiment of the present disclosure. [Figure 11C] 1 is a flowchart illustrating an example of the operation of a user device according to an embodiment of the present disclosure. [Figure 11D] 1 is a flowchart illustrating an example of the operation of a user device according to an embodiment of the present disclosure. [Figure 12A] 1 is a flowchart illustrating an example of an operation of an advertisement distribution device according to an embodiment of the present disclosure. [Figure 12B]1 is a flowchart illustrating an example of an operation of an advertisement distribution device according to an embodiment of the present disclosure. [Figure 12C] 1 is a flowchart illustrating an example of an operation of an advertisement distribution device according to an embodiment of the present disclosure. [Figure 13A] A flowchart showing an example of the operation of a fixture according to an embodiment of the present disclosure. [Figure 13B] A flowchart showing an example of the operation of a fixture according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. However, the scope of the invention is not limited to the examples shown in the drawings. In the following description, the same reference numerals are used to designate components having the same functions and configurations, and the description thereof will be omitted.

[0011] (Embodiment) <Summary> According to an embodiment of the present disclosure, a content display system (e.g., an advertisement distribution system 1 described later) includes a device (e.g., a fixture 30 described later) having a first object (e.g., products 35A to 35C that are part of a fixture 30 described later, an advertisement area 32) indicating that virtual content (e.g., AR content) is to be superimposed, and a second object (e.g., lights 36A to 36C and a door 37 described later) capable of a predetermined action (e.g., a light is turned on, a door is opened), and an information terminal (e.g., a user device 10 described later). The information terminal acquires a first image including the first object, for example, via an imaging device (e.g., an imaging device 15 described later) included in the information terminal. The information terminal displays a second image in which virtual content associated with the first object is superimposed on the first image on a display unit. The device causes the second object to perform a predetermined action in response to a touch on the first object or the virtual content displayed on the display unit.

[0012] The content display system further includes an information processing device (for example, an advertisement distribution device 20 described later). The information processing device generates a second image in response to recognizing the first object in the first image transmitted from the information terminal to the information processing device, and transmits the second image to the information terminal.

[0013] In this way, the content display system displays virtual content superimposed on an image including a first object, and causes a second object possessed by the device to perform a predetermined action in response to a touch on the displayed first object or virtual content. This allows the second object to perform a predetermined action in response to a touch by a user (customer) of the information terminal, thereby increasing appeal to customers through interactive communication with the customers.

[0014] <Advertisement distribution system configuration> FIG. 1 is a block diagram illustrating a schematic configuration example of an advertisement delivery system according to an embodiment of the present disclosure.

[0015] As shown in FIG. 1, the advertisement delivery system 1 includes a user device 10, an advertisement delivery device 20, a fixture 30, and a network 40. The user device 10 and the advertisement delivery device 20 are communicatively connected to each other via the network 40, and the advertisement delivery device 20 and the fixture 30 are communicatively connected to each other via the network 40. The network 40 may be a local area network (LAN), the Internet, a virtual private network (VPN), a mobile communication network, or a combination thereof. The user device 10 and the fixture 30 may be communicatively connected to each other via the network 40, or may be communicatively connected directly to each other. The advertisement delivery system 1 is an example of a content display system according to the present disclosure.

[0016] The user device 10 has a configuration capable of connecting to the advertisement distribution device 20 via, for example, a web browser or a dedicated application installed on the user device 10. For example, the following processing of the user device 10 may be performed via the web browser or the dedicated application.

[0017] The user device 10 acquires image data (also simply referred to as an image) generated by a camera (for example, an imaging device 15 described below) included in the user device 10, and transmits the image data to the advertisement distribution device 20 via the network 40. Here, such an image may include a part or all of the fixtures 30. The part or all of the fixtures 30 are examples of real-world objects or objects that indicate the superimposition of virtual content (hereinafter also referred to as AR content, AR content data, or AR objects), as described below.

[0018] The user device 10 receives an image (hereinafter also referred to as display data) output (transmitted) by the advertisement distribution device 20 via the network 40. Here, such an image may be an image in which AR content is superimposed by the advertisement distribution device 20 on the image transmitted by the user device 10, or may be an image transmitted by the user device 10 to the advertisement distribution device 20. The user device 10 displays the received image.

[0019] The user device 10 transmits the coordinates (touch position coordinates) of the position touched by the user on the display device (or display unit or screen) of the user device 10 to the advertisement distribution device 20 via the network 40. As described later, in response to the user touching specific AR content or the like, an event occurs in which a specific object of the fixture 30 performs a predetermined operation.

[0020] 1 shows one user device 10, there may be multiple user devices 10. The user device 10 may be called an information terminal, a user terminal, a terminal, a client, a client device, or the like.

[0021] The advertisement distribution device 20 is installed in a cloud environment to host an advertisement distribution web service. The advertisement distribution device 20 is accessed by the user device 10 when the user device 10 specifies a uniform resource locator (URL) that hosts the advertisement distribution web service via a web browser or a dedicated application. As a result, the advertisement distribution web service is provided to the user device 10.

[0022] The advertisement distribution device 20 receives the image transmitted by the user device 10 via the network 40. As described above, such an image may include a part or all of the fixture 30.

[0023] When the advertisement delivery device 20 recognizes a target for superimposing AR content (also called an AR target or a recognition target) in an image, i.e., a real-world object or object indicating that virtual content is to be superimposed, the advertisement delivery device 20 performs the following in response to this recognition: the advertisement delivery device 20 generates display data (image) in which the AR content associated with the AR target in the image is superimposed, for example, at the AR target position in the image, a position within the AR target, or a predetermined position a predetermined distance away from the AR target.

[0024] On the other hand, if the advertisement distribution device 20 does not recognize the AR target in the image, it determines to use the received image as display data.

[0025] The advertisement distribution device 20 receives the touch position coordinates transmitted by the user device 10 via the network 40. The advertisement distribution device 20 determines whether or not the user has touched a specific AR content or an AR target on the display device of the user device 10, i.e., whether or not the user's touch is directed to a specific AR content or an AR target, based on the received touch position coordinates. When the advertisement distribution device 20 determines that the user has touched a specific AR content or an AR target, the advertisement distribution device 20 generates a predetermined event (a predetermined operation of an object owned by the fixture 30). For example, the predetermined event generated by the advertisement distribution device 20 may be the illumination of a light, which is an example of an object, or the opening of a door, which is another example of an object. The advertisement distribution device 20 outputs (transmits) a command corresponding to the event, i.e., a command for causing an object owned by the fixture 30 to perform a predetermined operation, to the fixture 30 via the network 40.

[0026] The advertisement distribution device 20 outputs (transmits) the generated display data to the user device 10 via the network 40.

[0027] Although one advertisement distribution device 20 is shown in FIG. 1, the functions of the advertisement distribution device 20 may be distributed among a plurality of devices, for example, for load balancing.

[0028] In this embodiment, the fixture 30 is a sales promotion product shelf on which a product (e.g., cosmetics) is placed. On the fixture 30, advertising information (illustrations, photos, characters, etc.) of the product and a two-dimensional code (e.g., QR code (registered trademark)) including information for accessing the advertisement distribution device 20 (the above-mentioned URL) are written. The user device 10 can access (connect to) the advertisement distribution device 20 by scanning the two-dimensional code.

[0029] The fixture 30 receives an instruction to cause an object owned by the fixture 30 to perform a predetermined operation, the instruction having been transmitted by the advertisement distribution device 20, via the network 40. In response to receiving the instruction (i.e., in response to a user's touch on a specific AR content or AR target displayed on the display unit of the user device 10), the fixture 30 causes the object to perform a predetermined operation.

[0030] <User device configuration> 2 is a block diagram showing an example of a hardware configuration of a user device according to an embodiment of the present disclosure. The user device 10 may be realized by any computer or information processing device, such as a tablet, a smartphone, a notebook PC (Personal Computer), or a wearable device such as smart glasses or a head-mounted display.

[0031] 2, the user device 10 may include a storage device 11, a processing device 12, a user interface (UI) device 13, a communication device 14, an imaging device (photographing device) 15, and a bus 16. The storage device 11, the processing device 12, the UI device 13, the communication device 14, and the imaging device 15 may be connected to each other via the bus 16.

[0032] The programs or instructions for implementing the functions and processes of the user device 10 described above and below may be downloaded from some external device (e.g., a server) via a network, etc. Also, such programs or instructions may be provided from a removable storage medium such as a CD-ROM (Compact Disc Read Only Memory) or a flash memory.

[0033] Storage device 11 is realized by a random access memory (RAM), a flash memory, a hard disk drive, etc., and stores installed programs or instructions as well as files, data, etc. used in executing the programs or instructions. Storage device 11 may include a non-transitory storage medium.

[0034] The processing device 12 may be realized, for example, by a general-purpose processor or controller (circuit), or may be realized by a dedicated processor or controller (circuit). When realized by a general-purpose processor, the processing device 12 may be realized by one or more central processing units (CPUs), graphics processing units (GPUs), processing circuitry, etc., which may be composed of one or more processor cores. In this case, the processing device 12 executes the functions and processes of the user device 10 described above and below according to a program or instruction stored in the storage device 11, data such as parameters used to execute the program or instruction, etc. Such a program may be a program that causes the computer or processing device 12 to function as the user device 10 according to the present disclosure when executed by the computer or processing device 12. In addition, such a program may be a program that causes the computer or processing device 12 to execute a method related to advertisement delivery according to the present disclosure when executed by the computer or processing device 12.

[0035] The UI device 13 may be composed of input devices such as a keyboard, a mouse, a camera, a microphone, etc., output devices such as a display, a speaker, a headset, a printer, etc., and input / output devices such as a haptic device, a touch panel, etc., and realizes an interface between a user and the user device 10. For example, a user operates the user device 10 by operating a graphical user interface (GUI) displayed on a display or a touch panel using a keyboard, a mouse, etc.

[0036] The communication device 14 may be realized by various communication circuits that execute communication processes with external devices, the Internet, a LAN, a VPN, or other communication networks.

[0037] The imaging device 15 may be realized by including a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor. The imaging device 15 captures an image of a scene around the user device 10 to generate image data. The imaging device 15 outputs the generated image data to a communication control unit 101 and an image acquisition unit 102 described later, and stores the image data in the storage device 11 (an image storage unit 111 described later). The imaging device 15 generates image data including the above-mentioned two-dimensional code, a part or all of the fixture 30, etc.

[0038] The above-described hardware configuration of the user device 10 is merely an example, and the user device 10 according to the present disclosure may be realized by any other appropriate hardware configuration.

[0039] FIG. 3 is a block diagram illustrating an example of a functional configuration of a user device according to an embodiment of the present disclosure.

[0040] 3, the user device 10 includes a communication control unit 101, an image acquisition unit 102, a display data acquisition unit 103, a display control unit 104, an image storage unit 111, and a display data storage unit 112. The communication control unit 101, the image acquisition unit 102, the display data acquisition unit 103, and the display control unit 104 may be realized by the processing device 12 as hardware, as software (e.g., an application executed on a web browser or a dedicated application), or as a combination of hardware and software. The image storage unit 111 and the display data storage unit 112 may be realized by the storage device 11.

[0041] The communication control unit 101 acquires as an input an image including a two-dimensional code output by the imaging device 15, and connects to the advertisement distribution device 20 by accessing the URL included in the two-dimensional code via the communication device 14 (if the user permits access). Note that this process of the communication control unit 101 does not necessarily need to be realized via a web browser or a dedicated application, and may be realized via a general-purpose function of the user device 10.

[0042] The communication control unit 101 controls communication with the advertisement distribution device 20 by the image acquisition unit 102 and the display data acquisition unit 103 .

[0043] When the user inputs to the user device 10 via the UI device 13 to end image capture (photography) by the imaging device 15 in order to end advertisement display, the communication control unit 101 transmits image capture end information to the advertisement distribution device 20.

[0044] When a connection with the advertisement delivery device 20 is established by the communication control unit 101, the image acquisition unit 102 acquires, as an input, an image output by the imaging device 15. For example, the image acquisition unit 102 acquires a first image including a first object indicating that virtual content is to be superimposed. The image acquisition unit 102 transmits the acquired image to the advertisement delivery device 20 via the communication device 14. Note that the image acquisition unit 102 may transmit, together with the image, coordinate information when the image is displayed on the display device of the user device 10 to the advertisement delivery device 20.

[0045] The display data acquisition unit 103 acquires (receives) display data in which the AR content is or is not superimposed on the transmitted image from the advertisement distribution device 20 via the communication device 14. The display data acquisition unit 103 outputs the acquired display data to the display control unit 104 and stores it in the display data storage unit 112.

[0046] The display control unit 104 receives as input the display data output by the display data acquisition unit 103, generates a screen including the received display data, and displays the screen on a display device (display unit) included in the UI device 13. For example, the display control unit 104 displays on the display unit a second image in which virtual content associated with the first object is superimposed on the first image.

[0047] The display control unit 104 transmits to the advertising distribution device 20 coordinate information when the image is displayed on the display device of the user device 10, and, if the user touches the display device of the user device 10, the coordinates of the position touched by the user on the display device.

[0048] The image storage unit 111 stores image data generated by the imaging device 15 .

[0049] The display data storage unit 112 stores the display data acquired by the display data acquisition unit 103 .

[0050] <Configuration of the ad distribution device> 4 is a block diagram showing an example of a hardware configuration of an advertisement distribution device according to an embodiment of the present disclosure. The advertisement distribution device 20 may be realized by a calculation device (calculator, computer) such as a server or an information processing device.

[0051] 4, the advertisement distribution device 20 may include a storage device 21, a processing device 22, a UI device 23, a communication device 24, and a bus 25. The storage device 21, the processing device 22, the UI device 23, and the communication device 24 may be connected to each other via the bus 25.

[0052] The programs or instructions for implementing the functions and processes of the advertisement distribution device 20 described above and below may be downloaded from some external device (e.g., a server) via a network, etc. Also, such programs or instructions may be provided from a removable storage medium such as a CD-ROM or a flash memory.

[0053] Storage device 21 may be implemented by RAM, flash memory, a hard disk drive, etc., and stores installed programs or instructions as well as files, data, etc. used in executing the programs or instructions. Storage device 11 may include a non-transitory storage medium.

[0054] The processing device 22 may be realized by, for example, a general-purpose processor or controller (circuit), or may be realized by a dedicated processor or controller (circuit). When realized by a general-purpose processor, the processing device 22 may be realized by one or more CPUs, GPUs, processing circuits, etc., which may be composed of one or more processor cores. In this case, the processing device 22 executes the functions and processes of the advertisement distribution device 20 described above and below according to a program or instruction stored in the storage device 21, data such as parameters used to execute the program or instruction, etc. Such a program may be a program that causes the computer or processing device 22 to function as the advertisement distribution device 20 according to the present disclosure when executed by the computer or processing device 22. In addition, such a program may be a program that causes the computer or processing device 22 to execute a method related to advertisement distribution according to the present disclosure when executed by the computer or processing device 22.

[0055] The UI device 23 may be composed of input devices such as a keyboard, a mouse, a camera, a microphone, etc., output devices such as a display, a speaker, a headset, a printer, etc., and input / output devices such as a haptic device, a touch panel, etc., and realizes an interface between a user and the advertisement distribution device 20. For example, a user operates the advertisement distribution device 20 by operating a GUI displayed on a display or a touch panel using a keyboard, a mouse, etc.

[0056] The communication device 24 may be realized by various communication circuits that execute communication processes with external devices, the Internet, a LAN, a VPN, or other communication networks.

[0057] The above-described hardware configuration of the advertisement distribution device 20 is merely an example, and the advertisement distribution device 20 according to the present disclosure may be realized by another appropriate hardware configuration.

[0058] FIG. 5 is a block diagram illustrating an example of a functional configuration of an advertisement distribution device according to an embodiment of the present disclosure.

[0059] 5, the advertisement distribution device 20 includes a communication control unit 201, an image acquisition unit 202, a display data generation unit 203, a display data output unit 204, an event generation unit 205, a command output unit 206, a content data storage unit 211, an image storage unit 212, and a display data storage unit 213. The communication control unit 201, the image acquisition unit 202, the display data generation unit 203, the display data output unit 204, the event generation unit 205, and the command output unit 206 may be realized as hardware, software, or a combination of hardware and software by the processing device 12. The content data storage unit 211, the image storage unit 212, and the display data storage unit 213 may be realized by the storage device 21.

[0060] The communication control unit 201 establishes a connection with the user device 10 that has accessed the URL via the communication unit 24 (connects to the user device 10).

[0061] The communication control unit 201 controls communication with the user device 10 by the image acquisition unit 202 and the display data output unit 204, and communication with the fixture 30 by the command output unit 206.

[0062] The communication control unit 201 receives, via the communication unit 24, the shooting end information transmitted by the user device 10 (communication control unit 101).

[0063] The communication control unit 201 receives, via the communication device 24, coordinate information and touch position coordinates (when the user touches) transmitted by the user device 10 (display control unit 104) when an image is displayed on the display device of the user device 10. The communication control unit 201 outputs the received coordinate information and touch position coordinates (when the user touches) to the event generation unit 205.

[0064] When the imaging device 15 finishes imaging (photographing), the communication control unit 201 transmits, via the communication device 24 to the fixture 30, processing end information indicating the end of the method or processing according to the present disclosure.

[0065] The image acquiring unit 202 acquires (receives) an image transmitted by the user device 10 (image acquiring unit 102) via the communication device 24. For example, the image acquiring unit 202 acquires a first image including a first object indicating that virtual content is to be superimposed from the user device 10. The image acquiring unit 202 outputs the acquired image to the display data generating unit 203 and also stores the image in the image storage unit 212.

[0066] The display data generating unit 203 receives the image output by the image acquiring unit 202 as an input.

[0067] When the display data generating unit 203 recognizes (or identifies or detects) an AR target in an image, the display data generating unit 203 reads out, from the content data storage unit 211, AR content data associated with the recognized AR target.

[0068] The display data generating unit 203 calculates (local) feature amounts of the image, and compares the calculated feature amounts with feature amounts of AR targets stored in advance, thereby recognizing the AR target within the image.

[0069] For example, the AR target may be a flat surface such as an advertising surface on which an advertising photo or the like is attached, a curved surface (such as the side of a cone or cylinder), or a floor surface. The AR target may exist on a flat surface, a curved surface, or a floor surface as an advertising photo or the like. The AR target may also be the sky.

[0070] The display data generating unit 203 superimposes the read AR content on the received image data at a designated or predetermined position such as a recognition position where the AR target is recognized, a position near the recognition position, or a position a predetermined distance away from the recognition position, with reference to the association information described later, and generates the image data with the AR content superimposed as display data. For example, in response to recognizing a first object in a first image, the display data generating unit 203 generates a second image in which virtual content associated with the first object is superimposed on the first image.

[0071] On the other hand, if the display data generating unit 203 does not recognize the AR target in the image, it determines to use the received image data as display data.

[0072] The display data generating unit 203 outputs the display data (image data on which the AR content is superimposed or accepted image data) to the display data output unit 204, and also causes the display data storage unit 213 to store the same.

[0073] The display data output unit 204 receives the display data output by the display data generation unit 203 as an input, and transmits (outputs) the display data to the user device 10 via the communication device 24. For example, the display data output unit 204 transmits a second image generated by the display data generation unit 203 to the user device 10.

[0074] The event generating unit 205 receives as input the coordinate information and touch position coordinates output by the communication control unit 201. The event generating unit 205 determines whether or not the user has touched a specific displayed AR content or AR target based on the received coordinate information and touch position coordinates and the display image data most recently transmitted to the user device 10. The event generating unit 205 is an example of a touch determination unit according to the present disclosure.

[0075] When the event generating unit 205 determines that the user has touched a specific AR content or an AR target, the event generating unit 205 generates a predetermined event. For example, an event for a specific AR content or an AR target generated by the event generating unit 205 may be an object of the fixture 30 performing a predetermined action (for example, a light flashing or a door opening).

[0076] When an event is to be generated, the event generating unit 205 outputs an event generation instruction to the command output unit 206. The event generation instruction includes information indicating an object owned by the fixture 30 and information indicating a predetermined action (see the "event content" field in FIG. 9D).

[0077] The command output unit 206 receives as input the event occurrence instruction output by the event generation unit 205. The command output unit 206 transmits (outputs) a command corresponding to the event (a command for causing an object possessed by the fixture 30 to perform a predetermined operation) to the fixture 30 via the communication device 24.

[0078] The content data storage unit 211 stores AR content data (including AR content for events). The AR content data may be any content data that can be displayed via a display device, such as a still image, a moving image, an animation effect, a three-dimensional (3D) object, a 3D animation, or text.

[0079] The content data storage unit 211 also stores association information (for example, in a table format) that associates AR targets with AR content (for example, see FIG. 9A). This allows the display data generation unit 203, when recognizing an AR target in an image, to read out the AR content associated with the recognized AR target from the content data storage unit 211 by referring to the association information.

[0080] The image storage unit 212 stores the image data acquired by the image acquisition unit 202 .

[0081] The display data storage unit 213 stores the display data generated by the display data generation unit 203 and image data accepted by the display data generation unit 203 and used as the display data.

[0082] <Furniture composition> Fig. 6 is an external perspective view showing an example of a fixture according to an embodiment of the present disclosure. As shown in Fig. 6, fixture 30 includes advertising surface 31, advertising area 32, advertising distribution device access information area 33, and product display area 34. Fixture 30 also includes products 35A-35C, lights 36A-36C, and door 37. Fixture 30 further includes an information processing device (not shown).

[0083] The advertising surface 31 is a surface on which various advertising information (illustrations, photos, text, etc.) of the product is written. In this embodiment, the advertising surface 31 is a flat surface. However, the advertising surface 31 may be a curved surface.

[0084] Advertisement information is written in the advertisement area 32. There may be a plurality of advertisement areas 32.

[0085] In the ad-delivery device access information area 33, a two-dimensional code including a URL that hosts an ad-delivery web service, which is information for accessing the ad-delivery device 20, is written.

[0086] Products 35A to 35C (lipstick in FIG. 6) are displayed in the product display area 34. The products 35A to 35C are placed on lights 36A to 36C, respectively.

[0087] When the door 37 is opened, a tester (not shown) placed inside is exposed.

[0088] In this embodiment, the advertising area 32 and the products 35A to 35C are AR targets. However, a part or all of one or more advertising areas 32 may be an AR target, the entire fixture 30 may be an AR target, or a part or all of the advertising surface 31 may be an AR target. The fixture 30, the advertising surface 31, the advertising area 32, the products 35A to 35C, the lights 36A to 36C, and the door 37 are examples of real-world objects or objects.

[0089] FIG. 7 is a block diagram illustrating an example of a hardware configuration of an information processing device included in a fixture according to an embodiment of the present disclosure.

[0090] 7, the fixture 30 (information processing device) may include a storage device 301, a processing device 302, a UI device 303, a communication device 304, and a bus 305. The storage device 301, the processing device 302, the UI device 303, and the communication device 304 may be connected to each other via the bus 305. In addition, some of the objects (e.g., the lights 36A to 36C, the door 37) included in the fixture 30 described above may be connected to the processing device 302 and the like via the bus 305.

[0091] The programs or instructions for implementing the functions and processes of the fixture 30 described above and below may be downloaded from some external device (e.g., a server) via a network, etc. Also, such programs or instructions may be provided from a removable storage medium such as a CD-ROM or a flash memory.

[0092] Storage device 301 is implemented by RAM, flash memory, a hard disk drive, etc., and stores installed programs or instructions as well as files, data, etc. used in the execution of the programs or instructions. Storage device 301 may include a non-transitory storage medium.

[0093] The processing device 302 may be realized, for example, by a general-purpose processor or controller (circuit), or may be realized by a dedicated processor or controller (circuit). When realized by a general-purpose processor, the processing device 302 may be realized by one or more CPUs, GPUs, processing circuits, etc., which may be composed of one or more processor cores. In this case, the processing device 302 executes the functions and processing of the fixture 30 described above and below according to a program or instruction stored in the storage device 301, data such as parameters used to execute the program or instruction, etc. Such a program may be a program that causes the computer or processing device 302 to function as the fixture 30 according to the present disclosure when executed by the computer or processing device 302. In addition, such a program may be a program that causes the computer or processing device 302 to execute a method related to advertisement delivery according to the present disclosure when executed by the computer or processing device 302.

[0094] The UI device 303 may be composed of input devices such as a keyboard, a mouse, a camera, a microphone, etc., output devices such as a display, a speaker, a headset, a printer, etc., and input / output devices such as a haptic device, a touch panel, etc., and realizes an interface between the user and the fixture 30. For example, the user operates the fixture 30 by operating the GUI displayed on the display or touch panel using a keyboard, a mouse, etc.

[0095] The communication device 304 may be realized by various communication circuits that execute communication processes with external devices, the Internet, a LAN, a VPN, or other communication networks.

[0096] The hardware configuration of the fixture 30 described above is merely an example, and the fixture 30 according to the present disclosure may be realized by other appropriate hardware configurations.

[0097] FIG. 8 is a block diagram illustrating an example of a functional configuration of an information processing device included in the fixture according to an embodiment of the present disclosure.

[0098] 8, the fixture 30 (information processing device) includes a communication control unit 311, a command acquisition unit 322, and an operation execution control unit 323. The communication control unit 311, the command acquisition unit 322, and the operation execution control unit 323 may be realized by the processing device 302 as hardware, software, or a combination of hardware and software.

[0099] The communication control unit 311 receives, via the communication device 304, the processing end information transmitted by the advertisement distribution device 20 (the communication control unit 201).

[0100] The communication control unit 311 controls communication between the command acquisition unit 312 and the advertisement distribution device 20 .

[0101] The command acquisition unit 312 acquires (receives) a command transmitted by the advertisement distribution device 20 (command output unit 206) via the communication device 304. The command acquired by the command acquisition unit 312 is a command for causing an object included in the fixture 30 to execute a predetermined operation. The command acquisition unit 312 outputs the acquired command to the operation execution control unit 313.

[0102] The action execution control unit 313 receives as input the command output by the command acquisition unit 312, and causes the object included in the fixture 30 to execute a predetermined action in accordance with the received command. As a result, the object included in the fixture 30 executes the predetermined action.

[0103] <Access to ad distribution device by user device> FIG. 9A is a diagram for explaining an example of a display on a user device and an example of access by the user device to an advertisement distribution device according to an embodiment of the present disclosure.

[0104] As shown in FIG. 9A, for example, when a user turns on the camera function of the user device 10 and takes a picture of the advertisement distribution device access information area 33 of the fixture 30, the user device 10 can access the URL included in the two-dimensional code and connect to the advertisement distribution device 20.

[0105] <Displaying data for display on user device> 9B and 9C are diagrams for explaining a display example on a user device according to an embodiment of the present disclosure. FIG. 10A is a diagram showing a data table as an example of association information between AR content and an AR target according to an embodiment of the present disclosure. FIG. 10B is a diagram showing a data table as an example of association information between AR content and an event occurring in response to a touch on the AR content according to an embodiment of the present disclosure. FIG. 10C is a diagram showing a data table as an example of association information between an AR target and an event occurring in response to a touch on the AR target according to an embodiment of the present disclosure. FIG. 10D is a diagram showing an exemplary data table showing event contents according to an embodiment of the present disclosure.

[0106] As shown in FIG. 10A, the data table 900 has a “content ID” field 901 , a “target ID” field 902 , and a “content description” field 903 .

[0107] In the “content ID” field 901, information for identifying the AR content to be superimposed on the image transmitted from the user device 10 is stored.

[0108] Information for identifying an AR target is stored in the “target ID” field 902. Note that the target ID “t1” is information for identifying the product 35A, the target ID “t2” is information for identifying the product 35B, the target ID “t3” is information for identifying the product 35C, and the target ID “t11” is information for identifying the advertisement area 32.

[0109] The "content description" field 903 stores a description of the corresponding content.

[0110] The advertisement distribution device 20 can refer to a data table such as the data table 900 to generate display data on which the AR content is superimposed, and cause the user device 10 to display the display data.

[0111] As shown in FIG. 10B, the data table 920 has a “content ID” field 921 and an “event ID” field 922.

[0112] The "content ID" field 921 stores information for identifying the AR content.

[0113] The “event ID” field 922 stores information for identifying an event that occurs when the user touches the corresponding AR content displayed on the display device of the user device 10.

[0114] As shown in FIG. 10C, the data table 940 has a “target ID” field 941 and an “event ID” field 942 .

[0115] The "target ID" field 941 stores information for identifying the AR target.

[0116] The “Event ID” field 942 stores information that identifies an event that occurs when the user touches a corresponding AR target displayed on the display device of the user device 10 .

[0117] As shown in FIG. 10D, the data table 960 has an “event ID” field 961 and an “event content” field 962.

[0118] The "event ID" field 961 stores information for identifying the event.

[0119] The "event content" field 962 stores the corresponding event content.

[0120] By referring to data tables such as data tables 920, 940, and 960, the advertising distribution device 20 can generate an event in which an object possessed by the fixture 30 performs a predetermined action in response to a user's touch on a specific AR content or AR target.

[0121] 9B and 9C, it is assumed that the user device 10 is connected to the advertisement delivery device 20 and there is no event corresponding to the content ID "c11". Images generated by the imaging device 15 of the user device 10 are sequentially transmitted from the user device 10 to the advertisement delivery device 20, and images (display data) with or without AR content superimposed on the images generated by the imaging device 15 are sequentially transmitted from the advertisement delivery device 20 to the user device 10 and displayed on the user device 10. At this time, since the products 35A-35C having the target IDs "t1"-"t3", respectively, and the advertising area 32 (a person's upper body photograph) having the target ID "t11" are AR targets, the advertisement distribution device 20 recognizes the products 35A-35C and the advertising area 32 in the image as AR targets, and superimposes the AR content by reading out the t1 selection buttons to t3 selection buttons having the content IDs "c1"-"c3" corresponding to the products 35A-35C, respectively, and the promotional video having the content ID "c11" corresponding to the advertising area 32 from the content data storage unit 211. FIG. 9A illustrates a situation in which the user device 10 displays display data (an image generated by the imaging device 15) before the advertisement distribution device 20 recognizes the products 35A-35C and the advertising area 32, which are AR targets. 9B illustrates a state in which the user device 10 displays display data (AR contents 91A-91C (t1 selection buttons-t3 selection buttons) having content IDs "c1"-"c3", respectively, and AR content 91D (promotional video) corresponding to content ID "c11" are superimposed on an image generated by the imaging device 15) after the products 35A-35C and the advertising area 32, which are AR targets, are recognized by the advertising distribution device 20. As a result, the user device 10 displays the AR contents associated with the products 35A-35C and the advertising area 32, superimposed on the upper left of the image near the products 35A-35C in the image.

[0122] Here, when the user touches the AR content 91A displayed on the display device of the user device 10, the user device 10 transmits touch position coordinates to the advertisement distribution device 20. Then, the advertisement distribution device 20 determines that the user has touched the AR content 91A based on the received touch position coordinates. The advertisement distribution device 20 refers to the data tables 920 and 960 and generates an event with the event ID "e1" corresponding to the content ID "c1". As a result, the advertisement distribution device 20 transmits a command to the fixture 30 to turn on the light 36A. As a result, the light 36A turns on. The same applies to the AR contents 91B and 91C.

[0123] Furthermore, when the user touches the advertising area (AR target) 32 displayed on the display device of the user device 10, the user device 10 transmits touch position coordinates to the advertising distribution device 20. Then, the advertising distribution device 20 determines that the user has touched the displayed advertising area (AR target) based on the received touch position coordinates. The advertising distribution device 20 refers to the data tables 940 and 960 and generates an event with the event ID "e11" corresponding to the target ID "t11". As a result, the advertising distribution device 20 transmits a command to the fixture 30 to open the door 37. As a result, the door 37 opens.

[0124] By having the above-mentioned configuration, the user device 10 acquires a first image including a first object indicating that virtual content is to be overlaid, and displays on the display unit a second image in which the virtual content associated with the first object is overlaid on the first image.

[0125] By having the above-mentioned configuration, the advertisement distribution device 20 receives a first image from the user device 10, and in response to recognizing a first object in the first image, generates a second image and transmits the second image to the user device 10.

[0126] The fixture 30 causes the second object to perform a predetermined action in response to a touch on the first object or virtual content displayed on the display unit of the user device 10.

[0127] As described above, the advertisement delivery system 1 displays a specific AR object superimposed on an image including a specific first object possessed by the fixture 30, and causes a specific second object possessed by the fixture 30 to execute a predetermined action in response to a user (customer) touching the displayed specific first object or specific AR object. This makes it possible to increase appeal to the user (customer) through interactive communication with the user (customer).

[0128] <Operation of the ad distribution system> Next, an operation example of the advertisement delivery system 1 will be described with reference to FIGS. 11A to 13B.

[0129] 11A is a flowchart illustrating an example of an operation of a user device according to an embodiment of the present disclosure. The example of the operation illustrated in FIG. 11A relates to access to the advertisement distribution device 20 by the user device 10.

[0130] In step S101, the user device 10 (communication control unit 101) connects to the advertisement distribution device 20 by accessing a URL included in a two-dimensional code in an image of the furniture 30 generated by the imaging device 15. Then, the flow ends.

[0131] Fig. 11B is a flowchart showing an example of an operation of a user device according to an embodiment of the present disclosure. The example of the operation shown in Fig. 11B is related to the transmission of an image by the user device 10 to the advertisement distribution device 20. In the example of the operation shown in Fig. 11B, it is assumed that the user device 10 is connected to the advertisement distribution device 20.

[0132] In step S121, the user device 10 (image acquisition unit 102) acquires an image of the fixture 30 or the like from the imaging device 15.

[0133] In step S122, the user device 10 (image acquisition unit 102) transmits the captured image of the fixture 30 or the like to the advertisement distribution device 20. Then, the flow ends.

[0134] The process shown in FIG. 11B is repeatedly executed until the user inputs an instruction to end image capture to the user device 10 (see FIG. 11D).

[0135] 11C is a flowchart showing an example of an operation of a user device according to an embodiment of the present disclosure. The example of the operation shown in FIG. 11C relates to the reception and display of display data from the advertisement distribution device 20 and the transmission of touch position coordinates to the advertisement distribution device 20 by the user device 10. In the example of the operation shown in FIG. 11C, it is assumed that the user device 10 is connected to the advertisement distribution device 20.

[0136] In step S141, the user device 10 (the display data acquisition unit 103) receives display data from the advertisement distribution device 20. The display data is an image in which AR content is superimposed on an image transmitted from the user device 10 to the advertisement distribution device 20, or an image transmitted from the user device 10 to the advertisement distribution device 20.

[0137] In step S142, the user device 10 (display control unit 104) generates a screen including the display data, and causes a display device (display unit) included in the user device 10 to display the screen including the display data.

[0138] In step S143, the user device 10 (display control unit 104) determines whether or not the user has touched the screen (display unit) before the user device 10 receives the next display data from the advertisement distribution apparatus 20, for example.

[0139] If the user touches the screen (display unit) (step S143; YES), in step S144, the user device 10 (display control unit 104) transmits the coordinates of the position where the user touched the screen (display unit) (touch position coordinates) to the advertisement distribution device 20. Then, the flow ends.

[0140] On the other hand, if the user does not touch the screen (display unit) (step S143; NO), the flow ends.

[0141] The process shown in FIG. 11C is repeatedly executed until the user inputs an instruction to end image capture to the user device 10 (see FIG. 11D).

[0142] 11D is a flowchart showing an example of an operation of a user device according to an embodiment of the present disclosure. The example of operation shown in FIG. 11D relates to the end of the method related to advertisement distribution according to the present disclosure. In the example of operation shown in FIG. 11D, it is assumed that the user device 10 is connected to the advertisement distribution device 20.

[0143] In step S161, the user device 10 (communication control unit 101) transmits shooting end information to the advertisement distribution device 20 in response to a user's input to end shooting to the user device 10. Then, the flow ends. This ends the method related to advertisement distribution according to the present disclosure.

[0144] Fig. 12A is a flowchart showing an example of an operation of the advertisement distribution device according to an embodiment of the present disclosure. The example of the operation shown in Fig. 12A corresponds to the example of the operation shown in Fig. 11A, and relates to a connection between the advertisement distribution device 20 and the user device 10 (and the fixture 30).

[0145] In step S201, the advertisement distribution device 20 (the communication control unit 201) connects to the user device 10 in response to the user device 10 accessing a URL.

[0146] In step S202, the advertisement distribution device 20 (communications control unit 201) connects to the fixture 30. Then, the flow ends.

[0147] 12B and 12C are flowcharts showing an example of the operation of the advertisement delivery device according to an embodiment of the present disclosure. The example of the operation shown in FIG. 12B and FIG. 12C corresponds to the example of the operation shown in FIG. 11B to FIG. 11D, and relates to the reception of an image from the user device 10, the transmission of display data to the user device 10, the occurrence of an event, the end of the method for advertisement delivery according to the present disclosure, etc. (and the transmission of a command to the fixture 30) by the advertisement delivery device 20. In the example of the operation shown in FIG. 12B and FIG. 12C, it is assumed that the advertisement delivery device 20 is connected to the user device 10.

[0148] In step S251, the advertisement distribution apparatus 20 (the communication control unit 201) determines whether or not image-taking by the user device 10 has ended by determining whether or not image-taking end information has been received.

[0149] If the shooting has not ended (Step S251; NO), the advertisement distribution device 20 (image acquisition unit 202) acquires the image transmitted by the user device 10 in Step S252.

[0150] In step S253, the advertisement distribution device 20 (the display data generating unit 203) determines whether or not a recognizable AR target is present in the image.

[0151] If there is no recognizable AR target in the image, that is, if the AR target is not recognized (step S253; NO), in step S254, the advertisement distribution device 20 (display data generation unit 203) determines whether or not AR content exists in the previous display data.

[0152] If the AR content does not exist (step S254; NO), in step S255, the advertisement distribution device 20 (display data generation unit 203) determines to use the image as display data without superimposing the AR content. Then, the flow proceeds to step S258. Note that step S255 is also executed when the image acquired in step S252 is the image initially transmitted by the user device 10.

[0153] On the other hand, if the AR content exists (step S254; YES), in step S256, the advertisement distribution device 20 (display data generation unit 203) decides to clear the AR content and use the image as display data. Then, the flow proceeds to step S258.

[0154] In step S253, if there is a recognizable AR target in the image, that is, if the AR target is recognized (step S253; YES), in step S257, the advertisement distribution device 20 (display data generation unit 203) refers to the association information and generates display data in which the AR content associated with the AR target is superimposed on the image. Then, the flow proceeds to step S258.

[0155] In step S258, the advertisement-delivery device 20 (the display data output unit 204) outputs (transmits) the display data to the advertisement-delivery device 20.

[0156] In step S259, the advertisement distribution device 20 (the communication control unit 201) determines whether or not the touch position coordinates have been received.

[0157] If the touch position coordinates have not been received (step S259; NO), the flow returns to step S251.

[0158] On the other hand, when the touch position coordinates are received (step S259; YES), in step S260, the advertisement distribution device 20 (event generation unit 205) determines whether or not the user has touched a specific AR content or an AR target, based on the touch position coordinates.

[0159] If the user is not touching a specific AR content or AR target (step S260; NO), the flow returns to step S251.

[0160] On the other hand, if the user touches a specific AR content or an AR target (step S260; YES), in step S261, the advertisement distribution device 20 (event generation unit 205) generates an event for the specific AR content or the AR target, and the advertisement distribution device 20 (command output unit 206) transmits a corresponding command to the fixture 30. Then, the flow returns to step S251.

[0161] If the shooting has ended (Step S251; YES), the advertisement distribution device 20 (communications control unit 201) transmits processing end information to the fixture 30 in Step S262.

[0162] In step S263, the advertisement distribution device 20 (image acquisition unit 202, etc.) clears all processing (for example, deletes images from the image storage unit 212, etc.), and the flow then ends.

[0163] Fig. 13A is a flowchart showing an example of the operation of the fixture according to an embodiment of the present disclosure. The example of the operation shown in Fig. 13A corresponds to the example of the operation shown in Fig. 12A, and relates to the connection between the fixture 30 and the advertisement distribution device 20.

[0164] In step S301, the fixture 30 (the communication control unit 311) connects to the advertisement distribution device 20 (in response to access to the URL by the user device 10).

[0165] Fig. 13B is a flowchart showing an example of the operation of the fixture according to an embodiment of the present disclosure. The example of operation shown in Fig. 13B corresponds to the example of operation shown in Fig. 12B and Fig. 12C, and relates to the reception of a command from the advertisement distribution device 20 by the fixture 30, the control of the execution of an operation according to the command, and the end of the method for advertisement distribution according to the present disclosure. In the example of operation shown in Fig. 13B, the fixture 30 is connected to the advertisement distribution device 20.

[0166] In step S351, the fixture 30 (communications control unit 311) determines whether or not processing has ended (whether or not shooting by the user device 10 has ended) by determining whether or not processing end information has been received.

[0167] If the processing has not ended (Step S351; NO), in Step S352, the fixture 30 (command acquisition unit 312) determines whether or not the command transmitted by the advertisement distribution device 20 has been received.

[0168] If no command has been received (step S352; NO), the flow returns to step S351.

[0169] On the other hand, if a command has been received (step S352; YES), in step S353, the fixture 30 (the action execution control unit 313) causes the object possessed by the fixture 30 to execute an action in accordance with the command. Then, the flow returns to step S351.

[0170] If the processing is completed (step S351; YES), in step S354, the fixture 30 (the operation execution control unit 313, etc.) clears all processing (for example, closes the door 37, etc.), and the flow ends.

[0171] <Modification> Modifications of the above embodiment will now be described.

[0172] [Variation 1] In the above embodiment, an example of recognizing an AR target in an image by so-called markerless AR has been described, but the present disclosure is not limited to this example. For example, the AR target may be recognized by the display data generating unit 203 using an AR marker, which is a pattern image that serves as a marker for specifying a position where the AR content is to be displayed.

[0173] [Variation 2] In the above embodiment, an example has been described in which the user device 10 accesses the advertisement distribution device 20 by reading a URL included in a two-dimensional code in image data generated by the imaging device 15, but the present disclosure is not limited to this example. For example, if the user device 10 supports RFID (radio frequency identifier) ​​such as NFC (near field communication), an RF tag storing a URL may be provided on the fixture 30, and the user device 10 may access the advertisement distribution device 20 (URL) by reading the RF tag using the RFID. Also, for example, the user device 10 may access the advertisement distribution device 20 (URL) in response to a user inputting a URL into the user device 10.

[0174] [Variation 3] In the above embodiment, an example has been described in which the advertisement distribution device 20 generates display data in which virtual content is superimposed on an image including a real-world object, but the present disclosure is not limited to this example. For example, the user device 10 may have a function (the same or similar functional unit as described above) provided in the advertisement distribution device 20 in the form of an application. That is, the user device 10 may generate and display display data in which virtual content associated with a real-world object is superimposed on an image including the real-world object, and transmit a command to the fixture 30 in response to a user's touch on the displayed real-world object or virtual content. The command may be transmitted from the user device 10 to the fixture 30 via the advertisement distribution device 20, or may be transmitted directly from the user device 10 to the fixture 30.

[0175] [Variation 4] In the above embodiment, an example has been described in which the control objects, the execution of which is controlled, are the lights 36A to 36C and the door 37, but the present disclosure is not limited to this example. For example, the control object may be a display (UI device 303) included in the fixture 30, and the predetermined action may be to cause the display to display special content.

[0176] [Other variations] The names of the above-mentioned devices (user device 10, advertisement distribution device 20, etc.) and the functional units constituting the devices (communication control unit 101, image acquisition unit 102, etc.) are merely examples and may be changed as appropriate. Furthermore, two or more of the functional units constituting the devices may be combined and integrated, or one functional unit may be divided into functional units. Furthermore, the transmission function and / or reception function possessed by each functional unit may be summarized as a notification unit, a notified unit, a communication unit, a transmission / reception unit, a transmission unit, a reception unit, etc.

[0177] The order of steps in the flowcharts shown in the figures and described above is merely an example and may be rearranged as appropriate, and some steps may be performed in parallel.

[0178] As described above, according to one aspect of the present disclosure, a virtual content is displayed superimposed on an image including a first object possessed by a device, and a second object possessed by the device is caused to perform a predetermined action in response to a touch on the displayed first object or the virtual content. This allows the second object to perform a predetermined action in response to a touch by a user (customer), thereby increasing appeal to customers through interactive communication with the customer.

[0179] Although the embodiments have been described above with reference to the drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art can come up with various modified or amended examples within the scope of the claims. It is understood that such modified or amended examples also belong to the technical scope of the present disclosure. In addition, each component or each item in the embodiment (including modified or amended examples) may be arbitrarily combined within the scope of the present disclosure. [Explanation of symbols]

[0180] 1. Advertising distribution system 10. User Devices 101 Communication control unit 102 Image acquisition unit 103 Display data acquisition unit 104 Display control unit 111 Image storage unit 112 Display data storage unit 20 Advertisement distribution device 201 Communication control section 202 Image acquisition unit 203 Display data generation unit 204 Display data output section 205 Event Generation Section 206 Command output section 211 Content data storage unit 212 Image storage unit 213 Display data storage unit 30 Furniture 31 Advertising space 32 Advertising Area 33 Ad distribution device access information area 34 Product display area 35A, 35B, 35C products 36A, 36B, 36C Light 37 Door 91A, 91B, 91C, 92 AR Content 311 Communication Control Unit 312 Instruction acquisition part 313 Action Execution Control Unit

Claims

1. A device having a first object indicating that virtual content is to be overlaid and a second object capable of performing a predetermined operation; an information terminal that acquires a first image including the first object and displays, on a display unit, a second image in which virtual content associated with the first object is superimposed on the first image; Equipped with the device causes the second object to execute the predetermined action in response to a touch on the first object or the virtual content displayed on the display unit; Content display system.

2. The content display system further includes an information processing device, The information terminal transmits the first image to the information processing device, the information processing device receives the first image from the information terminal, and in response to recognizing the first object in the first image, generates the second image and transmits the second image to the information terminal. The content display system according to claim 1 .

3. the information terminal transmits information regarding a position where the touch has been performed on the display unit to the information processing device; The information processing device includes: receiving information about the position where the touch was performed from the information terminal; determining that the touch is relative to the first object or the virtual content based on information regarding a location where the touch was made; In response to the determination, a command is transmitted to the device to cause the second object to execute the predetermined action. The content display system according to claim 2 .

4. The information terminal generates the second image in response to recognizing the first object in the first image. The content display system according to claim 1 .

5. The information terminal includes: determining that the touch is relative to the first object or the virtual content based on information regarding a location where the touch was made; In response to the determination, a command is transmitted to the device to cause the second object to execute the predetermined action. The content display system according to claim 4 .

6. an image acquisition unit that acquires a first image indicating that virtual content is to be superimposed and including a first object included in the device; a display data generating unit that generates a second image in which virtual content associated with the first object is superimposed on the first image in response to the recognition of the first object in the first image; a display control unit that causes the second image to be displayed on a display unit; a touch determination unit that determines that the touch is made on the first object or the virtual content based on information about a position where the touch is made on the display unit; a command output unit configured to transmit, in response to the determination, a command to the device for causing a second object included in the device to perform a predetermined action; An information terminal comprising:

7. an image acquisition unit that acquires a first image, which indicates that virtual content is to be superimposed and includes a first object that the device has, from an information terminal; a display data generating unit that generates a second image in which virtual content associated with the first object is superimposed on the first image in response to the recognition of the first object in the first image; a display data output unit that transmits the second image to the information terminal; a communication control unit that receives information about a position where a touch has been made on a display unit of the information terminal from the information terminal; a touch determination unit that determines that the touch is made on the first object or the virtual content based on information about a position where the touch is made; a command output unit configured to transmit, in response to the determination, a command to the device for causing a second object included in the device to perform a predetermined action; An information processing device comprising:

8. acquiring a first image including a first object on the device, the first image indicating that virtual content is to be overlaid; generating a second image in response to recognizing the first object in the first image, the second image having virtual content associated with the first object superimposed on the first image; displaying the second image on a display unit; determining that the touch is on the first object or the virtual content based on information about a position where the touch is made on the display unit; transmitting, in response to the determination, a command to the device to cause a second object possessed by the device to execute a predetermined action; A program for causing an information terminal to execute processing including the above.

9. acquiring, from an information terminal, a first image indicating that virtual content is to be superimposed and including a first object possessed by the device; generating a second image in response to recognizing the first object in the first image, the second image having virtual content associated with the first object superimposed on the first image; transmitting the second image to the information terminal; receiving information about a position where a touch has been made on a display unit of the information terminal from the information terminal; determining that the touch is relative to the first object or the virtual content based on information regarding a location where the touch is made; transmitting, in response to the determination, a command to the device to cause a second object possessed by the device to execute a predetermined action; A program for causing an information processing device to execute a process including the steps of:

Images