Event delivery program, event delivery system, and event delivery method

Abstract

To enable holding an event using augmented reality and to enable performing product sales or service provision in the event.SOLUTION: An event providing program causes a server to implement: an object display function to superimpose a first object being a virtual object on a real space and display it on a display device; an expression control function to control an expression which uses one or more virtual objects including the first object; and an authority acquiring information display function to display information for allowing a user to acquire predetermined authority (authority acquiring information) for the object, the event, or a commodity or a service related to the expression, on the display device.SELECTED DRAWING: Figure 3

Description

Technical Field

[0004] , , , ,

[0005] , , , ,

[0001] At least one embodiment of the present invention relates to an event providing program, an event providing system, and an event providing method.

Background Art

[0002] Patent Document 1 describes a method for using augmented reality together with a live event. The method includes steps in which a computer-controlled device receives a data stream corresponding to a live event. The data stream includes a live video. The live video includes live objects. The method includes steps in which the computer-controlled device receives an input from a user. The input from the user affects the behavior of virtual objects. The method includes steps in which the computer-controlled device presents a live event extended by virtual objects.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, if an event is held by displaying (AR display) an artist in a music live or an actor in a stage play as a character or an object so as to superimpose it on the real space, the live event itself can be carried out in an augmented space (hereinafter, augmented space) extended by using augmented reality. Further, if products and services related to the event are provided together with the event, it becomes possible to conduct merchandise sales and service provision on the augmented space for the participants of the event.

[0005] The objective of at least one embodiment of the present invention is to solve the above problems and enable the hosting of an event using augmented reality, and to enable the sale of goods and provision of services at said event. [Means for solving the problem]

[0006] From a non-limiting perspective, an event provision program according to one embodiment of the present invention provides a server with an object display function that causes a first object, which is a virtual object, to be superimposed on the real space and displayed on a display device; an expression control function that controls an expression using one or more virtual objects, including the first object; and an authorization acquisition information display function that causes the display device to display information (hereinafter referred to as "authorization acquisition information") that allows a user to acquire predetermined authorizations regarding the object, the event, or the expression.

[0007] In a non-limiting view, an event provision system according to one embodiment of the present invention is an event provision system comprising a communication network, a server, and a user terminal, and includes object display means for superimposing a first object, which is a virtual object, onto the real space and displaying it on a display device; expression control means for controlling an expression using one or more virtual objects, including the first object; and authority acquisition information display means for displaying on the display device an indication (hereinafter referred to as "authorization acquisition information") for allowing a user to acquire predetermined authority regarding the object, the event, or the expression.

[0008] From a non-limiting perspective, an event provision program according to one embodiment of the present invention provides a user terminal with an object display function that causes a first object, which is a virtual object, to be superimposed on the real space and displayed on a display device; an expression control function that controls an expression using one or more virtual objects, including the first object; and an authorization acquisition information display function that causes the display device to display an indication (hereinafter referred to as "authorization acquisition information") that allows the user to acquire predetermined authorizations for the object, the event, or the expression.

[0009] In a non-limiting view, an event provision method according to one embodiment of the present invention is an event provision method using a computer device, which includes an object display process that superimposes a first object, which is a virtual object, onto the real space and displays it on a display device; an expression control process that controls an expression using one or more virtual objects, including the first object; and an expression acquisition information display process that displays on the display device an indication (hereinafter referred to as "instruction acquisition information") that allows a user to acquire a predetermined authority regarding the object, the event, or the expression.

[0010] In a non-limiting view, an event provision method according to one embodiment of the present invention is an event provision method by an event provision system comprising a communication network, a server, and a user terminal, and includes an object display process that superimposes a first object, which is a virtual object, onto the real space and displays it on a display device; an expression control process that controls an expression using one or more virtual objects, including the first object; and an authorization acquisition information display process that displays on the display device an indication (hereinafter referred to as "authorization acquisition information") that allows a user to acquire predetermined authorization for the object, the event, or the expression. [Effects of the Invention]

[0011] Each embodiment of the present invention resolves one or more of the shortcomings. [Brief explanation of the drawing]

[0012] [Figure 1] This figure shows an example of the configuration of an event delivery system corresponding to at least one embodiment of the present invention. [Figure 2] This is a block diagram showing the configuration of a server corresponding to at least one embodiment of the present invention. [Figure 3] This flowchart shows an example of event provisioning processing corresponding to at least one embodiment of the present invention. [Figure 4] This is a block diagram showing the configuration of a server corresponding to at least one embodiment of the present invention. [Figure 5] This flowchart shows an example of event provisioning processing corresponding to at least one embodiment of the present invention. [Figure 6] This is a block diagram showing the configuration of a server corresponding to at least one embodiment of the present invention. [Figure 7] This flowchart shows an example of event provisioning processing corresponding to at least one embodiment of the present invention. [Figure 8] This is a block diagram showing the configuration of a server corresponding to at least one embodiment of the present invention. [Figure 9] This flowchart shows an example of event provisioning processing corresponding to at least one embodiment of the present invention. [Figure 10] This figure shows an example of the configuration of an event delivery system corresponding to at least one embodiment of the present invention. [Figure 11] This is a block diagram showing the configuration of an event delivery system corresponding to at least one embodiment of the present invention. [Figure 12] This flowchart shows an example of event provisioning processing corresponding to at least one embodiment of the present invention. [Figure 13] This is a block diagram showing the configuration of a user terminal corresponding to at least one embodiment of the present invention. [Figure 14] A flowchart showing an example of event provision processing corresponding to at least one embodiment of the present invention. [Figure 15] A block diagram showing the configuration of a server corresponding to at least one embodiment of the present invention. [Figure 16] A flowchart showing an example of event provision processing corresponding to at least one embodiment of the present invention. [Figure 17] A conceptual diagram showing an example of display by a display device corresponding to at least one embodiment of the present invention. [Figure 18] A conceptual diagram showing an example of display by a display device corresponding to at least one embodiment of the present invention. [Figure 19] A conceptual diagram showing an example of display by a display device corresponding to at least one embodiment of the present invention. [Figure 20] A conceptual diagram showing an example of display by a display device corresponding to at least one embodiment of the present invention. [Figure 21] A conceptual diagram showing an example of display by a display device corresponding to at least one embodiment of the present invention. [Figure 22] A conceptual diagram showing an example of display by a display device corresponding to at least one embodiment of the present invention. [Figure 23] A conceptual diagram showing an example of display by a display device corresponding to at least one embodiment of the present invention. [Figure 24] A conceptual diagram showing an example of display by a display device corresponding to at least one embodiment of the present invention.

Embodiments for Carrying Out the Invention

[0013] Hereinafter, examples of embodiments of the present invention will be described with reference to the drawings. The various components in each embodiment described below can be combined as appropriate, provided that no inconsistencies arise. Furthermore, some aspects described in one embodiment may be omitted in other embodiments. Also, operations and processes unrelated to the characteristic features of each embodiment may be omitted. Moreover, the order of the various processes constituting the various flows and sequences described below is not necessarily in any particular order, provided that no inconsistencies arise in the processing content.

[0014] [First Embodiment] An overview of the first embodiment of the present invention will be described below. In the following, an event provision program executed on a server will be used as an example to describe the first embodiment.

[0015] Figure 1 shows an example of the configuration of an event provision system corresponding to at least one embodiment of the present invention. The event provision system 100 comprises an event provision processing server 10 (server 10) and user terminals 20 used by users of the event provision system 100. User terminals 20A, 20B, and 20C are examples of user terminals 20. The configuration of the event provision system 100 is not limited thereto. For example, the event provision system 100 may be configured so that a single user terminal is used by multiple users. The event provision system 100 may also comprise multiple servers. For example, the event provision system 100 may further comprise other servers 40.

[0016] The event provision system 100 may further include a receptionist device 60. The receptionist device 60 is used by receptionist SPs who interact with users visiting, for example, a merchandise tent (see Figure 21) as described later. Specific examples of the receptionist device 60 include a server device, a client device, and user terminals as described later.

[0017] Server 10, user terminals 20, other servers 40, and responder devices 60 are each connected to a communication network 30, such as the Internet, in a manner that allows communication. The connections between Server 10, user terminals 20, other servers 40, and responder devices 60 via the communication network 30 may be wired or wireless. For example, multiple user terminals 20 may connect to the communication network 30 by performing data communication via wireless communication lines with base stations managed by a telecommunications carrier.

[0018] The event provision system 100, comprising a server 10 and a user terminal 20, realizes various functions for executing various processes in response to user operations.

[0019] Server 10 comprises a processor 11, memory 12, and storage device 13. The processor 11 is, for example, a central processing unit such as a CPU (Central Processing Unit) that performs various calculations and controls. If Server 10 is equipped with a GPU (Graphics Processing Unit), some of the various calculations and controls may be performed by the GPU. Server 10 uses the data read into memory 12 to perform various information processing with the processor 11 and stores the obtained processing results in storage device 13 as needed.

[0020] The storage device 13 functions as a storage medium for storing various types of information. The configuration of the storage device 13 is not particularly limited, but from the viewpoint of reducing the processing load on the user terminal 20, it is preferable that it be configured to store all the various types of information necessary for control performed by the event provision system 100. Examples of such configurations include HDDs and SSDs. However, the storage device for storing various types of information only needs to have a storage area that is accessible to the server 10, and may, for example, have a dedicated storage area outside the server 10. The server 10 is managed by the administrator of the event provision system 100, etc., and has various functions for providing information related to various processes to multiple user terminals 20. The server 10 is composed of an information processing device such as a game server capable of rendering AR display data.

[0021] The user terminal 20 consists of a communication terminal capable of providing the user with information services accompanied by AR display. Examples of such communication terminals include, for example, a mobile phone terminal 20A such as a smartphone, and AR glasses 20B and 20C. The communication terminal may also be a stationary game console, a portable game console, a personal computer, a gaming PC, a so-called wearable device, or a combination of multiple of the aforementioned communication terminals. The configuration of the user terminal that the event provision system 100 may include is not limited to these, and is acceptable as long as the user can see the AR display.

[0022] The user terminal 20 comprises a processor 21, memory 22, and storage device 23. The processor 21 is, for example, a central processing unit such as a CPU (Central Processing Unit) that performs various calculations and controls. If the user terminal 20 is equipped with a GPU (Graphics Processing Unit), some of the various calculations and controls may be performed by the GPU. The user terminal 20 uses the data read into the memory 22 to perform various information processing using the processor 21, and stores the obtained processing results in the storage device 23 as needed. The storage device 23 functions as a storage medium for storing various information.

[0023] The user terminal 20 is connected to the communication network 30 and is equipped with hardware and software for performing various processes by communicating with the server 10, other servers 40, or the responder device 60. Note that each of the multiple user terminals 20 may also be configured to communicate directly with one another without going through the server 10, other servers 40, or the responder device 60.

[0024] The user terminal 20 may have a built-in display device. The display device may be connected to the user terminal 20 wirelessly or via a wired connection. The display device displays images such as still images or videos. The display device may also display a game screen. The display device may display the appearance of the responder SP based on data acquired directly from the responder device 60 via the communication network 30, or further via the server 10, etc. The display device can also perform augmented reality (AR) display superimposed on real space. Examples of display devices include hologram display devices capable of displaying holograms, and projection devices that project images (including game screens) onto a screen, etc. In the example in Figure 1, the touch panel screen of the mobile phone terminal 20A, the AR glasses 20B, and the AR displays of 20C correspond to display devices.

[0025] The user terminal 20 may have a built-in input device. Alternatively, the input device may be connected to the user terminal 20 wirelessly or via a wired connection. The input device accepts user input. In response to user input, the processor in the server 10 or the processor in the user terminal 20 executes various control processes. Examples of input devices include the touch panel screen of a mobile phone terminal 20A, and a controller CTR wirelessly or via a wired connection to the AR glasses 20B. The camera on the user terminal 20 may also be considered an input device. The user performs operation input by gestures, such as moving their hand in front of the camera (gesture input).

[0026] In addition, the user terminal 20 may be equipped with other output devices such as speakers. These other output devices may output various types of information to the user. This information may include, for example, voice messages spoken by the service provider SP. The other output devices may output voice information indicating the contents of the products the user purchases or the services the user receives. Note that the other output devices are not limited to voice output devices and may output other types of information. Furthermore, other output devices using haptic technology may output tactile information.

[0027] The other server 40 comprises a processor 41, memory 42, and storage device 43. The processor 41 is, for example, a central processing unit such as a CPU (Central Processing Unit) that performs various calculations and controls. If the other server 40 is equipped with a GPU (Graphics Processing Unit), some of the various calculations and controls may be performed by the GPU. The other server 40 uses the data read into memory 42 to perform various information processing with the processor 41 and stores the obtained processing results in storage device 43 as needed.

[0028] The storage device 43 functions as a storage medium for storing various types of information. The configuration of the storage device 43 is not particularly limited. The storage device for storing various types of information only needs to have a storage area that is accessible to other servers 40, and for example, it may be configured to have a dedicated storage area outside of other servers 40. Other servers 40 may be managed by the administrator of the event provision system 100 or by other persons, and have various functions for providing information related to various processes to multiple user terminals 20. Other servers 40 may be, for example, a server that manages the manufacturing and delivery of physical goods. Other servers 40 may also be a server that manages the distribution of videos related to live events.

[0029] The responder device 60 comprises a processor 61, memory 62, and storage device 63. The processor 61 is, for example, a central processing unit such as a CPU (Central Processing Unit) that performs various calculations and controls. If the responder device 60 is equipped with a GPU (Graphics Processing Unit), some of the various calculations and controls may be performed by the GPU. The responder device 60 uses the data read into the memory 62 to perform various information processing using the processor 61, and stores the obtained processing results in the storage device 63 as needed. The storage device 63 functions as a storage medium for storing various information.

[0030] The responder device 60 is connected to the communication network 30 and includes hardware and software for performing various processes by communicating with the server 10, the user terminal 20, or other servers 40.

[0031] The customer service device 60 includes a camera 64 and an input device 65. The camera 64 captures images of a customer service provider SP, such as a salesperson at a merchandise tent (see Figure 21). The captured image data, which includes the image of the customer service provider SP, is used in the cash register area (described later) to allow the customer service provider SP to interact with the user when the user purchases goods. The input device 65 receives operation input from the customer service provider SP, etc. In response to the operation input from the customer service provider SP, etc., the processor 61 of the customer service device 60 executes various control processes. Examples of input devices 65 include a keyboard, mouse, and microphone for voice operation. The camera 64 of the customer service device 60 can also be considered an input device. In this case, the customer service provider SP performs operation input by gestures such as moving their hands in front of the camera 64 (gesture input).

[0032] The responder device 60 includes a display device 66. The display device 66 may be built into the responder device 60. The display device 66 may be connected to the responder device 60 wirelessly or via a wired connection. The display device 66 displays images such as still images or videos. The display device 66 displays the image of the user purchasing a product or receiving a service. The display device 66 may also display information indicating the contents of the product the user is purchasing or the contents of the service the user is receiving.

[0033] In addition, the responder device 60 may be equipped with other output devices such as speakers. These other output devices output information relating to users using the merchandise tent, etc. This user-related information may include, for example, voices spoken by users who purchase goods or receive services. These other output devices may output voice information indicating the contents of the goods purchased by the user or the services received by the user. Note that these other output devices are not limited to voice output devices and may output other types of information such as smell information, taste information, and tactile information.

[0034] Figure 2 is a block diagram showing the configuration of a server corresponding to at least one embodiment of the present invention. Server 10A, an example of the configuration of Server 10, comprises at least an object display unit 101, a representation control unit 102, and an authorization acquisition information display unit 103. The processor in Server 10A refers to an event provision program held (installed) in a storage device and executes that program to functionally realize the object display unit 101, the representation control unit 102, and the authorization acquisition information display unit 103.

[0035] The object display unit 101 has the function of superimposing a first object, which is a virtual object, onto the real space and displaying it on the display device. The expression control unit 102 has the function of controlling the expression using one or more virtual objects, including the first object. The authorization acquisition information display unit 103 has the function of displaying information (authorization acquisition information) on the display device that allows the user to acquire predetermined authorizations for goods or services related to an object, event, or expression.

[0036] Next, the program execution process in the first embodiment of the present invention will be described. Figure 3 is a flowchart showing an example of event provisioning processing corresponding to at least one embodiment of the present invention.

[0037] The object display unit 101 superimposes a first object, which is a virtual object, onto the real space and displays it on the display device (St11). The expression control unit 102 controls the expression using one or more virtual objects, including the first object (St12). The authorization acquisition information display unit 103 displays information (authorization acquisition information) on the display device that allows the user to acquire predetermined authorizations for goods or services related to the object, event, or expression (St13).

[0038] A virtual object refers to an object that can be output to a virtual space rather than the real world. For example, a virtual chair or car represented by 3D modeling data obtained by scanning the shape of an object in the real world, such as a chair or car, falls under the category of a virtual object. Also, 3D modeling data representing living beings such as humans, animals, plants, and monsters also falls under the category of a virtual object. A virtual object may also be a virtual representation of a fictional subject, such as a fairy or a dragon. Note that the data dimensions used to represent a virtual object are not limited to three dimensions. For example, a 2D data representation of such a subject also falls under the category of a virtual object.

[0039] Real space refers to the space in the real world. Virtual space, which extends real space, refers to a virtual space (AR space) that can be superimposed and output onto real space.

[0040] Displaying a virtual object on a display device means outputting display data for displaying a virtual object to a device equipped with a display device (e.g., a user terminal 20) or a display device. Displaying a virtual object on a display device may also mean outputting a command for displaying a virtual object to a device that already possesses display data (e.g., a user terminal 20 capable of generating or storing display data) or a display device.

[0041] Expression means presenting some kind of information to the recipient in an external form, such as singing, dancing, acting, reciting poetry, playing a musical instrument, giving a lecture, running, jumping, or crying. Controlling an expression using a virtual object may include controlling the actions of the object, such as making the object sing, dance, act, recite poetry, play a musical instrument, give a lecture, run, jump, or cry. Furthermore, controlling an expression using a virtual object may also mean conveying some kind of information to the recipient through one or more virtual objects, such as making five virtual cars drive in a circle to represent the Olympic rings.

[0042] An event refers to a venue where information is presented externally to the audience, such as a live music event, a theatrical performance, or a seminar.

[0043] A commodity refers to an item that is the subject of a transaction. A service refers to the provision of some kind of benefit by one entity to another entity. Regarding whether a virtual object, which is essentially data, should be interpreted as a commodity or as the subject of a service, either interpretation is acceptable.

[0044] The prescribed rights (hereinafter abbreviated as "prescribed rights") relating to goods or services that are objects, events, or representations include, for example, the right to own, borrow, or use a physical object in the real world if the goods are physical objects in the real world. If the goods are data, the prescribed rights include the right to acquire, copy, modify, or use this data. For services, the prescribed rights include the right to receive the service.

[0045] As non-restrictive examples, specific examples of the prescribed authority are given. - The right to purchase or borrow physical items (e.g., figurines) that are modeled after the virtual characters appearing in the event. - The right to purchase or borrow physical items of clothing worn by the virtual characters appearing in the event. - The right to purchase or borrow physical items of possessions such as guitars owned by the virtual characters appearing in the event. - The right to purchase or borrow physical items corresponding to the stage equipment appearing in the event (including both virtual and real-world stage equipment). - The right to acquire or use data representing the virtual characters appearing in the event, the clothing worn by those characters, or the virtual items owned by those characters. - The right to add sound effects to physical items or virtual objects appearing in the event, such as changing the engine sound of a virtual car. - The right to purchase or borrow physical items related to the event, such as official event T-shirts and towels. - The right to purchase or borrow physical media such as CDs and DVDs containing music used in the event. The right to purchase or borrow physical media such as CDs and DVDs containing music related to the event or expression (for example, music provided by other artists belonging to the same agency as the event participants, or other music included in a compilation album that includes music used at the event). The right to download or stream data containing the above music. The right to purchase or borrow goods related to subjects that appeared at the event or were included in the expression, such as receiving a delivery of bell peppers if an event participant says "bell pepper". The right to use data representing the above subjects in video games, games with AR displays, etc. The right to feature the above subjects in other media related to the event (animation, manga, games, etc.). These are merely examples, and the specified authority may be other than those listed above.

[0046] Information that allows a user to acquire specific permissions (permission acquisition information) refers to information that, when presented with it, allows the user to express their intention to acquire those permissions, such as information that displays a purchase button regarding the right to purchase goods.

[0047] Displaying authorization information on a display device means outputting display data for displaying authorization information to a device equipped with a display device (e.g., user terminal 20) or a display device. Displaying authorization information on a display device may also mean outputting a command for displaying authorization information to a device that already has display data (e.g., user terminal 20 capable of generating or storing display data) or a display device.

[0048] As one aspect of the first embodiment, it is possible to hold an event using augmented reality, and to sell goods or provide services at such an event.

[0049] [Second Embodiment] A second embodiment of the present invention will now be described. In the following description, an event provision program executed on a server will be used as an example of the second embodiment. The server may be the server 10 of the event provision system 100 shown in Figure 1.

[0050] Figure 4 is a block diagram showing the configuration of a server corresponding to at least one embodiment of the present invention. Server 10B, an example of the configuration of Server 10, comprises at least an object display unit 101, a representation control unit 102, and an authorization acquisition information display unit 103B. The processor of Server 10A refers to an event provision program held (installed) in a storage device and executes that program to functionally realize the object display unit 101, the representation control unit 102, and the authorization acquisition information display unit 103B.

[0051] The object display unit 101 has the function of superimposing a first object, which is a virtual object, onto the real space and displaying it on the display device. The expression control unit 102 has the function of controlling the expression using one or more virtual objects, including the first object.

[0052] The authorization acquisition information display unit 103B has the function of displaying authorization acquisition information on the display device when it acquires user input operation information for product information, which is information indicating a product or service displayed on the display device.

[0053] Next, the program execution process in a second embodiment of the present invention will be described. Figure 5 is a flowchart showing an example of event provisioning processing corresponding to at least one embodiment of the present invention.

[0054] The object display unit 101 superimposes a first virtual object onto the real space and displays it on the display device (St21). The expression control unit 102 controls the expression using one or more virtual objects, including the first object (St22).

[0055] The authorization acquisition information display unit 103B acquires user input operation information for product information, which is information indicating a product or service displayed on the display device (St23). When the authorization acquisition information display unit 103B acquires the aforementioned input operation information, it displays information (authorization acquisition information) on the display device that allows the user to acquire predetermined authorization for a product or service related to an object, event, or representation (St24).

[0056] The terms used in steps St21 and St22, as well as the meaning of the authorization acquisition information, are the same as in the first embodiment, so their explanation will be omitted.

[0057] User input information refers to information entered by the user. For example, information entered by tapping or swiping a designated area on the touch panel display of the user terminal 20, or by voice input, falls under the category of user input information. Information entered by a user using AR glasses through gesture input also falls under the category of user input information. In addition, information entered by the user via various input devices such as keyboards and mice also falls under the category of user input information.

[0058] Product information refers to information that indicates a product or service. For example, if a T-shirt is the product to be purchased, then image information or icon image information of a T-shirt displayed on the display device would be considered product information. For a service that allows users to stream a specific song, then text information or icon image information indicating the song's title would be considered product information. Product information may be displayed using augmented reality (AR). AR display means displaying an object superimposed on real space. Clothing worn by a virtual character object (including virtual humans) displayed on the display device, the character itself (products would be 1 / N size figures, etc.), merchandise with team logos worn by professional players in a multi-player battle royale game tournament, or the gaming PC used by the professional player in the tournament may be displayed using AR as product information. AR-displayed accessories worn by a person in real space reflected in the display device also qualify as product information.

[0059] Product information may be displayed in a manner other than AR display. For example, actual stage props placed at an event venue in the real world may be reflected on a display device such as a smartphone's touch panel display or AR glasses. The actual props reflected on the display device may constitute product information indicating related products (e.g., miniature toys of stage props). The user can, for example, indicate their intention to purchase the related product by tapping the area on the touch panel display where the actual props are reflected. To achieve this, the server 10B or user terminal 20 may provide a function to set the area of ​​the actual props reflected on the display device as an input area, so that the user can select an area of ​​the actual props reflected on the display device and input data.

[0060] As one aspect of the second embodiment, information for obtaining permission can be displayed when a user expresses an intention to obtain permission, such as to purchase a product. This allows users who do not intend to obtain permission to focus on participating in the event.

[0061] [Third Embodiment] A third embodiment of the present invention will now be described. In the following, an event provision program executed on a server will be used as an example to describe the third embodiment. The server may be the server 10 of the event provision system 100 shown in Figure 1.

[0062] Figure 6 is a block diagram showing the configuration of a server corresponding to at least one embodiment of the present invention. Server 10C, an example of the configuration of Server 10, comprises at least an object display unit 101, a representation control unit 102, and an authorization acquisition information display unit 103C. The processor of Server 10A refers to an event provision program held (installed) in a storage device and executes that program to functionally realize the object display unit 101, the representation control unit 102, and the authorization acquisition information display unit 103C.

[0063] The object display unit 101 has the function of superimposing a first object, which is a virtual object, onto the real space and displaying it on the display device. The expression control unit 102 has the function of controlling the expression using one or more virtual objects, including the first object.

[0064] The authorization acquisition information display unit 103C has the function of displaying information (authorization acquisition information) on the display device that allows the user to acquire predetermined authorizations for goods or services related to an object, event, or representation. The authorization acquisition information display unit 103C displays the authorization acquisition information in a second area different from the first area on which the first object is displayed by the display device.

[0065] Next, the program execution process in the third embodiment of the present invention will be described. Figure 7 is a flowchart showing an example of event provisioning processing corresponding to at least one embodiment of the present invention.

[0066] The object display unit 101 superimposes a first object, which is a virtual object, onto the real space and displays it on the display device (St31). The expression control unit 102 controls the expression using one or more virtual objects, including the first object (St32). The authorization acquisition information display unit 103C displays authorization acquisition information in a second area different from the first area on the display device where the first object is displayed (St33).

[0067] Since the terms related to steps St31 to St33 are the same as those in the first or second embodiment, their explanation will be omitted.

[0068] As one aspect of the third embodiment, by separating the first area used for events from the second area for purchasing products, the display for purchasing products does not hinder the user's participation in the event.

[0069] [Fourth Embodiment] An overview of a fourth embodiment of the present invention will now be described. In the following, an event provision program executed on a server will be used as an example to describe the fourth embodiment. The server may be the server 10 of the event provision system 100 shown in Figure 1.

[0070] Figure 8 is a block diagram showing the configuration of a server corresponding to at least one embodiment of the present invention. Server 10D, an example of the configuration of Server 10, comprises at least an object display unit 101, a representation control unit 102D, and a permission acquisition information display unit 103. The processor in Server 10D refers to an event provision program held (installed) in a storage device and executes that program to functionally realize the object display unit 101, the representation control unit 102D, and the permission acquisition information display unit 103.

[0071] The object display unit 101 has the function of superimposing a first object, which is a virtual object, onto the real space and displaying it on the display device. The expression control unit 102D has the function of controlling the expression using one or more virtual objects, including the first object, depending on whether or not permission has been acquired. The permission acquisition information display unit 103 has the function of displaying information (permission acquisition information) on the display device that allows the user to acquire predetermined permission for an object, event, or product or service related to the expression.

[0072] Next, the program execution process in the fourth embodiment of the present invention will be described. Figure 9 is a flowchart showing an example of event provisioning processing corresponding to at least one embodiment of the present invention.

[0073] The object display unit 101 superimposes a first object, which is a virtual object, onto the real space and displays it on the display device (St41). The expression control unit 102D controls the expression using one or more virtual objects, including the first object, depending on whether or not permission has been acquired (St42). The permission acquisition information display unit 103 displays information (permission acquisition information) on the display device that allows the user to acquire predetermined permission for goods or services related to the object, event, or expression.

[0074] Controlling the representation using one or more virtual objects, including the first object, depending on whether or not permission has been obtained means controlling the representation such that at least a part of it differs between a user who has obtained permission and a user who has not obtained permission.

[0075] Other terms relating to steps St41 to St43 are the same as those in the first to third embodiments, so their explanation will be omitted.

[0076] As one aspect of the fourth embodiment, different effects can be displayed during an event depending on whether the user has acquired permission or not. This allows users to experience different events even if the event is essentially the same, thus personalizing the event.

[0077] [Fifth Embodiment] A fifth embodiment of the present invention will be outlined below. In the following description, an event provision system will be used as an example to illustrate the fifth embodiment.

[0078] Figure 10 shows an example of the configuration of an event provision system corresponding to at least one embodiment of the present invention. The event provision system 100A comprises an event provision processing server 10 (server 10) and user terminals 20 used by users of the event provision system 100A. User terminals 20A, 20B, and 20C are examples of user terminals 20. The configuration of the event provision system 100A is not limited thereto. For example, the event provision system 100A may be configured so that a single user terminal is used by multiple users. The event provision system 100A may also comprise multiple servers. For example, the event provision system 100A may further comprise other servers 40.

[0079] The event provision system 100A may further include a receptionist device 60. The receptionist device 60 is used by receptionist SPs who interact with users visiting, for example, the merchandise tent (see Figure 21) described later. Specific examples of the receptionist device 60 include a server device, a client device, and a user terminal.

[0080] Server 10, user terminals 20, other servers 40, and responder devices 60 are each connected to a communication network 30, such as the Internet, in a manner that allows communication. The connections between Server 10, user terminals 20, other servers 40, and responder devices 60 via the communication network 30 may be wired or wireless. For example, multiple user terminals 20 may connect to the communication network 30 by performing data communication via wireless communication lines with base stations managed by a telecommunications carrier.

[0081] The event provision system 100A, by comprising a server 10 and a user terminal 20, realizes various functions for executing various processes in response to user operations.

[0082] Server 10 comprises a processor 11, memory 12, and storage device 13. The processor 11 is, for example, a central processing unit such as a CPU (Central Processing Unit) that performs various calculations and controls. If Server 10 is equipped with a GPU (Graphics Processing Unit), some of the various calculations and controls may be performed by the GPU. Server 10 uses the data read into memory 12 to perform various information processing with the processor 11 and stores the obtained processing results in storage device 13 as needed.

[0083] The storage device 13 functions as a storage medium for storing various types of information. The configuration of the storage device 13 is not particularly limited, but from the viewpoint of reducing the processing load on the user terminal 20, it is preferable that it be configured to store all the various types of information necessary for control performed by the event provision system 100A. Examples of such configurations include HDDs and SSDs. However, the storage device for storing various types of information only needs to have a storage area that is accessible to the server 10, and may, for example, have a dedicated storage area outside the server 10. The server 10 is managed by the administrator of the event provision system 100A, etc., and has various functions for providing information related to various processes to multiple user terminals 20. The server 10 is composed of an information processing device such as a game server capable of rendering AR display data.

[0084] The user terminal 20 is comprised of a communication terminal capable of providing the user with information services accompanied by AR display. Examples of such communication terminals include, for example, mobile phone terminals 20A such as smartphones, and AR glasses 20B and 20C. The communication terminal may also be a stationary game console, a portable game console, a personal computer, a gaming PC, a so-called wearable device, or a combination of multiple of the aforementioned communication terminals. The configuration of the user terminal that the event provision system 100A may include is not limited to these, and is acceptable as long as the configuration allows the user to view the AR display.

[0085] The user terminal 20 comprises a processor 21, memory 22, and storage device 23. The processor 21 is, for example, a central processing unit such as a CPU (Central Processing Unit) that performs various calculations and controls. If the user terminal 20 is equipped with a GPU (Graphics Processing Unit), some of the various calculations and controls may be performed by the GPU. The user terminal 20 uses the data read into the memory 22 to perform various information processing using the processor 21, and stores the obtained processing results in the storage device 23 as needed. The storage device 23 functions as a storage medium for storing various information.

[0086] The user terminal 20 is connected to the communication network 30 and is equipped with hardware and software for performing various processes by communicating with the server 10, other servers 40, or the responder device 60. Note that each of the multiple user terminals 20 may also be configured to communicate directly with one another without going through the server 10, other servers 40, or the responder device 60.

[0087] The user terminal 20 may have a built-in display device. The display device may be connected to the user terminal 20 wirelessly or via a wired connection. The display device displays images such as still images or videos. The display device may also display a game screen. The display device may display the appearance of the responder SP based on data acquired directly from the responder device 60 via the communication network 30, or further via the server 10, etc. The display device can also perform augmented reality (AR) display superimposed on real space. Examples of display devices include hologram display devices capable of displaying holograms, and projection devices that project images (including game screens) onto a screen, etc. In the example in Figure 10, the touch panel screen of the mobile phone terminal 20A, the AR glasses 20B, and the AR displays of 20C correspond to display devices.

[0088] The user terminal 20 may have a built-in input device. Alternatively, the input device may be connected to the user terminal 20 wirelessly or via a wired connection. The input device accepts user input. In response to user input, the processor in the server 10 or the processor in the user terminal 20 executes various control processes. Examples of input devices include the touch panel screen of a mobile phone terminal 20A, and a controller CTR wirelessly or via a wired connection to the AR glasses 20B. The camera on the user terminal 20 may also be considered an input device. The user performs operation input by gestures, such as moving their hand in front of the camera (gesture input).

[0089] In addition, the user terminal 20 may be equipped with other output devices such as speakers. These other output devices may output various types of information to the user. This information may include, for example, voice messages spoken by the service provider SP. The other output devices may output voice information indicating the contents of the products the user purchases or the services the user receives. Note that the other output devices are not limited to voice output devices and may output other types of information. Furthermore, an output device using haptics technology may output other tactile information.

[0090] The other server 40 comprises a processor 41, memory 42, and storage device 43. The processor 41 is, for example, a central processing unit such as a CPU (Central Processing Unit) that performs various calculations and controls. If the other server 40 is equipped with a GPU (Graphics Processing Unit), some of the various calculations and controls may be performed by the GPU. The other server 40 uses the data read into memory 42 to perform various information processing with the processor 41 and stores the obtained processing results in storage device 43 as needed.

[0091] The storage device 43 functions as a storage medium for storing various types of information. The configuration of the storage device 43 is not particularly limited. The storage device for storing various types of information only needs to have a storage area that is accessible to other servers 40, and for example, it may be configured to have a dedicated storage area outside of other servers 40. Other servers 40 may be managed by the administrator of the event provision system 100A or by other persons, and have various functions for providing information related to various processes to multiple user terminals 20. Other servers 40 may be, for example, a server that manages the manufacturing and delivery of physical goods. Other servers 40 may also be a server that manages the distribution of videos related to live events.

[0092] The responder device 60 comprises a processor 61, memory 62, and storage device 63. The processor 61 is, for example, a central processing unit such as a CPU (Central Processing Unit) that performs various calculations and controls. If the responder device 60 is equipped with a GPU (Graphics Processing Unit), some of the various calculations and controls may be performed by the GPU. The responder device 60 uses the data read into the memory 62 to perform various information processing using the processor 61, and stores the obtained processing results in the storage device 63 as needed. The storage device 63 functions as a storage medium for storing various information.

[0093] The responder device 60 is connected to the communication network 30 and includes hardware and software for performing various processes by communicating with the server 10, the user terminal 20, or other servers 40.

[0094] The customer service device 60 includes a camera 64 and an input device 65. The camera 64 captures images of a customer service provider SP, such as a salesperson at a merchandise tent (see Figure 21). The captured image data, which includes the image of the customer service provider SP, is used in the cash register area (described later) to allow the customer service provider SP to interact with the user when the user purchases goods. The input device 65 receives operation input from the customer service provider SP, etc. In response to the operation input from the customer service provider SP, etc., the processor 61 of the customer service device 60 executes various control processes. Examples of input devices 65 include a keyboard, mouse, and microphone for voice operation. The camera 64 of the customer service device 60 can also be considered an input device. In this case, the customer service provider SP performs operation input by gestures such as moving their hands in front of the camera 64 (gesture input).

[0095] The responder device 60 includes a display device 66. The display device 66 may be built into the responder device 60. The display device 66 may be connected to the responder device 60 wirelessly or via a wired connection. The display device 66 displays images such as still images or videos. The display device 66 displays the image of the user purchasing a product or receiving a service. The display device 66 may also display information indicating the contents of the product the user is purchasing or the contents of the service the user is receiving.

[0096] In addition, the responder device 60 may be equipped with other output devices such as speakers. These other output devices output information relating to users using the merchandise tent, etc. This user-related information may include, for example, voices spoken by users who purchase goods or receive services. These other output devices may output voice information indicating the contents of the goods purchased by the user or the services received by the user. Note that these other output devices are not limited to voice output devices and may output other types of information such as smell information, taste information, and tactile information.

[0097] Figure 11 is a block diagram showing the configuration of an event provision system corresponding to at least one embodiment of the present invention. The event provision system 100A comprises at least an object display unit 501, a representation control unit 502, and an authorization acquisition information display unit 503. One or more processors in the event provision system 100A refer to an event provision program held (installed) in one or more storage devices in the event provision system 100A and execute that program to functionally realize the object display unit 501, the representation control unit 502, and the authorization acquisition information display unit 503.

[0098] The object display unit 501 has the function of superimposing a first object, which is a virtual object, onto the real space and displaying it on the display device. The expression control unit 502 has the function of controlling the expression using one or more virtual objects, including the first object. The authorization acquisition information display unit 503 has the function of displaying information (authorization acquisition information) on the display device that allows the user to acquire predetermined authorizations for goods or services related to an object, event, or expression.

[0099] Next, the program execution process in the fifth embodiment of the present invention will be described. Figure 12 is a flowchart showing an example of event provisioning processing corresponding to at least one embodiment of the present invention.

[0100] The object display unit 501 superimposes a first object, which is a virtual object, onto the real space and displays it on the display device (St51). The expression control unit 502 controls the expression using one or more virtual objects, including the first object (St52). The authorization acquisition information display unit 503 displays information (authorization acquisition information) on the display device that allows the user to acquire predetermined authorizations for goods or services related to the object, event, or expression (St53).

[0101] A virtual object refers to an object that can be output to a virtual space rather than the real world. For example, a virtual chair or car represented by 3D modeling data obtained by scanning the shape of an object in the real world, such as a chair or car, falls under the category of a virtual object. Also, 3D modeling data representing living beings such as humans, animals, plants, and monsters also falls under the category of a virtual object. A virtual object may also be a virtual representation of a fictional subject, such as a fairy or a dragon. Note that the data dimensions used to represent a virtual object are not limited to three dimensions. For example, a 2D data representation of such a subject also falls under the category of a virtual object.

[0102] Real space refers to the space in the real world. Virtual space, which extends real space, refers to a virtual space (AR space) that can be superimposed and output onto real space.

[0103] Displaying a virtual object on a display device means outputting display data for displaying a virtual object to a device equipped with a display device (e.g., a user terminal 20) or a display device. Displaying a virtual object on a display device may also mean outputting a command for displaying a virtual object to a device that already possesses display data (e.g., a user terminal 20 capable of generating or storing display data) or a display device.

[0104] Expression means presenting some kind of information to the recipient in an external form, such as singing, dancing, acting, reciting poetry, playing a musical instrument, giving a lecture, running, jumping, or crying. Controlling an expression using a virtual object may include controlling the actions of the object, such as making the object sing, dance, act, recite poetry, play a musical instrument, give a lecture, run, jump, or cry. Furthermore, controlling an expression using a virtual object may also mean conveying some kind of information to the recipient through one or more virtual objects, such as making five virtual cars drive in a circle to represent the Olympic rings.

[0105] An event refers to a venue where information is presented externally to the audience, such as a live music event, a theatrical performance, or a seminar.

[0106] A commodity refers to an item that is the subject of a transaction. A service refers to the provision of some kind of benefit by one entity to another entity. Regarding whether a virtual object is interpreted as a commodity or as the subject of a service provision, either interpretation is acceptable.

[0107] The prescribed rights (hereinafter abbreviated as "prescribed rights") relating to goods or services that are objects, events, or representations include, for example, the right to own, borrow, or use a physical object in the real world if the goods are physical objects in the real world. If the goods are data, the prescribed rights include the right to acquire, copy, modify, or use this data. For services, the prescribed rights include the right to receive the service.

[0108] As non-restrictive examples, specific examples of the prescribed authority are given. - The right to purchase or borrow physical items (e.g., figurines) that are modeled after the virtual characters appearing in the event. - The right to purchase or borrow physical items of clothing worn by the virtual characters appearing in the event. - The right to purchase or borrow physical items of possessions such as guitars owned by the virtual characters appearing in the event. - The right to purchase or borrow physical items corresponding to the stage equipment appearing in the event (including both virtual and real-world stage equipment). - The right to acquire or use data representing the virtual characters appearing in the event, the clothing worn by those characters, or the virtual items owned by those characters. - The right to add sound effects to physical items or virtual objects appearing in the event, such as changing the engine sound of a virtual car. - The right to purchase or borrow physical items related to the event, such as official event T-shirts and towels. - The right to purchase or borrow physical media such as CDs and DVDs containing music used in the event. The right to purchase or borrow physical media such as CDs and DVDs containing music related to the event or expression (for example, music provided by other artists belonging to the same agency as the event participants, or other music included in a compilation album that includes music used at the event). The right to download or stream data containing the above music. The right to purchase or borrow goods related to subjects that appeared at the event or were included in the expression, such as receiving a delivery of bell peppers if an event participant says "bell pepper". The right to use data representing the above subjects in video games, games with AR displays, etc. The right to feature the above subjects in other media related to the event (animation, manga, games, etc.). These are merely examples, and the specified authority may be other than those listed above.

[0109] Information that allows a user to acquire specific permissions (permission acquisition information) refers to information that, when presented with it, allows the user to express their intention to acquire those permissions, such as information that displays a purchase button regarding the right to purchase goods.

[0110] Displaying authorization information on a display device means outputting display data for displaying authorization information to a device equipped with a display device (e.g., user terminal 20) or a display device. Displaying authorization information on a display device may also mean outputting a command for displaying authorization information to a device that already has display data (e.g., user terminal 20 capable of generating or storing display data) or a display device.

[0111] As one aspect of the fifth embodiment, an event using augmented reality can be held, and merchandise sales and service provision can be carried out at said event.

[0112] [Sixth Embodiment] A sixth embodiment of the present invention will now be described. In the following, an event provision program executed on a user terminal will be described as an example of the sixth embodiment. Note that the user terminal 20Z may be any of the user terminals 20, 20A to 20C shown in Figure 1 or Figure 10.

[0113] Figure 13 is a block diagram showing the configuration of a user terminal corresponding to at least one embodiment of the present invention. User terminal 20Z, an example of the configuration of user terminal 20, comprises at least an object display unit 201, a representation control unit 202, and a permission acquisition information display unit 203. The processor in user terminal 20Z refers to an event provision program held (installed) in a storage device and executes that program, thereby functionally realizing the object display unit 201, the representation control unit 202, and the permission acquisition information display unit 203.

[0114] The object display unit 201 has the function of superimposing a first object, which is a virtual object, onto the real space and displaying it on the display device. The expression control unit 202 has the function of controlling the expression using one or more virtual objects, including the first object. The authorization acquisition information display unit 203 has the function of displaying information (authorization acquisition information) on the display device that allows the user to acquire predetermined authorizations for goods or services related to an object, event, or expression.

[0115] Next, the program execution process in the sixth embodiment of the present invention will be described. Figure 14 is a flowchart showing an example of event provisioning processing corresponding to at least one embodiment of the present invention.

[0116] The object display unit 201 superimposes a first object, which is a virtual object, onto the real space and displays it on the display device (St61). The expression control unit 202 controls the expression using one or more virtual objects, including the first object (St62). The authorization acquisition information display unit 203 displays information (authorization acquisition information) on the display device that allows the user to acquire predetermined authorizations for goods or services related to the object, event, or expression (St63).

[0117] A virtual object refers to an object that can be output to a virtual space rather than the real world. For example, a virtual chair or car represented by 3D modeling data obtained by scanning the shape of an object in the real world, such as a chair or car, falls under the category of a virtual object. Also, 3D modeling data representing living beings such as humans, animals, plants, and monsters also falls under the category of a virtual object. A virtual object may also be a virtual representation of a fictional subject, such as a fairy or a dragon. Note that the data dimensions used to represent a virtual object are not limited to three dimensions. For example, a 2D data representation of such a subject also falls under the category of a virtual object.

[0118] Real space refers to the space in the real world. Virtual space, which extends real space, refers to a virtual space (AR space) that can be superimposed and output onto real space.

[0119] Displaying a virtual object on a display device means outputting display data for displaying a virtual object to a device equipped with a display device (e.g., a user terminal 20) or a display device. Displaying a virtual object on a display device may also mean outputting a command for displaying a virtual object to a device that already possesses display data (e.g., a user terminal 20 capable of generating or storing display data) or a display device.

[0120] Expression means presenting some kind of information to the recipient in an external form, such as singing, dancing, acting, reciting poetry, playing a musical instrument, giving a lecture, running, jumping, or crying. Controlling an expression using a virtual object may include controlling the actions of the object, such as making the object sing, dance, act, recite poetry, play a musical instrument, give a lecture, run, jump, or cry. Furthermore, controlling an expression using a virtual object may also mean conveying some kind of information to the recipient through one or more virtual objects, such as making five virtual cars drive in a circle to represent the Olympic rings.

[0121] An event refers to a venue where information is presented externally to the audience, such as a live music event, a theatrical performance, or a seminar.

[0122] A commodity refers to an item that is the subject of a transaction. A service refers to the provision of some kind of benefit by one entity to another entity. Regarding whether a virtual object is interpreted as a commodity or as the subject of a service provision, either interpretation is acceptable.

[0123] The prescribed rights (hereinafter abbreviated as "prescribed rights") relating to goods or services that are objects, events, or representations include, for example, the right to own, borrow, or use a physical object in the real world if the goods are physical objects in the real world. If the goods are data, the prescribed rights include the right to acquire, copy, modify, or use this data. For services, the prescribed rights include the right to receive the service.

[0124] As non-restrictive examples, specific examples of the prescribed authority are given. - The right to purchase or borrow physical items (e.g., figurines) that are modeled after the virtual characters appearing in the event. - The right to purchase or borrow physical items of clothing worn by the virtual characters appearing in the event. - The right to purchase or borrow physical items of possessions such as guitars owned by the virtual characters appearing in the event. - The right to purchase or borrow physical items corresponding to the stage equipment appearing in the event (including both virtual and real-world stage equipment). - The right to acquire or use data representing the virtual characters appearing in the event, the clothing worn by those characters, or the virtual items owned by those characters. - The right to add sound effects to physical items or virtual objects appearing in the event, such as changing the engine sound of a virtual car. - The right to purchase or borrow physical items related to the event, such as official event T-shirts and towels. - The right to purchase or borrow physical media such as CDs and DVDs containing music used in the event. The right to purchase or borrow physical media such as CDs and DVDs containing music related to the event or expression (for example, music provided by other artists belonging to the same agency as the event participants, or other music included in a compilation album that includes music used at the event). The right to download or stream data containing the above music. The right to purchase or borrow goods related to subjects that appeared at the event or were included in the expression, such as receiving a delivery of bell peppers if an event participant says "bell pepper". The right to use data representing the above subjects in video games, games with AR displays, etc. The right to feature the above subjects in other media related to the event (animation, manga, games, etc.). These are merely examples, and the specified authority may be other than those listed above.

[0125] Information that allows a user to acquire specific permissions (permission acquisition information) refers to information that, when presented with it, allows the user to express their intention to acquire those permissions, such as information that displays a purchase button regarding the right to purchase goods.

[0126] Displaying authorization information on a display device means outputting display data for displaying authorization information to a device equipped with a display device (e.g., user terminal 20) or a display device. Displaying authorization information on a display device may also mean outputting a command for displaying authorization information to a device that already has display data (e.g., user terminal 20 capable of generating or storing display data) or a display device.

[0127] As one aspect of the sixth embodiment, an event using augmented reality can be held, and merchandise sales and service provision can be carried out at said event.

[0128] [Seventh Embodiment] An overview of the seventh embodiment of the present invention will be described below. In the following, an event provision program executed on a server will be used as an example to describe the seventh embodiment.

[0129] Figure 15 is a block diagram showing the configuration of a server corresponding to at least one embodiment of the present invention. Server 10Z, an example of the configuration of Server 10, comprises an object display unit 101, an expression control unit 102, an authorization acquisition information display unit 103, an augmented reality diorama display unit 104, and a recommendation information display unit 105. The processor of Server 10Z refers to an event provision program held (installed) in a storage device and executes that program to functionally realize the object display unit 101, the expression control unit 102, the authorization acquisition information display unit 103, the augmented reality diorama display unit 104, and the recommendation information display unit 105.

[0130] The object display unit 101 has the function of superimposing a first object, which is a virtual object, onto the real space and displaying it on the display device. The expression control unit 102 has the function of controlling the expression using one or more virtual objects, including the first object. The authorization acquisition information display unit 103 has the function of displaying information (authorization acquisition information) on the display device that allows the user to acquire predetermined authorizations for goods or services related to an object, event, or expression.

[0131] The augmented reality diorama display unit 104 has the function of displaying an augmented reality diorama superimposed on real space on a display device. The recommendation information display unit 105 has the function of displaying recommendation information on a display device that indicates products or services for which the user is recommended to obtain authorization, based on the content of an expression using one or more virtual objects including a first object, user input information, or predetermined stored information.

[0132] A virtual object refers to an object that can be output to a virtual space rather than the real world. For example, a virtual chair or car represented by 3D modeling data obtained by scanning the shape of an object in the real world, such as a chair or car, falls under the category of a virtual object. Also, 3D modeling data representing living beings such as humans, animals, plants, and monsters also falls under the category of a virtual object. A virtual object may also be a virtual representation of a fictional subject, such as a fairy or a dragon. Note that the data dimensions used to represent a virtual object are not limited to three dimensions. For example, a 2D data representation of such a subject also falls under the category of a virtual object.

[0133] Real space refers to the space in the real world. Virtual space, which extends real space, refers to a virtual space (AR space) that can be superimposed and output onto real space.

[0134] Displaying a virtual object on a display device means outputting display data for displaying a virtual object to a device equipped with a display device (e.g., a user terminal 20) or a display device. Displaying a virtual object on a display device may also mean outputting a command for displaying a virtual object to a device that already possesses display data (e.g., a user terminal 20 capable of generating or storing display data) or a display device.

[0135] Expression means presenting some kind of information to the recipient in an external form, such as singing, dancing, acting, reciting poetry, playing a musical instrument, giving a lecture, running, jumping, or crying. Controlling an expression using a virtual object may include controlling the actions of the object, such as making the object sing, dance, act, recite poetry, play a musical instrument, give a lecture, run, jump, or cry. Furthermore, controlling an expression using a virtual object may also mean conveying some kind of information to the recipient through one or more virtual objects, such as making five virtual cars drive in a circle to represent the Olympic rings.

[0136] An event refers to a venue where information is presented externally to the audience, such as a live music event, a theatrical performance, or a seminar.

[0137] A commodity refers to an item that is the subject of a transaction. A service refers to the provision of some kind of benefit by one entity to another entity. Regarding whether a virtual object is interpreted as a commodity or as the subject of a service provision, either interpretation is acceptable.

[0138] The prescribed rights (hereinafter abbreviated as "prescribed rights") relating to goods or services that are objects, events, or representations include, for example, the right to own, borrow, or use a physical object in the real world if the goods are physical objects in the real world. If the goods are data, the prescribed rights include the right to acquire, copy, modify, or use this data. For services, the prescribed rights include the right to receive the service.

[0139] As non-restrictive examples, specific examples of the prescribed authority are given. - The right to purchase or borrow physical items (e.g., figurines) that are modeled after the virtual characters appearing in the event. - The right to purchase or borrow physical items of clothing worn by the virtual characters appearing in the event. - The right to purchase or borrow physical items of possessions such as guitars owned by the virtual characters appearing in the event. - The right to purchase or borrow physical items corresponding to the stage equipment appearing in the event (including both virtual and real-world stage equipment). - The right to acquire or use data representing the virtual characters appearing in the event, the clothing worn by those characters, or the virtual items owned by those characters. - The right to add sound effects to physical items or virtual objects appearing in the event, such as changing the engine sound of a virtual car. - The right to purchase or borrow physical items related to the event, such as official event T-shirts and towels. - The right to purchase or borrow physical media such as CDs and DVDs containing music used in the event. The right to purchase or borrow physical media such as CDs and DVDs containing music related to the event or expression (for example, music provided by other artists belonging to the same agency as the event participants, or other music included in a compilation album that includes music used at the event). The right to download or stream data containing the above music. The right to purchase or borrow goods related to subjects that appeared at the event or were included in the expression, such as receiving a delivery of bell peppers if an event participant says "bell pepper". The right to use data representing the above subjects in video games, games with AR displays, etc. The right to feature the above subjects in other media related to the event (animation, manga, games, etc.). These are merely examples, and the specified authority may be other than those listed above.

[0140] Information that allows a user to acquire specific permissions (permission acquisition information) refers to information that, when presented with it, allows the user to express their intention to acquire those permissions, such as information that displays a purchase button regarding the right to purchase goods.

[0141] Displaying authorization information on a display device means outputting display data for displaying authorization information to a device equipped with a display device (e.g., user terminal 20) or a display device. Displaying authorization information on a display device may also mean outputting a command for displaying authorization information to a device that already has display data (e.g., user terminal 20 capable of generating or storing display data) or a display device.

[0142] User input information refers to information entered by the user. For example, information entered by tapping or swiping a designated area on the touch panel display of the user terminal 20, or by voice input, falls under the category of user input information. Information entered by a user using AR glasses through gesture input also falls under the category of user input information. In addition, information entered by the user via various input devices such as keyboards and mice also falls under the category of user input information.

[0143] Product information refers to information that indicates a product or service. For example, if a T-shirt is the product to be purchased, then image information or icon image information of a T-shirt displayed on the display device would be considered product information. For a service that allows users to stream a specific song, then text information or icon image information indicating the song's title would be considered product information. Product information may be displayed using augmented reality (AR). AR display means displaying an object superimposed on real space. Clothing worn by a virtual character object (including virtual humans) displayed on the display device, the character itself (products would be 1 / N size figures, etc.), merchandise with team logos worn by professional players in a multi-player battle royale game tournament, or the gaming PC used by the professional player in the tournament may be displayed using AR as product information. AR-displayed accessories worn by a person in real space reflected in the display device also qualify as product information.

[0144] Product information may be displayed in a manner other than AR display. For example, actual stage props placed at an event venue in the real world may be reflected on a display device such as a smartphone's touch panel display or AR glasses. The actual props reflected on the display device may constitute product information indicating related products (e.g., miniature toys of stage props). The user can, for example, indicate their intention to purchase the related product by tapping the area on the touch panel display where the actual props are reflected. To achieve this, the server 10Z or user terminal 20 may provide a function to set the area of ​​the actual props reflected on the display device as an input area, so that the user can select an area of ​​the actual props reflected on the display device and input data.

[0145] Controlling the representation using one or more virtual objects, including the first object, depending on whether or not permission has been obtained means controlling the representation such that at least a part of it differs between a user who has obtained permission and a user who has not obtained permission.

[0146] An augmented reality diorama refers to a virtual diorama that is superimposed onto real space. For example, cities, roads, railway lines, buildings, buildings, stadiums for baseball or soccer, shrines and temples, castles, mountains, seas, rivers, swamps, forests, fields, rice paddies, airports, underground passages, caves, etc., represented as 3D model data for augmented reality (AR) display are examples of augmented reality dioramas. It should be noted that augmented reality dioramas do not necessarily have to be based on existing modern landscapes. For example, medieval European cityscapes and buildings, as well as fictional cityscapes and buildings that appear in animation or comics, can also be included as augmented reality dioramas.

[0147] Displaying an augmented reality diorama on a display device means outputting display data to a device equipped with a display device (e.g., a user terminal 20) or a display device in order to display the augmented reality diorama superimposed on real space. Alternatively, displaying an augmented reality diorama on a display device may also mean outputting a command to a device that already possesses display data (e.g., a user terminal 20 capable of generating or storing display data) or a display device in order to display the augmented reality diorama superimposed on real space.

[0148] User input information refers to information input attributable to the user. For example, information entered via the user terminal 20 falls under the category of user input information. Server 10Z may obtain input information from the user terminal 20 via the communication network 30.

[0149] Furthermore, input information separately acquired by server 10Z without the user terminal 20 may be used as user input information. For example, if server 10Z acquires log data of the behavioral history of an avatar associated with a user or the user themselves when they have acted in an augmented space corresponding to an event such as an augmented reality diorama or live event, this log data may be considered user input information. Also, information related to the user acquired by server 10Z from an external system such as another server 40 also constitutes user input information. For example, purchase history information when a user purchases a product using another product sales site, usage history information when a user uses public transportation such as trains, content data posted by a user on an SNS service, and text information indicating statements made via an SNS service may also be included in the user input information referred to here.

[0150] Accumulated information refers to information stored in a memory device accessible from the processing entity (in this case, server 10Z). For example, stored information includes attribute information of each user (age, preferences, etc.), purchase history information, and detailed event information (performance name, performance date, performance venue, performers, organizer, co-organizer, music used, name of the tie-in animation program, etc.). Accumulated information may also be information accumulated as the event progresses. For example, the total number of participants in the currently ongoing live event, how many people are watching each stage, feedback and evaluation information entered via user terminal 20, etc., regarding the live event, and estimated information indicating the level of excitement at the live event are all considered accumulated information. In other words, accumulated information is a variety of information not limited to information related to individual users. Furthermore, accumulated information may also include various attribute information of virtual objects displayed in AR, and various attribute information of augmented reality dioramas.

[0151] Recommendation information refers to information that indicates products or services for which users are encouraged to obtain permission. For example, the title information of a CD containing songs performed by an artist at a live music event, and links to websites where this CD can be purchased, fall under the category of recommendation information. Similarly, for a live stage play event, URL information for a website that allows users to obtain permission to stream past performances by the theater company that organized the play also falls under the category of recommendation information. The types of recommendation information are not limited to these examples.

[0152] Displaying recommendation information on a display device means outputting display data for displaying recommendation information to a device equipped with a display device (e.g., a user terminal 20) or a display device. Displaying recommendation information on a display device may also mean outputting a command for displaying recommendation information to a device that already has display data (e.g., a user terminal 20 capable of generating or storing display data) or a display device.

[0153] Next, the program execution process in the seventh embodiment of the present invention will be described. Figure 16 is a flowchart showing an example of event provisioning processing corresponding to at least one embodiment of the present invention.

[0154] The augmented reality diorama display unit 104 displays an augmented reality diorama superimposed on real space on the display device (St111). The object display unit 101 superimposes a first object, which is a virtual object, onto real space and displays it on the display device (St112). The expression control unit 102 controls the expression using one or more virtual objects, including the first object (St113). The recommendation information display unit 105 displays recommendation information on the display device indicating products or services for which user authorization is recommended, according to the content of the expression using one or more virtual objects, including the first object, user input information, or predetermined stored information (St114). The authorization acquisition information display unit 103 acquires user input operation information for product information, which is information indicating products or services displayed on the display device (St115). The authorization acquisition information display unit 103 then, upon acquiring the aforementioned input operation information, displays information (authorization acquisition information) on the display device that allows the user to acquire predetermined authorizations for goods or services related to an object, event, or representation (St116).

[0155] Furthermore, steps St111, St114, and St115 do not necessarily need to be provided.

[0156] Figure 17 is a conceptual diagram showing an example of display by a display device corresponding to at least one embodiment of the present invention. In this embodiment, the display device DISP is a touch panel display provided by the user terminal 20. However, as described above, the display device DISP may be a display device built into or connected to other devices such as AR glasses. The image captured by the camera (not shown) provided by the user terminal 20 is displayed by the display device DISP. Therefore, the display device DISP reflects a television that exists in the real world. A first object OBJ1, which is a virtual object, is displayed superimposed on the real world on the display device DISP (step St112). In addition, a second object OBJ2, which is a virtual object, is also displayed superimposed on the real world on the display device DISP. In this embodiment, the first object OBJ1 corresponds to an artist at a music live event, and the second object OBJ2 corresponds to a synthesizer for performance. In Figure 17, the entire display screen of the display device DISP corresponds to the first area.

[0157] The expression control unit 102 controls the expression using one or more virtual objects, including the first object (St113). In this embodiment, the expression control unit 102 controls the expression using the first object OBJ1 and the second object OBJ2. The content of the expression is, for example, the holding of a music live performance using virtual objects. Under the control of the expression control unit 102, the AR-displayed first object OBJ1, which corresponds to the artist, sings, dances, and talks. The AR-displayed second object OBJ2, which corresponds to the synthesizer, emits sound when the first object OBJ1, the artist, plays it. Note that the expression is not limited to visual expression only. Expressions using various types of information that the user can receive are possible, such as auditory expression (singing or instrument sounds, etc.) and tactile expression (vibrations from a woofer, etc.).

[0158] Figure 18 is a conceptual diagram showing an example of a display by a display device corresponding to at least one embodiment of the present invention. In this embodiment, the jacket, the first object OBJ1 displayed on the display device DISP, is available for purchase by the user as a product. In other words, the jacket corresponds to product information, which is information indicating a product or service displayed on the display device.

[0159] In this embodiment, the user taps the jacket portion of the first object OBJ1 displayed on the display device DISP, which is a touch panel display. This tap corresponds to user input information. The user terminal 20 transmits the input information to the server 10Z, and the authorization acquisition information display unit 103 of the server 10Z acquires the input information (St115). The authorization acquisition information display unit 103 then displays the authorization acquisition information R1 and R2 on the display device DISP (St116).

[0160] In Figure 18, the authorization acquisition information R1 and R2 are information that allows the user to acquire the authority to possess a product (jacket) related to the first object OBJ1, which corresponds to the artist. Here, the authorization acquisition information R1 shows product information (product title, product price, product image, etc.) when purchasing the jacket, and the Buy button for the user to indicate their intention to purchase corresponds to the authorization acquisition information. The user can purchase the jacket by tapping the Buy button displayed on the display device DISP, and in this example, the jacket, which is the product, is mailed to the user's home, for example, based on control by server 10Z and other servers 40.

[0161] The display method for the authorization acquisition information shown in Figure 18 is merely an example. The authorization acquisition information may be displayed in other ways. For example, the second area is dynamically displayed as a pop-up on the display device DISP, but the first and second areas may be displayed separately from the beginning. In Figure 18, the authorization acquisition information is displayed using AR (Augmented Reality), but this is not required. Also, in Figure 18, the authorization acquisition information is displayed in a second area separate from the first area, but the authorization acquisition information may be displayed in the first area without a second area.

[0162] Furthermore, in Figure 18, the authorization acquisition information is displayed when the server 10Z acquires user input information, but the authorization acquisition information may be displayed without acquiring input information. For example, the authorization acquisition information display unit 103 may display the authorization acquisition information when predetermined conditions are met, such as when a music live event is progressing and the artist has finished singing the third song. The predetermined conditions are not limited to the above example. The predetermined conditions may be any conditions relating to an object, event, or representation. For example, the predetermined conditions may be that 20 minutes after the start of the live event the number of viewers of the live event has reached 1000 or more, or that the artist has mentioned today's costume during the music live performance.

[0163] Figure 19 is a conceptual diagram showing an example of display by a display device corresponding to at least one embodiment of the present invention. Figure 19 shows a case where authorization acquisition information is displayed in the first area without providing a second area.

[0164] Users can purchase synthesizers that appear at music live events as merchandise. Therefore, the second object OBJ2, which corresponds to the synthesizer and is displayed on the display device DISP, represents the merchandise information. The user taps the displayed second object OBJ2. This tap corresponds to user input information. The user terminal 20 sends the input information to the server 10Z, and the authorization acquisition information display unit 103 of the server 10Z acquires the input information (St115). The authorization acquisition information display unit 103 then displays the authorization acquisition information R3 on the display device DISP (St116).

[0165] In Figure 19, the authorization acquisition information R3 is information that allows the user to acquire the authority to possess the synthesizer, which is the product. In other words, the pop-up bubble displayed at the top of the second object OBJ2 and the product information (product title, model number, product price, etc.) displayed within the bubble correspond to the authorization acquisition information. In this embodiment, the user can purchase the synthesizer by double-tapping the second object OBJ2, and in this embodiment, the synthesizer, which is the product, is mailed to the user's home.

[0166] In the example above, the synthesizer corresponds to a product related to the second object, but it is also a product related to the music live event. Therefore, the synthesizer can also be interpreted as a product related to the expression in the music live performance.

[0167] Product information may also be information indicating a specific service. For example, when a user taps the hand portion of the first object OBJ1, which represents an artist, information indicating a service such as "You can shake hands with the artist at a later date" may be displayed on the display device DISP. The content of the service is not limited to shaking hands; various services are conceivable. For example, the first object OBJ1 is an avatar used by a poster on a video sharing site, and the service may involve having this avatar appear in a specific animated program or game.

[0168] Figure 20 is a conceptual diagram showing an example of display by a display device corresponding to at least one embodiment of the present invention. Figure 20 shows an example (step St111) in which the augmented reality diorama display unit 104 displays the augmented reality diorama ARD1 on the display device DISP. In this embodiment, the live stage at a music concert event is displayed as an augmented reality diorama using AR. The first object OBJ1 is displayed on the display device DISP together with the augmented reality diorama ARD1.

[0169] Figure 21 is a conceptual diagram showing an example of display by a display device corresponding to at least one embodiment of the present invention. Figure 21 shows an example of displaying the augmented reality diorama ARD2 in the second area of ​​the display device DISP (step St111). The augmented reality diorama display unit 104 may display the augmented reality diorama ARD2 at a timing different from step St111, and may also display the augmented reality diorama ARD2 without displaying the first object OBJ1. In the example of this embodiment, when the user taps the jacket portion (product information, etc.) of the first object OBJ1 displayed on the touch panel display in Figure 20, the display device DISP displays as shown in Figure 21. When the jacket portion (product information, etc.) of the first object OBJ1 is tapped, a text message or voice message such as "Please come to the merchandise tent to purchase products" may be output to the user terminal 20 or the display device under the control of the server 10Z or the user terminal 20.

[0170] In Figure 20, the user was pointing the tablet device (user terminal 20) towards the direction of the TV in their home. In Figure 21, the user is pointing the tablet device in the opposite direction from the TV in their home. Therefore, the display device DISP shows a clock hanging on the wall of the home, which exists in the real world. The augmented reality diorama ARD2 displayed on the display device DISP corresponds to a merchandise tent at a music festival event.

[0171] In Figure 21, the authorization acquisition information R4 is information that allows the user to acquire the authority to possess a product (jacket) related to the first object OBJ1, which corresponds to the artist. The pop-up speech bubble and the product information displayed within the speech bubble (product title, product price, product image, etc.) correspond to the authorization acquisition information R4. The authorization acquisition information R4 is displayed by the display device DISP together with the augmented reality diorama ARD2. In this embodiment, the user can purchase the jacket by double-tapping the speech bubble, and in this embodiment, the product, a synthesizer, is mailed to the user's home.

[0172] The display device DISP further displays the register area RR and the corresponding characters A1 to A5. The control of the register area RR and corresponding characters A1 to A5 may be performed by the server 10Z or by the user terminal 20. When the server 10Z controls the register area RR and corresponding characters A1 to A5, the server 10Z may further include a register area display unit and a corresponding character display unit. In this case, the processor of the server 10Z refers to an event provision program held (installed) in the storage device and executes that program to functionally realize the register area display unit and the corresponding character display unit.

[0173] The register area display unit has the function of displaying a virtual register area on the display device to allow the user to acquire the aforementioned permissions. The response character display unit has the function of displaying one or more response characters that respond to the user on the display device along with the register area.

[0174] In the example shown in Figure 21, five customer service characters A1 to A5 are positioned in the register area RR. Rock Festival 2021 is being held virtually in the augmented space, and customer service character A1 assists the user with inquiries about the sale of official Rock Festival 2021 merchandise. Customer service character A2 assists the user with inquiries about the sale of merchandise for artists performing on the Wind Stage, one of the three AR-displayable stages at Rock Festival 2021. Customer service character A3 assists the user with inquiries about the sale of merchandise for artists performing on the Sea Stage, one of the three AR-displayable stages at Rock Festival 2021. Customer service character A4 assists the user with inquiries about the sale of merchandise for artists performing on the Forest Stage, one of the three AR-displayable stages at Rock Festival 2021. Customer service character A5 assists the user with inquiries about joining the fan clubs of all artists performing at Rock Festival 2021. For example, a user can tap a displayed character to display a list screen (not shown) of tradable goods or services on the display device DISP, and then conduct a transaction for the desired goods or services. The user interface for the transaction is not particularly limited and can be designed and implemented as appropriate by a person skilled in the art.

[0175] Figure 22 is a conceptual diagram showing an example of a display by a display device corresponding to at least one embodiment of the present invention. Figure 22 illustrates a stage play event in which a story of fighting a dragon is performed in an augmented reality space.

[0176] In this embodiment, the display device DISP is a touch panel display provided by the user terminal 20. However, as described above, the display device DISP may be a display device built into or connected to another device, such as AR glasses. The image captured by the camera (not shown) provided by the user terminal 20 is displayed on the display device DISP. Therefore, the display device DISP reflects a television that exists in the real world. The display device DISP displays the virtual object Dragon EN superimposed on the real world. The display device DISP displays the first object OBJ1, a virtual object, superimposed on the real world (step St112). The first object OBJ1 is the main character in a play fighting Dragon EN. The augmented reality diorama ARD3 corresponds to the stage of the play. In Figure 22, the entire display screen of the display device DISP corresponds to the first region.

[0177] The expression control unit 102 controls the expression using one or more virtual objects, including the first object (St113). In this embodiment, the expression control unit 102 controls the expression using the first object OBJ1 and the dragon EN. The expression here refers to the holding of a stage play live using virtual objects. Under the control of the expression control unit 102, the AR-displayed first object OBJ1, which corresponds to the main character, fights against the AR-displayed dragon EN. Note that the content of the expression is not limited to visual only. For example, it is possible to use various types of information that the user can receive, such as auditory expressions (BGM during battle) and tactile expressions (vibrations when an attack hits an enemy).

[0178] Figure 23 is a conceptual diagram showing an example of display by a display device corresponding to at least one embodiment of the present invention. The same matters as in Figure 22 will not be explained. In Figure 23, a second area is provided on the display device DISP. Furthermore, the recommendation area RE is displayed on the display device DISP.

[0179] First, let's explain the recommendation area RE. The recommendation information display unit 105 displays recommendation information on the display device (St114) that indicates products or services for which the user is recommended to obtain authorization, based on the content of the representation using one or more virtual objects including the first object, user input information, or predetermined stored information. In Figure 23, this recommendation information is displayed in the recommendation area RE.

[0180] In the recommendation area RE shown in Figure 23, recommendation information is displayed that indicates a service that allows the protagonist, the first object OBJ1, to equip a shield to protect himself from dragon attacks, in accordance with the content of the expression using the first object OBJ1, namely, "a story about fighting dragons."

[0181] Furthermore, in response to the string "theme song" entered by the user via the user terminal 20, recommendation information indicating a service that plays the song "Nobori Ryu" sung by artist "Mr.SEJ" as background music in the play is displayed in the recommendation area RE. This recommendation information may be displayed in the form of a musical note icon, the artist's name, and artwork including the song title. However, the display form is not limited to this. In this embodiment, it is assumed that the fictional artist "Mr.SEJ" is the person who sings the ending theme song (a different song from "Nobori Ryu") in the animated version of the stage play "AR Quest". In addition, performance information for the stage play "AR Quest" (voice actors, production company, theme song, etc.) is stored as accumulated information in a storage device accessible from server 10Z. Based on this accumulated information, recommendation information indicating a figure, which is official merchandise of artist "Mr.SEJ" involved in the stage play "AR Quest", is displayed in the recommendation area RE.

[0182] Since recommendation information is information that indicates a product or service, it also falls under the category of product information, which is information that indicates a product or service displayed on the display device. Therefore, when a user taps the recommendation information displayed in the recommendation area RE, the authorization acquisition information display unit 103 displays authorization acquisition information R5 and R6 on the display device DISP based on the input operation information related to this tap.

[0183] In Figure 23, the permission acquisition information R5 and R6 are information that grants the user permission to receive the service of playing the song "Nobori Ryu" as background music in the film. In other words, the artwork representing the song and the Buy button for the user to indicate their intention to purchase correspond to the permission acquisition information. By swiping the artwork (permission acquisition information R5) to the Buy button (permission acquisition information R6), the user can listen to the song via streaming playback. Note that the display method of the permission acquisition information shown in Figure 23 is merely an example, and other display methods may be adopted.

[0184] Here, the expression control unit 102 has the function of controlling the expression depending on whether or not permission has been acquired. In the example of this embodiment, if a user acquires the permission to receive the service of playing the song "Rising Dragon" as background music in the play, the server 10Z acquires information indicating that the user has acquired permission, and according to this information, the expression control unit 102 switches the background music of the stage "AR Quest" from another song to "Rising Dragon". The expression control unit 102 also controls the actions of the first object OBJ1, which is the main character, and makes the main character bow. For example, the expression control unit 102 may change the content of the expression depending on whether or not permission has been acquired. Note that the change in the content of the expression is not limited to changing the background music or predetermined actions by objects. For example, the development of the story may be branched depending on whether or not permission has been acquired. Virtual items that appear in the stage play event may also change depending on whether or not permission has been acquired. For example, for a user who has acquired the permission to use a shield, the first object OBJ1, which is the main character in the play, will be given a shield. For a user who has acquired the permission to use a sword, the first object OBJ1, which is the main character in the play, will be given a sword, and so on. In the case of a music live event, the songs (performances) sung by the artist may be changed depending on whether or not permission has been obtained. The attributes of virtual objects such as the first object used in the expression may also be changed depending on whether or not permission has been obtained. For example, if the first object is an avatar, the "voice quality" attribute of that avatar may be changed. For example, if the user has not obtained permission, the first object OBJ1 may speak in the voice of voice actor A, and if the user has obtained permission, it may speak in the voice of voice actor B. This makes it possible to provide basically uniform expressions to event participants while also providing individualized expressions according to the user's purchasing behavior, etc.

[0185] Furthermore, user authorization may be obtained at a time other than when the presentation is in progress. For example, user authorization may be obtained one week before the live event in which the first object OBJ1 is displayed in AR. The user purchases the product in advance and sends the product information of the product that has arrived at their home to the server 10Z via the user terminal 20 or the like. There are no particular limitations on how the product information is transmitted to the server 10Z. For example, the user touches the NFC tag embedded in the product to the user terminal 20, and the user terminal 20 reads the data from the product. The processor in the user terminal 20 generates product information based on this data reading and sends it to the server 10Z. Alternatively, the user terminal 20 may use a barcode reader to read the barcode attached to the product, generate product information, and send it to the server 10Z. In addition, the camera in the user terminal 20 may take a picture of the product, and the user terminal 20 or the server 10Z may perform image recognition on the captured data to extract product information. When product information is transmitted to the server 10Z via the user terminal 20, the expression control unit 102 of the server 10Z can recognize that the user has previously obtained permission. The expression control unit 102 may control the expression depending on whether or not the user has previously obtained permission.

[0186] (Display of permission acquisition information and recommendation information according to the progress of the expression) Next, the display of authorization acquisition information and recommendation information according to the progress of the presentation will be described. The authorization acquisition information display unit 103 has the function of displaying authorization acquisition information on the display device in a display manner based on the progress of the presentation. The recommendation information display unit 105 has the function of displaying recommendation information on the display device in a display manner based on the progress of the presentation.

[0187] The progression of an expression refers to changes in the state of an event, such as, in a music concert, the artist finishing the first song and starting the second, making a mistake with the lyrics and starting over, or returning to the stage for an encore. In a theatrical performance, it refers to changes in the state of the event, such as the curtain rising, the actors appearing, the story unfolding, or the scene changing.

[0188] Display modes refer to, for example, the timing of the display, the position of the display, the color of the display, the size of the display, the font of the display, whether or not there are effects to emphasize the display, whether or not there are animations, whether or not it is an AR display, whether or not it is a text display or an image display, whether or not it is a display that accepts user input, etc. The above is merely an example, and other display modes are also possible.

[0189] Figure 24 is a conceptual diagram showing an example of a display by a display device corresponding to at least one embodiment of the present invention. Figure 24 shows the scene after the main character, the first object OBJ1, has defeated the dragon EN.

[0190] In the recommendation area RE shown in Figure 24, recommendation information is displayed indicating a service that plays the song "VICTORY," sung by artist "Mr.SEJ," as background music in the play, based on the content of the expression using the first object OBJ1, namely, "in a story about fighting a dragon, the dragon was defeated." In other words, the authorization acquisition information display unit 103 controls the timing of displaying the recommendation information about the song "VICTORY" triggered by the first object OBJ1, the protagonist, defeating the dragon EN during the progress of the stage play. Furthermore, the authorization acquisition information display unit 103 controls the display manner of the recommendation information about the song "VICTORY" so that the text information and border are emphasized in bold, triggered by the first object OBJ1, the protagonist, defeating the dragon EN during the progress of the stage play.

[0191] In addition, the authorization acquisition information display unit 103 may trigger various conditions, such as when an artist finishes singing three songs and begins talking at a music live event, when a predetermined number of seconds have elapsed since the climax scene at a theatrical live event, or when a certain player uses a predetermined item at a game competition event, and display recommendation information on the display device DISP in various display modes.

[0192] The second area shown in Figure 24 is located on the upper left side of the display device DISP, and the authorization acquisition information R7 and R8 are displayed in this second area. The authorization acquisition information R7 and R8 is information that allows the user to acquire the right to receive the service of playing the song "VICTORY" as background music in the game. If the authorization acquisition information is displayed, it may prevent the user from becoming immersed in the performance taking place in the event. Therefore, in this embodiment, the authorization acquisition information display unit 103 controls the display mode so that when the protagonist, the first object OBJ1, defeats the dragon EN, the display position of the authorization acquisition information R7 and R8 moves from the left side of the display device DISP (see Figure 23) to the upper left side of the display device DISP (see Figure 24). In other words, the authorization acquisition information display unit 103 displays the authorization acquisition information on the display device in a display mode based on the progress of the performance.

[0193] The manner in which the display position of the authorization acquisition information can be changed is not limited to the above-described manner. For example, the authorization acquisition information display unit 103 or the recommendation information display unit 105 may display the authorization acquisition information (second area) or recommendation information (recommendation area RE) on the display device DISP so as to superimpose it on other objects reflected on the display device DISP. Furthermore, the authorization acquisition information or recommendation information may be displayed in an opaque state. This makes it possible to hide certain objects that are not to be perceived by the user as part of the expression at events such as music concerts or theatrical performances using the authorization acquisition information (second area) or recommendation information (recommendation area RE).

[0194] For example, one representation involves the object display unit 101 displaying the first object OBJ1 on the display device DISP so that the first object OBJ1 is superimposed on video data (including not only downloaded playback, but also streaming playback and live broadcast data) of live events such as music concerts and theatrical performances held in the real world. This representation can be interpreted as a mixed event that combines an event in the real world with augmented reality. An example of a predetermined object that the user does not want to perceive in the above representation is event staff positioned below the live stage in the real world in a mixed event. These event staff members stand or crouch below the stage, waiting to ensure the safety of the audience and artists.

[0195] As one aspect of the seventh embodiment, an event using augmented reality can be held, and merchandise sales and service provision can be carried out at said event.

[0196] As one aspect of the seventh embodiment, information for obtaining permission can be displayed when a user expresses an intention to obtain permission, such as to purchase a product. This allows users who do not intend to obtain permission to focus on participating in the event.

[0197] As one aspect of the seventh embodiment, by separating the first area used for events from the second area for purchasing products, the display for purchasing products does not hinder the user's participation in the event.

[0198] As one aspect of the seventh embodiment, different effects can be displayed during an event depending on whether the user has acquired permission or not. This allows users to experience different events even if the event is essentially the same, thus personalizing the event.

[0199] As one aspect of the seventh embodiment, the user can express their intention to acquire permissions in an augmented reality space.

[0200] As one aspect of the seventh embodiment, information for obtaining permissions can be displayed in an appropriate manner in response to an event.

[0201] As one aspect of the seventh embodiment, the event venue itself can be set up in augmented reality space, and users can freely take actions such as participating in the event, purchasing goods, or receiving services at the event venue represented by augmented reality.

[0202] As one aspect of the seventh embodiment, it is possible to recommend appropriate products or services to users in events taking place in augmented reality space.

[0203] As one aspect of the seventh embodiment, recommendation information can be displayed in an appropriate manner for an event.

[0204] As described above, each embodiment of the present application solves one or more of the shortcomings. Note that the effects of each embodiment are non-limiting effects or examples of effects.

[0205] In each of the embodiments described above, the multiple user terminals 20, 20A to 20C and the server 10 execute the various processes described above in accordance with various control programs (for example, event provision programs) stored in their own storage devices.

[0206] Furthermore, the configuration of the event provision system 100 and the event provision system 100A is not limited to the configurations described as examples of each embodiment above. For example, the server 10 may perform some or all of the processing described as being performed by a user terminal, or any of the user terminals 20, 20A to 20C may perform some or all of the processing described as being performed by the server 10. Also, any of the user terminals 20, 20A to 20C may perform some or all of the storage unit (memory device) provided by the server 10. In other words, the event provision system 100 or the event provision system 100A may be configured such that one of the user terminals or the server performs some or all of the functions provided by the other.

[0207] Furthermore, the program may be configured to implement some or all of the functions described above as examples of each embodiment in a standalone device that does not include a communication network.

[0208] [Note] The above-described embodiments are written in such a way that at least the following inventions can be put into practice by a person with ordinary skill in the art to which the invention pertains. [1] Event provision program, On the server, The object display function superimposes a virtual object, the first object, onto the real world and displays it on the display device, A representation control function that controls the representation using one or more virtual objects, including the first object, A function for displaying authorization information, which causes the display device to display information (hereinafter referred to as "authorization acquisition information") that allows the user to acquire predetermined authorizations for the object, the event, or the product or service related to the expression, An event provision program designed to achieve this goal. [2] The authorization acquisition information display function includes a function that displays the authorization acquisition information on the display device when it acquires user input operation information for product information, which is information indicating the product or service displayed on the display device. The event provision program described in [1] to make this happen. [3] The authorization acquisition information display function includes a function to display the authorization acquisition information in a second area different from the first area in the display device where the first object is displayed. An event provision program as described in [1] or [2] to make this happen. [4] The expression control function controls the expression depending on whether or not the authority has been acquired. An event provision program described in any one of the following [1] to [3] to achieve this. [5] The display of the aforementioned authorization acquisition information is superimposed onto the real world. An event offering program as described in any one of the items [1] through [4]. [6] The aforementioned authorization acquisition information display function includes a function that displays the authorization acquisition information on the display device in a display manner based on the progress of the aforementioned expression. An event provision program described in any one of the following [1] through [5] to achieve this. [7] To the aforementioned server, Further realization of the augmented reality diorama display function, which displays an augmented reality diorama superimposed on real space on a display device. At least one of the first object and the authorization acquisition information is displayed by the display device together with the augmented reality diorama. An event offering program as described in any one of the items [1] through [6]. [8] To the aforementioned server, A recommendation information display function that displays recommendation information on the display device indicating products or services for which the user is recommended to acquire authorization, in accordance with the content of the aforementioned expression, the user's input information, or predetermined stored information. An event provision program described in any one of the following [1] through [7] to achieve this. [9] The recommendation information display function includes a function that displays the recommendation information on the display device in a display manner based on the progression of the expression. The event provision program described in [8] to make this happen.

[10] A program for enabling a user terminal capable of communicating with the server to implement at least one of the functions that the event provision program described in any one of items [1] to [9] enables the server to implement.

[11] A server on which one of the event delivery programs described in [1] through [9] is installed.

[12] An event delivery system comprising a communication network, a server, and user terminals, An object display means that superimposes a first object, which is a virtual object, onto the real space and displays it on a display device, A representation control means for controlling a representation using one or more virtual objects, including the first object, A means for displaying authorization acquisition information, which causes the display device to display an indication (hereinafter referred to as "authorization acquisition information") that allows the user to acquire predetermined authorization for the object, the event, or the product or service related to the expression, An event provision system, including the above.

[13] Event provision program, On the user terminal, The object display function superimposes a virtual object, the first object, onto the real world and displays it on the display device, A representation control function that controls the representation using one or more virtual objects, including the first object, A function for displaying authorization information, which causes the display device to display an indication (hereinafter referred to as "authorization acquisition information") that allows the user to acquire predetermined authorization for the object, the event, or the product or service related to the expression, An event provision program designed to achieve this goal.

[14]

[13] A user terminal on which the event delivery program described above is installed.

[15] Event provision program, In a computer device, The object display function superimposes a virtual object, the first object, onto the real world and displays it on the display device, A representation control function that controls the representation using one or more virtual objects, including the first object, A function for displaying authorization information, which causes the display device to display an indication (hereinafter referred to as "authorization acquisition information") that allows the user to acquire predetermined authorization for the object, the event, or the product or service related to the expression, An event provision program designed to achieve this goal.

[16] A method of providing events using a computer device, The object display process involves superimposing a virtual object, the first object, onto the real space and displaying it on a display device. A representation control process that controls the representation using one or more virtual objects, including the first object, The process includes displaying authorization acquisition information on the display device, which displays an indication (hereinafter referred to as "authorization acquisition information") that allows the user to acquire predetermined authorization for the object, event, or product or service related to the expression, How to provide the event.

[17] An event delivery method comprising an event delivery system comprising a communication network, a server, and a user terminal, The object display process involves superimposing a virtual object, the first object, onto the real space and displaying it on a display device. A representation control process that controls the representation using one or more virtual objects, including the first object, The process includes displaying authorization acquisition information on the display device, which displays an indication (hereinafter referred to as "authorization acquisition information") that allows the user to acquire predetermined authorization for the object, event, or product or service related to the expression, How to provide the event. [Industrial applicability]

[0209] According to one embodiment of the present invention, an event provision program, an event provision system, and an event provision method are useful for holding an event using augmented reality and for selling goods and providing services at the event. [Explanation of Symbols]

[0210] 10, 10A, 10B, 10C, 10D, 10Z Server 11, 21, 41, 61 processors 12, 22, 42, 62 memory 13, 23, 43, 63 storage device 20, 20A, 20B, 20C, 20Z User Terminals 30 Communication Networks 40 Other Servers 60. Interceptor device 64 Cameras 65 Input device 66 Display device 100, 100A Event Provisioning System 101, 201, 501 Object display section 102, 102D, 202, 502 Expression Control Unit 103, 103B, 103C, 203, 503 Information display unit for obtaining authorization 104 Augmented Reality Diorama Display Unit 105 Recommendation Information Display Unit A1-A5 Corresponding Characters ARD1~ARD3 Augmented Reality Diorama CTR Controller DISP display device OBJ1 First Object OBJ2 Second Object R1~R8 Permission Acquisition Information RE Recommendation Area RR Resistance Area SP (Special Service) Representative

Claims

1. A program that uses augmented reality, On the server, The first object, which is a virtual object, is superimposed onto the real space and displayed on the display device; this is an object display function. A representation control function that controls the representation using one or more virtual objects, including the first object, A product information display function that, when it obtains first input operation information indicating that a predetermined first input operation has been performed by a user on the first object displayed on the display device, displays product information including string information and / or image information corresponding to the product on the display device, When the first input operation information is obtained, or when a second input operation information is obtained indicating that a predetermined second input operation has been performed by the user on the product information displayed on the display device after the product information has been displayed, the display device will show information (hereinafter referred to as "authorization acquisition information") that allows the user to acquire predetermined authority over the product, which is a real object in real space related to the object or representation, thereby realizing an authorization acquisition information display function, The aforementioned authorization acquisition information is information that, upon receiving the aforementioned display of such information, allows the user to express their intention to acquire authorization. program.

2. The expression control function includes a function for controlling the movement of the first object, The program according to claim 1.

3. The first object is generated based on 3D modeling data. The program according to claim 1.

4. The program according to any one of claims 1 to 3, wherein the aforementioned prescribed authority includes the authority to purchase the product or to be leased the product.

5. The program according to any one of claims 1 to 3, wherein the aforementioned prescribed authority includes the authority to use the physical object.

6. A program that uses augmented reality, On the server, The first object, which is a virtual object, is superimposed onto the real space and displayed on the display device; this is an object display function. A representation control function that controls the representation using one or more virtual objects, including the first object, A product information display function that, when it obtains first input operation information indicating that a predetermined first input operation has been performed by a user on the first object displayed on the display device, displays product information including string information and / or image information corresponding to the product on the display device, When the first input operation information is obtained, or when a second input operation information is obtained indicating that a predetermined second input operation has been performed by the user on the product information displayed on the display device after the product information has been displayed, the display device will show information (hereinafter referred to as "authorization acquisition information") that allows the user to acquire predetermined authority for the product, which is data related to the object or the representation, thereby realizing an authorization acquisition information display function, The aforementioned authorization acquisition information is information that, upon receiving the aforementioned display of such information, allows the user to express their intention to acquire authorization. program.

7. The expression control function includes a function for controlling the movement of the first object, The program according to claim 6.

8. The first object is generated based on 3D modeling data. The program according to claim 6.

9. The program according to any one of claims 6 to 8, wherein the aforementioned prescribed authority includes the authority to purchase or acquire the aforementioned data.

10. The program according to any one of claims 6 to 8, wherein the aforementioned prescribed authority includes the authority to copy or modify the aforementioned data.

11. The program according to any one of claims 6 to 8, wherein the aforementioned predetermined authority includes the authority to use the aforementioned data.

Images