Online tour delivery system, online tour delivery program, and online tour delivery method

Abstract

In a system that recreates tourist facilities and other attractions in a virtual space to provide a realistic online tourist experience, participants will be able to choose whether or not to have a guide join them. [Solution] The online tour provision system 1 sets up rights holders and guides who have rights to tourist destinations or tourist facilities (tourist facilities, etc.), sets up tourist content by recreating the tourist facilities, etc. in a virtual space and linking content to it, accepts participants for the service that provides the tourist content, accepts the participant's choice of whether or not they need a guide, and distributes the profits obtained to at least the rights holders.

Description

Technical Field

[0001] The present invention relates to an online tour providing system, an online tour providing program, and an online tour providing method.

Background Art

[0002] Digital twins that reproduce real space in virtual space have become widely known. For example, the interior of a building or the like is captured by an imaging device (such as a camera) to acquire necessary data, and the building or the like is constructed and reproduced on a virtual space.

[0003] For example, Non-Patent Document 1 reproduces real space in virtual space based on a huge amount of imaging data, and allows a user to know the local situation and the like via street view. Further, Non-Patent Document 2 discloses digital twin technology and a service using the technology.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Non-Patent Documents

[0005]

Non-Patent Document 1

Non-Patent Document 2

[0006] Digital twin technology is expected to have applications in fields such as tourism. By showcasing videos of tourist destinations online, it is hoped that viewers who are impressed by the videos will actually visit those destinations. Alternatively, if the video itself has value, it's conceivable that compensation could be paid for providing the video.

[0007] Furthermore, these business models also require handling processes associated with tourism, such as reservations, fee collection, and revenue sharing.

[0008] For example, Patent Document 1, although not a digital twin, discloses an online tourism system using drones, and also discloses reservation and payment management, as well as revenue sharing.

[0009] For example, in tourist destinations that include historical buildings and other tourist facilities, guides (tour guides / tour conductors) who possess deep knowledge of the history and historical value and can explain and introduce it play a crucial role. Participants can obtain information from signboards and other materials installed on-site, but their understanding will be further enhanced by receiving explanations from a guide.

[0010] For example, the guide can adjust their explanation based on the participants' reactions. If a participant looks confused, they can explain it again in simpler terms, or if a participant shows a strong reaction, they can provide more detailed explanations on that point. Another advantage of being a guide is that they can answer participants' questions in real time.

[0011] In other words, to widely implement tourism services using digital twin technology, it is essential to not only create attractive content but also to build a system that can provide smooth service to participants. Furthermore, guides with deep knowledge can add value to tourist destinations by highlighting their attractions.

[0012] On the other hand, some participants prefer to explore tourist attractions and other sites independently, without the guidance of a tour guide. Therefore, it would be preferable if participants could choose whether or not they require guidance from a guide. Improving usability in this way is particularly important for the widespread adoption of virtual tourism services and other related technologies.

[0013] The aforementioned prior art documents do not describe the participation of guides, or in particular, a system in which participants can choose whether or not to have a guide.

[0014] By the way, guides usually need to either accompany tourists or live near tourist destinations. However, this limits the ability of individuals who possess deep knowledge of the history and culture of a particular region (tourist destination) and are suitable to act as guides to work as such.

[0015] For example, suppose there is a person living in City A who is interested in the history and culture of City B, which is far away. They have acquired knowledge through self-study and possess enough knowledge to pass certification exams held by organizations in City B. This person would like to work, either full-time or part-time, to share the charms of City B with others, but due to their distance, this may not always be possible.

[0016] A shortage of tour guides is a problem in some tourist destinations, and solutions to this problem are desired. Furthermore, it is desirable to enhance the promotional content for tourist destinations through the creativity of the guides. [Overview of the project] [Problems that the invention aims to solve]

[0017] The problem we are trying to solve is that in systems that recreate tourist facilities and other attractions in a virtual space to provide a realistic online tourist experience, participants cannot choose the system components that are essential for tourism, particularly whether or not to include a guide. In addition, enriching the tourism content is necessary to broaden the options available to participants. [Means for Solving the Problem]

[0018] The present invention is most characterized by: setting a right holder who has rights to a sightseeing spot or a sightseeing facility (referred to as sightseeing facility, etc.) and a guide; setting sightseeing content by reproducing the sightseeing facility, etc. in a virtual space and associating content therewith; accepting participants in a service that provides the sightseeing content, and accepting the participant's selection of whether the guide is required at this time; and distributing the obtained revenue to at least the right holder.

[0019] The present invention has been made in view of the above problems, and employs, for example, the following means. That is, a right holder setting unit that sets a right holder who has rights to a sightseeing spot or a sightseeing facility (referred to as "sightseeing facility, etc."); a guide setting unit that sets a guide for the sightseeing facility, etc.; a sightseeing content creation unit that creates sightseeing content by associating text information, image information, or audio information with a three-dimensional model of the sightseeing facility, etc., constructed from at least image pickup data of the sightseeing facility, etc. obtained by an image pickup device; a participation application receiving unit that receives applications for participation from participants for a service that provides the sightseeing content; a guide necessity determining unit that determines whether the guide is required; a participation fee calculation unit that calculates a participation fee for the participant; an access authority granting unit that grants the participant access authority to the sightseeing content; a service publishing and providing unit that receives the participant's access to the sightseeing content and provides the service; a three-dimensional image changing unit that changes the image displayed on the display unit of the participant terminal by an operation on the participant terminal by the participant or an operation on the guide terminal by the guide; a distribution calculation unit that calculates at least the distribution to the right holder from the participation fee; comprising: the access authority granting unit is If a guide is required, grant the aforementioned access rights to the participants and the guide. The present invention provides an online tour delivery system characterized by granting the aforementioned access rights to participants when a guide is not required. [Effects of the Invention]

[0020] The online tour provision system of the present invention provides a realistic tourist experience by creating tourist content (i.e., recreating tourist facilities in a virtual space) by associating text information, image information, or audio information with a three-dimensional model of the tourist facilities, etc., constructed from image data of the tourist facilities, etc., by at least an imaging device. In addition, it has the advantage of being able to provide services that meet the needs of participants, such as allowing participants to decide whether or not they need a guide when applying for the tourist content.

[0021] Furthermore, as guides can conduct tours online, individuals with deep knowledge of the aforementioned tourist destinations can participate in the tours, thus solving the problem of guide shortages. [Brief explanation of the drawing]

[0022] [Figure 1] This is a diagram (network diagram) showing an overview of online tour provisioning system 1. [Figure 2] This diagram shows the service (online tour) provision screen. [Figure 3] This is a diagram showing the content display screen. [Figure 4] This is a diagram showing the screen in auto mode. [Figure 5] This is a diagram showing the quiz display screen. [Figure 6] This is a flowchart showing an overview of the online tour delivery process. [Figure 7] This is a flowchart showing the process of providing tourism content. [Figure 8] This is a hardware configuration diagram for server 10. [Figure 9]This is a hardware configuration diagram of a terminal (e.g., participant terminal 20). [Modes for carrying out the invention]

[0023] Embodiments of the present invention will be described with reference to the drawings. In the following embodiments, the same or corresponding parts may be denoted by the same reference numerals, and their descriptions may be omitted as appropriate. Furthermore, the drawings used below are for illustrative purposes only and may differ from the actual device configuration, user interface (UI), data structure, etc.

[0024] (Summary of the embodiment) The outline of this embodiment will be explained with reference to Figure 1. Figure 1 is a diagram (network diagram) showing an overview of the system (hereinafter referred to as "online tour provision system 1") that performs processing using the online tour provision program P1 of this embodiment. In the online tour provision system 1 of this embodiment, (usually multiple) servers 10 and terminals of participants, guides, travel agencies, and rights holders (referred to as participant terminal 20, guide terminal 30, travel agency terminal 40, and rights holder terminal 50, respectively) are connected to each other via a network N. The terminals of participants and guides are, for example, PCs (personal computers) or HMDs (head-mounted displays).

[0025] Server 10 constructs a three-dimensional model in a virtual space to recreate tourist facilities, etc., and associates various types of information (text, images, audio, etc.) with the three-dimensional model to create tourist content, and provides tours to participants to enjoy this tourist content. Participants select the service (tour) they wish to participate in from the available options, including whether or not they require a guide, and then apply to participate and pay the participation fee. After payment is complete, participants join the tour (connecting online) at a designated time, for example. Participants move around virtual tourist facilities and other locations through their avatars and enjoy the tour by viewing various information associated with three-dimensional models. Figure 2 shows a diagram illustrating a participant taking part in a tour (a diagram showing the service (online tour) provision screen) (when participant terminal 20 is a PC). Within the virtual space, participants and / or guides (hereinafter referred to as "participants, etc.") can move and converse freely as if moving in the real world. Further details will be provided later. The revenue generated from participation fees will be distributed to the rights holders of tourist facilities, the participating guides, and the travel agencies that facilitated the tours.

[0026] (Details of the embodiment) The online tour provision system 1 according to this embodiment will be described in detail below. The online tour provision system 1 includes a computer (server 10) equipped with an online tour provision program P1, and provides a system that enables participants to experience a realistic and highly flexible online tour. In other words, the online tour provision system 1 is a system in which the information processing performed by the online tour provision program P1 is concretely realized using hardware resources. The following describes, in order, the components that make up Online Tour Provisioning System 1: 1. User Interface, 2. Program Processing, 3. Data, and 4. Hardware Configuration.

[0027] (Definition of terms) Here, I will define some terms. The "administrator" is the administrator of Online Tour Provisioning System 1 and has special privileges over Online Tour Provisioning System 1. The administrator and the person who provides tourism content services through Online Tour Provisioning System 1 (the person who sets up the tour) may be the same person or different person, and a strict distinction is not made here.

[0028] A "rights holder" is a person who holds rights to a tourist destination or tourist facility. For example, this could be the owner or manager of a building that displays exhibits or other items, or a building with historical or academic value, or the rights holder of land that is part of a natural asset. Rights holders can be individuals or corporations. If there are multiple rights holders, it is sufficient for at least some of them to be considered rights holders. Furthermore, rights holders are not limited to direct rights holders; they may also be persons with authority, such as those entrusted with management by the direct rights holder, or agents. For example, if the tourist destination or its owner is the national or local government, a rights holder could be an organization entrusted with its management. In addition, the rights referred to here can be tangible or intangible. For example, rights relating to land, buildings, goods, or services. For simplicity, "tourist destination or tourist facility" will be written as "tourist facility, etc." Tourist destinations and tourist facilities are not separate concepts and can overlap. For example, a tourist facility may be the tourist destination itself, or a tourist destination may include multiple tourist facilities. Regarding the terms "tourist destination" or "tourist facility," whether they refer to something in the real world or a virtual world is determined by the context. Usually, they refer to tourist destinations or facilities in the real world.

[0029] "Participant" refers to a person who participates in the tourism content provision service set up in the online tour provision system 1 of this embodiment. The term "participant" includes both those who are planning to participate at the application stage and those who are participating in the service using the system. A "guide" is a person who guides participants in the tourism content provision service set up in the online tour provision system 1 of this embodiment. They may also be referred to as a tour guide, tour conductor, or host. Furthermore, a guide may be the person who creates the tourism content and sets up the tour. In this case, the guide has privileges over the tours they set up. As mentioned above, participants and / or guides may be referred to as "participants, etc."

[0030] "Imaging data" refers to image data (including still and moving images) acquired by the imaging device. It may also include audio data recorded by the recording device provided by the imaging device. For example, in this embodiment, in addition to pass-through video (image data captured by the imaging device and displayed in real time on the HMD's display), there is also point cloud data and image data for combining with point cloud data in three dimensions. The term "video" refers to both still and moving images, and to both real-time and recorded images. The term "video" is sometimes used interchangeably with "image" (especially moving images). A "three-dimensional model" is a model created as three-dimensional, three-dimensional data in a virtual space created by a computer (information processing device).

[0031] "Tourism content" is a virtual object in a virtual space that associates various types of information (text, images, audio, etc.) related to tourist facilities, etc., with (usually multiple) elements, using a three-dimensional model. In the following, the term "content" may be used in its general sense. For example, the various types of information related to the tourist facilities mentioned above can be considered content. "Tour" is the Japanese transliteration of "tour," but here it specifically refers to a service (or a collection of services) that provides tourist content, or a package of tourist content services. For example, it could be described as a tourism content that recreates "Shuri Castle" in a virtual space, and a tour that includes that tourism content (for example, a "Shuri Castle sightseeing tour"). A single tour may include multiple sightseeing attractions. For example, an "Okinawa sightseeing tour" that includes sightseeing attractions related to "Shuri Castle" and sightseeing attractions related to "Churaumi Aquarium."

[0032] A "tour tag" is a tag associated with a specific point of interest, such as a landmark, within a particular tour. Sometimes, various types of information (text, images, audio, etc.) are associated with the tag, while other times, there may be no corresponding tourist content (it simply indicates a location). For example, within the tourist content of "Shuri Castle," tags like "Shureimon Gate," "Main Hall," or "Souvenir Shop" might be used. An "avatar" is a character (object) or icon placed in a virtual space as a representation of a participant. Participant avatars and guide avatars are also referred to as participant objects and guide objects, respectively. Here, an object is one of the data in the virtual space, and participant objects and guide objects correspond to participants and guides, respectively. Participants can operate their avatars (objects) by operating their own terminals (inputting into the input section). In the virtual space, each participant object has a virtual camera. The image captured by this virtual camera becomes the participant's field of view, and the processor (122) displays this image on the display section of the participant's terminal. In this embodiment, the processor 122 displays avatars as semi-transparent human figures in the virtual space, and one participant can see the avatars of other participants. Furthermore, the processor 122 can change the viewpoint position through the operation of the participants. That is, the virtual camera can be set at the eye level of the avatar, or it can be placed slightly behind and above the avatar to create a viewpoint where the user's own avatar is viewed from slightly behind.

[0033] In the following, when the term "○○ processing" is used, it means that the computer's processor executes processing based on the "○○" program stored in the program storage. The same word will be used in place of "○○" throughout this paragraph. In other words, the "○○" program is a program that makes the computer function as a "○○" means by executing the "○○" process. Furthermore, this means that the control unit equipped with the processor also functions as a "○○" unit (or "○○" device). In this case, the "○○" section means that it executes the "○○" process based on the "○○" program. Furthermore, the "○○" process is expressed as a "○○" step (or "○○" procedure).

[0034] For example, the online tour provision program P1 is a program that makes a computer function as an online tour provision means by executing online tour provision processing. In this case, the control unit 12 of the computer equipped with the processor 122 functions as an online tour provision unit (or online tour provision device).

[0035] In the online tour provision system 1, each terminal (computer) of the participant, guide, and travel agency is equipped with a processor. However, when simply referring to a processor, it refers to the processor that performs processing by the online tour provision program P1, and in this embodiment, the processor 122 of the server 10.

[0036] For simplicity, in the following, the action of "the processor 122 of server 10 receiving a request from a terminal and returning data to be displayed in the terminal's browser" may be described as "the processor 122 displays (causes) the data to be displayed in the terminal's browser" or "the processor 122 displays (causes) the data to be displayed." Similarly, the action of "the processor 122 of server 10 causing data to be stored in the data storage unit 14b of storage unit 14" may be described as "the processor stores (data)." In addition, while it is the avatars of participants and guides that move (etc.) in the virtual space, for simplicity, it may be written as "participants and guides move (etc.)."

[0037] 1. User Interface (UI) First, the interface that the online tour provisioning system 1 of this embodiment displays on the participant terminal 20, etc., will be explained with reference to a diagram. The interface described below is a simplified version of the one that the processor 122 displays on the display unit (display) of the participant terminal 20 or the guide terminal 30.

[0038] Furthermore, only icons related to functions necessary for explanation will be displayed, and other publicly known icons will be omitted. For example, the exit button (× button) for closing (exiting) the screen has been omitted.

[0039] Online tour provision system 1 provides, for example, a function that allows participants and guides to gather online and tour (virtual) tourist destinations together. Each tour requires at least one participant. Depending on the participant's choice, the guide may also participate in the tour. Participants can move around the virtual space freely, or be controlled by a guide, and view sightseeing content.

[0040] Online tour provision system 1 offers multiple types of tours. Participants select and register for one of the multiple types. The multiple types include at least two categories: one based on the number of participants and another based on the content.

[0041] In this embodiment, there are eight main types of participants. This can be divided into two categories based on whether there is one participant or multiple participants (friends or acquaintances), whether there is a guide or not, and whether or not a third party is participating in the tour (2 possibilities x 2 possibilities x 2 possibilities). Here, "third party" refers not to the aforementioned "multiple" people who are acquaintances of the participants, but rather to complete strangers.

[0042] The participation fee varies depending on the type of tour. For example, if there are multiple participants, the per-person fee will be lower (by the organizer, etc.) than when there is only one participant. If there is no guide, the participation fee will be lower than when there is a guide. And if you participate in the tour with a third party, the participation fee will be lower than when you do not.

[0043] A single participant is sometimes referred to as a "standalone," while a group of multiple participants, such as friends, is sometimes called a "group." Furthermore, the presence of a guide may be referred to as "with a guide," and the absence of a guide as "without a guide." Similarly, the presence of a third party may be referred to as "with a third party," and the absence of a third party as "without a third party."

[0044] Furthermore, in this embodiment, there are two main types of content: "Manual Mode," in which participants are free to act as they please within the virtual space, and "Auto Mode," in which participants' actions are controlled by an administrator or guide.

[0045] In auto mode, participants do not control the movement of their own avatars. For example, they will move through the virtual space according to a predetermined route and time set by an administrator, or their avatars will move in conjunction with the movement of a guide avatar. In addition, administrators or guides may arbitrarily control the movements of participant avatars.

[0046] Even in manual mode, the processor 122 displays guiding icons such as arrows on the screen to indicate the correct route.

[0047] Participation fees vary depending on the type of content. Auto mode has a lower participation fee than manual mode.

[0048] If there is no guide, participants can choose between manual mode or automatic mode.

[0049] When a guide is present, participants can still choose between manual and automatic modes, but the guide can control the participants' actions.

[0050] Specifically, when the guide activates "guidance mode" by operating the guide terminal 30 (for example, by operating the guide mode icon (not shown) displayed on the screen) to acquire the participants' action rights, the guide can gather the avatars of the participants taking part in the tour near the guide's (own) avatar. The distance between the participant's avatar and the guide's avatar can be set arbitrarily. In this embodiment, this distance is zero (the participant's avatar and the guide's avatar are in the same position).

[0051] In this embodiment, the guide's field of view is shared with the participants, and the participants view the objects, scenery, etc., that are being captured by the virtual camera of the guide object. In this embodiment, while the guiding mode is activated, the processor 122 displays a flag icon on the screens of both the participant and the guide.

[0052] When the guide deactivates "leader mode," the participant avatars can resume their actions from the location where they were gathered (where the guide avatar is located) or from where they were before being gathered (free movement in manual mode, or continuation of the tour in auto mode).

[0053] For example, in a scenario with one participant, no guide, no third parties, and in manual mode, the participant can explore (virtual) tourist destinations alone and move freely within the virtual space.

[0054] Additionally, if there is only one participant, no guide, no third parties, and the activity is in auto mode, a guide video is provided. The guide video is a video (including audio and video) containing explanations by the guide. The processor 122 of server 10 displays a guide video and controls the participants' locations in the virtual space.

[0055] For example, suppose there is a tourist facility where Room A, a corridor, and Room B are connected. When the guide video is explaining Room A, the participant is also in Room A, and by changing their viewpoint from that location, the participant can view Room A in 360°. When the guide video transitions to Room B, the participant's location and viewpoint smoothly move from Room A to the corridor and then to Room B. This means that the participant can also view the corridor, which is the route from Room A to Room B. The duration of this transition can be set as appropriate by the administrator or guide.

[0056] Furthermore, the URLs associated with different types of tourism content may vary, such as by participant type or content type. This can help reduce server load. The following will explain using specific screenshots.

[0057] Figure 2 shows the service (online tour) provision screen. Participants can take action within the virtual space by performing actions on the screen (such as clicking and pointing).

[0058] While participants can converse with each other within the virtual space, the processor 122 recognizes the location of the participants and the walls, floors, or ceilings that divide the space, and controls the volume of the sound they hear.

[0059] For example, two or more participants can converse if they are close to each other, but they cannot converse if they are far apart. Furthermore, even if two or more participants are in close proximity to each other, they cannot converse if there is a wall, floor, or ceiling between them—that is, if they are in different rooms or on different floors, even if they are physically close. Details will be explained in the Program Processing section.

[0060] The service (online tour) delivery screen includes the main screen UI-11, participant display section UI-12, guide display section UI-13, operation icon display section UI-14 (UI-14A, UI-14B), and tour tag icon UI-15.

[0061] Here, if the participants' or guides' terminals are standard PCs or similar devices, and the display unit is a flat-screen display, the information will be displayed flatly as shown in Figure 2. On the other hand, if the participant's or guide's device is an HMD (Head-Mounted Display), the displayed image is stereoscopic, and the HMD wearer can enjoy an immersive experience as the tourist destination is displayed no matter where they look (360° - up, down, front, back, left, or right).

[0062] The main screen UI-11 displays video from a virtual camera of a participant or other object (avatar). Specifically, it shows virtual tourist facilities, etc., created based on data acquired in advance by taking pictures with an imaging device.

[0063] Here, we will explain the operation of the avatar in this embodiment. Participants can move to a location by pressing (clicking, pointing, etc.) on the floor area (displayed on the main screen UI-11) that is slightly ahead. In other words, they can move their avatar.

[0064] Furthermore, participants can change their viewpoint without changing their position. If the participant's device is an HMD (Head-Mounted Display), the HMD's sensors detect the movement when the participant moves their head and acquire it as input. The processor 122 receives this input and changes the angle of the virtual camera (yaw, pitch, or roll). In other words, if a participant wearing an HMD turns to the right, the image from that direction is displayed on the device's display (main screen UI-11).

[0065] If participants' devices are desktop PCs, they can change their viewpoint by dragging and dropping on the main screen UI-11. For example, suppose you drag (click with the mouse, for instance) in the upper right corner of the main screen UI-11, and then, while maintaining that state (maintaining the mouse click state), move the cursor to the center. In this case, the video that was in the upper right corner of the main screen UI-11 will move to the center of the main screen UI-11. In other words, from the perspective of the participants, the video displayed on the main screen UI-11 will be the video they saw when they were facing the upper right.

[0066] Furthermore, some buttons on input devices, such as the buttons on the controllers included with HMDs or the buttons on the mouse and keyboard of a desktop PC, become so-called action buttons. By pressing specific buttons, participants can perform some kind of action within the virtual space (hereinafter sometimes referred to as "action operation").

[0067] For example, when an avatar approaches a location where content is embedded (a tour tag), processor 122 displays the tour tag icon UI-15 (described later) on the main screen UI-11. When a participant then operates a specific button or other action, the embedded content is played.

[0068] In other words, when a participant object is within a certain distance from the location where content is embedded in the virtual space, the processor 122 accepts a predetermined operation from the participant and provides the content embedded in that location to the participant (referred to as "embedded content provision processing").

[0069] Furthermore, known techniques can be used as appropriate for the movement, viewpoint control, and action control described above.

[0070] Returning to the explanation of Figure 2, the participant display unit UI-12 displays images (usually moving images) of the participants taken by an imaging device (camera) built into or connected to the participant terminal 20. Similarly, the guide display unit UI-13 displays an image (usually a video) of the guide being captured by an imaging device (camera) built into the guide terminal 30 or connected to the participant terminal.

[0071] The operation icon display unit UI-14 (UI-14A, UI-14B) provides participants with pre-configured functions.

[0072] As shown in Figure 2, in this embodiment, the operation icon display unit UI-14 comprises operation icon display unit UI-14A and operation icon display unit UI-14B. Here, the operation icon display unit is shown separately as two units, but these may be integrated into one, or conversely, divided into three or more units. The following explains each of these points.

[0073] The normal operation icon display section has multiple operation icons, each with a shape and color corresponding to its function. However, for simplicity, they will be represented here using the letters A-G and the katakana characters A-E. Furthermore, although it is stated below that "this is an icon for participants to do ○○ to...", this is because processor 122 controls it to produce such an effect.

[0074] The icons provided by the UI-14A operation icon display unit are as follows: A: Guide display icon, B: Participant display icon, C: Tour tag list display icon, D: Auto mode icon, E: Viewpoint change icon, F: Zoom in icon, and G: Minimap icon.

[0075] A: The guide display icon is an icon that allows participants to choose whether or not to display the guide's image on the screen. Once selected, the guide display UI-13 disappears, and when selected again, the guide display UI-13 reappears. In this embodiment, participants can hear the guide's voice even if they do not display the guide's image. However, it is also possible to configure the settings so that the guide's voice is not heard (muted) when the guide's image is not displayed.

[0076] B: The participant display icon is an icon that allows participants to choose whether or not to display their images on the screen. In addition to the operation method, the fact that audio can be heard when the image is not displayed, and that the audio can be muted, are the same as with the guide display icon.

[0077] C: The tour tag list icon displays a list of locations for tour tags in the tour that a participant is taking. For example, the entrance, garden, first room, second room, or rooftop (in the example above, Shureimon Gate, main hall, or gift shop).

[0078] When a participant selects a location displayed in the list, they can move to that location. Specifically, processor 122 performs a process to move the participant's avatar to the selected location (the corresponding location in the virtual space). The speed of movement can be set by the administrator, allowing participants to move instantly or slowly.

[0079] In this embodiment, participants can instantly move to a location by selecting a tour tag. Because they can instantly move to their desired location, they can efficiently view tourist facilities. In addition, even in tourist facilities that have accessibility issues, such as those with barrier-free features, participants and guides can move around freely, which is an advantage.

[0080] D: The auto mode icon is used by participants to switch to auto mode. When the auto mode icon is selected, participants can sequentially visit virtual tourist facilities (including tourist content) according to a predetermined route for each tour (auto mode). At this time, processor 122 controls the participant's location information, so participants do not need to perform any movement operations themselves. In this embodiment, even if a participant applies to participate in manual mode, they can switch to auto mode.

[0081] E: The viewpoint change icon is used by participants to obtain the viewpoints of other participants. In other words, participants who select the viewpoint change icon can see what kind of video other participants are watching. Specifically, in response to an action by a participant, the processor 122 performs a process (referred to as "viewpoint switching process") to display an image from the virtual camera of another person's avatar (object) on the display unit of the participant who performed the action.

[0082] F: The zoom icon is used to enlarge what participants see in their field of view. It can enlarge nearby objects as well as distant objects. For example, it can be used to make small print on nearby objects that are difficult to read, or to make drawings on high ceilings that are not visible.

[0083] G: The minimap icon is an icon that allows participants to display a minimap on the screen. In this embodiment, the minimap displays the location of tour tags and other participants, including the user. Participants can move to the location of a tour tag by selecting that tour tag. The specific processing by processor 122 is the same as the processing of the C: tour tag list display icon described above.

[0084] The icons provided by the UI-14B operation icon display unit are: A: camera movement icon, B: slideshow icon, C: quiz enable / disable icon, and D: comment function icon.

[0085] A: The camera movement icon is used to change (or return to) the viewpoint of an object such as a participant within the virtual space.

[0086] In this embodiment, at least some of the tour tags automatically (forcibly) change the viewpoint of participants or others. For example, if the tour tag is related to letters carved into the floor, participants may lose track of where to look, so processor 122 directs the participant object's virtual camera downwards. In other words, processor 122 controls the virtual camera of the participant object at a specific location that has been set in advance.

[0087] Here, content is displayed either by the actions of the participants or automatically, but after the display is finished, the virtual camera's position returns to a position pre-set by the administrator after a pre-set time, when the participants press the camera movement icon, etc.

[0088] Furthermore, this event is managed by flags or similar mechanisms to prevent the same content from being displayed repeatedly. In other words, after viewing embedded content, processor 122 controls the system to prevent automatic camera movement or other actions from occurring.

[0089] In addition, the camera movement icon may also serve to reset the virtual camera's position when an abnormality in the viewpoint occurs due to incorrect operation by a participant.

[0090] I: The slideshow icon is activated in locations where multiple images are set as content, and it is an icon that allows multiple images to be displayed on the display area through a selection operation by participants, etc.

[0091] U: The quiz enable / disable icon is used to enable or disable quizzes. When enabled, participants can enjoy the quiz questions described below in locations where quizzes are set as content.

[0092] E: The comment function icon is an icon that allows participants to leave comments (text information) on virtual tourist facilities, etc. (These comments are referred to as "memory notes"). This is a function that allows participants to record their visits to virtual tourist facilities, for example, as a memory. Specifically, processor 122 acquires input from participants, etc., and associates this information with the location where the participant object is located. In this embodiment, only text information is retained, but it may also be possible to retain audio information, image information, etc.

[0093] The layout of elements displayed on the screens of participants, such as the main screen UI-11, participant display UI-12, guide display UI-13, and operation icon display UI-14, can be changed as appropriate (the same applies to the following screen elements).

[0094] Returning to Figure 2, the tour tag icon UI-15 is an icon that displays the location where content is embedded. In other words, the tour tag icon UI-15 indicates a location where text information, image information, or audio information that explains the location or items related to that location is set.

[0095] Tour tag icons UI-15 are typically set in multiple locations within the virtual space. Participants can access explanatory information by reaching these locations (making them visible on the screen) and performing the required actions (such as the actions described above).

[0096] In other words, when a predetermined operation (such as clicking or pointing) is performed on the tour tag icon UI-15, the processor 122 accepts the input and displays the content associated with that tour tag, such as text information, image information, or audio information, on the display of the participant's terminal.

[0097] More specifically, when a participant performs a predetermined operation (action) near the tour tag icon UI-15, the processor 122 displays the content on the content display unit UI-16, as described in the next section. Details of the process will be described later.

[0098] Figure 3 shows the content display screen. The processor 122 displays pre-configured explanatory information, such as text information, audio information, or image information (still images, moving images), on the content display unit UI-16. The content display unit UI-16 in Figure 3 displays a video with sound. When a participant clicks the play button (bottom right of the content display unit UI-16), the processor 122 plays the video. The bottom left of the content display unit UI-16 is the volume adjustment button. Note that the UI-14 operation icon display unit is omitted in Figure 3.

[0099] Figure 4 shows the screen in auto mode. Since the content has already been explained (such as screen elements like the participant display UI-12 and the content display screen), the drawing and explanation will be omitted to avoid repetition. In auto mode, the screen (video) displayed on the main screen UI-11 changes sequentially, and the guide provides commentary according to the location. (In some cases, participants can manipulate their viewpoint, while in other cases, the viewpoint is predetermined.)

[0100] In auto mode, the processor 122 displays the main screen UI-11 and the guide (video) display unit UI-17, and displays pre-set video footage for each tour and video footage of the guide on each. Note that the guide (video) display unit UI-17, which displays pre-stored video images via processor 122, is different from the guide display unit UI-13, which displays the guide in real time.

[0101] Although the participant display unit UI-12 is not shown in Figure 4, the processor 122 displays the participant display unit UI-12 when there are two or more participants.

[0102] Figure 5 shows the quiz display screen. Within the virtual space, quiz spots are provided either alongside the tour tag icon UI-15 or separately from the tour tag icon UI-15 (not shown). When the user performs a predetermined operation (action) near the quiz spot, the processor 122 displays the quiz screen UI-18 shown in Figure 5. Through the quizzes, participants can learn about the history and culture of tourist attractions and other related sites while having fun. Note that the operation icon display section UI-14 is omitted in Figure 5.

[0103] In this embodiment, rewards are awarded based on the number of correct answers (or correct answer rate) in the quiz. Rewards may include points that can be used in the area around the tourist destination or in other tourist content provision services (tours). In the online tour provision system 1, participants can purchase products or services specific to the tourist destination online. You can use the points mentioned above when making this online purchase. This means you can receive discounts, such as a reduction in price, based on the number of points you have. Alternatively, if a participant has accumulated points by taking multiple tours, they can use those points when booking a new tour.

[0104] With this configuration, participants and guides can join the tour from anywhere with a network connection. In other words, participants and guides can enjoy sightseeing no matter where they are or what the tourist destination is.

[0105] Participants using HMDs (Head-Mounted Displays) in particular can enjoy an immersive and realistic sightseeing experience through an online tour delivery system utilizing MR (Mixed Reality / Augmented Reality)1. Furthermore, since guides can perform their guiding duties from home, they do not have to bear the burden of traveling to tourist destinations, and can contribute to tourist guidance and, consequently, to the development of tourist destinations.

[0106] 2. Program Processing <Online tour provision processing> The program processing performed in the online tour provision system 1 of this embodiment will now be described. Note that "step" in the specification is represented as "S" in the diagram.

[0107] Figure 6 is a flowchart showing an overview of the online tour delivery process. First, we will use Figure 6 to explain the overall process, and then we will explain the individual processes. In Figure 6 and Figure 7 (described later), it is assumed that participants have applied for a sightseeing content provision service (tour) through a travel agency and have also requested the participation of a guide.

[0108] In Figure 6, the dotted arrows do not necessarily represent processes, but rather indicate contractual relationships between users, owners, or administrators (hereinafter referred to as "users, etc.") of each terminal.

[0109] A facility use agreement has been concluded between the service provider (user of server 10, etc.) related to the online tour provision system 1 and the rights holder of tourist facilities, etc. (user of rights holder terminal 50, etc.) (C1 in Figure 6). In other words, the rights holder grants permission to photograph tourist facilities, etc., with imaging devices, etc., to acquire photographic data, etc., to reproduce said tourist facilities, etc., in a virtual space, and to generate revenue from those tourist facilities, etc., in that virtual space.

[0110] Server 10 acquires a three-dimensional model constructed from image data of tourist facilities, etc., and stores the tourist content associated with the three-dimensional model (Step 1). In this embodiment, one piece of tourism content corresponds to one URL.

[0111] A platform usage agreement (and a fee collection agency agreement) has been concluded between the service provider (user of server 10, etc.) related to the online tour provision system 1 and the travel agency (user of travel agency terminal 40, etc.) (C2 in Figure 6).

[0112] In this embodiment, the travel agency introduces a service (tour). Participants (prospective participants) who view the service (e.g., online) apply to the travel agency (Step 2), and the travel agency terminal 40 notifies the server 10 of the application (Step 3).

[0113] The processor 122 of server 10 receives input from participants regarding whether or not they need a guide (step 4). If a guide is requested, it notifies the guide (guide terminal 30) (step 5). The participant responds (notifies) that they will participate (step 6).

[0114] Server 10 calculates the participation fee and notifies the participant (participant terminal 20) (step 7), and the participant makes the payment (step 8). Note that while Figure 6 depicts the transaction as a payment from participant terminal 20 to server 10 for simplicity, in reality, it is a transaction involving payment institutions (banks, credit card companies, etc.). In other words, it is the payment institution server that receives payment-related operations and sends payment notifications. The same applies to revenue sharing.

[0115] Server 10 grants access rights to participants (participant terminals 20) and guides (guide terminals 30) (step 9), and publishes the service (tour) in a form that can be accessed at a predetermined time (step 10). Server 10 then accepts participation (access) from participants (participant terminals 20) and guides (guide terminals 30) and provides services (steps 11 and 12).

[0116] Server 10 calculates revenue from participation fees and distributes the revenue according to predetermined fees and rates (Step 13). In the example shown in Figure 6, the revenue is distributed not only to the rights holder but also to the participating guides and the travel agency that facilitated the transaction.

[0117] Next, I will explain the individual processes.

[0118] In this embodiment, the processor 122 performs online tour provision processing based on the online tour provision program P1. The online tour provision program P1 includes at least a service provider setup program P11, a tourism content creation program P12, a tourism content provision program P13, and a quiz question generation program P14, and the processor 122 executes the service provider setup process, the tourism content creation process, the tourism content provision process, and the quiz question generation process, respectively, based on each of these programs. Furthermore, these programs, through the execution of their respective processes, cause the computer to function as a service provider configuration means, a tourism content creation means, a tourism content delivery means, and a quiz question generation means.

[0119] <2-1. Service Provider Configuration Process> In the service provider configuration process, processor 122 configures the provider of the service (the service that provides tourism content). In this embodiment, the processor 122 sets up a rights holder who has rights to tourist facilities and opens them to the public, a guide (and assistant guide) who provides tours of the tourist facilities, and a travel agency. In other words, the processor 122 stores information about the rights holder, the guide (including the assistant guide), and the travel agency in the storage unit 14.

[0120] The service provider setting process includes a rights holder setting process for setting rights holders who have rights to tourist destinations or tourist facilities ("tourist facilities, etc."), a guide setting process for setting guides for said tourist facilities, etc., and a travel agency setting process for setting travel agencies that introduce the services.

[0121] Processor 122 enables the distribution of profits to rights holders by designating (registering) rights holders who have rights to tourist facilities, etc. Similarly, processor 122 enables the distribution of profits to guides (and co-guides) by setting up (registering) guides (and co-guides) who will guide the participants. The information to be registered will be explained in the database section below.

[0122] Furthermore, processor 122 enables profit sharing with travel agencies by setting up (registering) travel agencies that introduce the service.

[0123] <2-2. Tourism Content Creation Process> In the tourism content creation process, processor 122 acquires a three-dimensional model based on tourist facilities in the real world and reproduces those tourist facilities in a virtual space. The processor 122 also associates information (content) such as text information, image information, or audio information with the three-dimensional model in the virtual space to create tourism content. Participants can enjoy (view, etc.) the services that provide the tourism content online.

[0124] In other words, the tourism content creation process involves creating tourism content by associating text information, image information, or audio information with a three-dimensional model of a tourist facility, etc., which is constructed from image data of the tourist facility, etc., taken by an imaging device.

[0125] In this context, the person who creates the tourism content is either the administrator or the guide. In other words, the guide may provide information on content that has been prepared in advance by the administrator or other relevant parties, or the guide may create and provide content themselves.

[0126] When a guide creates tourism content, they may either access server 10 to create the content or upload the data to server 10.

[0127] In other words, the above-described tourism content creation process can create tourism content by associating text information, image information, or audio information (content) acquired from a guide terminal with a three-dimensional model of the tourist facility, etc., constructed from image data of the tourist facility, etc., taken by an imaging device.

[0128] When a guide creates the content, they can offer unique content based on their own knowledge. In other words, even for the same tourist destination, different guides can offer different content, giving participants the advantage of being able to choose the content that best suits them. This is similar to how the emergence of video-sharing sites has led to the creation of numerous video channels, each with its own fanbase, allowing content creators to freely create content based on their own creativity. While current video sharing sites provide so-called two-dimensional content, online tour provision system 1 provides three-dimensional content.

[0129] Furthermore, in this embodiment, the administrator or guide may provide a portion of the above-mentioned tourist content to participants free of charge as introductory content. The introductory content is promotional material used to introduce the content's overview to participants. The introductory content is provided to participants free of charge.

[0130] In other words, processor 122 obtains introductory content that includes at least a portion of the tourism content and introduces the tourism content (referred to as "introductory content creation process"). The introductory content in this embodiment also associates text information, image information, or audio information (content) with a three-dimensional model of a tourist facility or the like. In this embodiment, the URL corresponding to the introductory content is different from the URL corresponding to the (paid) tourism content.

[0131] Next, we will explain the three-dimensional model. This section describes two types of three-dimensional models: the first three-dimensional model, constructed from (usually multiple) imaging data from an imaging device, and the second three-dimensional model, constructed from imaging data from an imaging device and distance data from a distance measuring sensor.

[0132] The first type of three-dimensional model is created from photographs or other sources, and therefore does not require distance data (or point cloud data) from a distance measuring sensor during the creation of the three-dimensional model. Furthermore, because it is based on imaging data, the first three-dimensional model contains information such as the color and patterns of objects like buildings.

[0133] The second three-dimensional model is constructed from imaging data obtained from an imaging device and distance data (point cloud data) obtained from a distance measuring sensor. A 3D model is constructed using point cloud data, and information from an imaging device is combined with it. As a result, the second 3D model contains information such as the color and pattern of objects like buildings.

[0134] The first three-dimensional model is created using techniques such as photogrammetry, NeRF (Neural Radiance Fields), or Gaussian splatting.

[0135] Photogrammetry is a method for obtaining the geometric properties of an object from photographic images, but here it specifically refers to the technique of generating a three-dimensional model of an object from multiple two-dimensional images of the object taken from various angles.

[0136] NeRF is a neural network that constructs a three-dimensional model from multiple two-dimensional images.

[0137] Gaussian Splatting acquires point cloud data using Structure from Motion (SfM) technology, which creates three-dimensional point cloud data from imaging data such as photographs and videos. This point cloud data is then replaced with a 3D Gaussian vector, which is then rasterized for comparison with the actual image. Finally, the 3D Gaussian vector is optimized by adjusting its parameters.

[0138] In this embodiment, in particular, a three-dimensional model is used, which is output by inputting imaging data from an imaging device into a "machine learning model for creating three-dimensional models".

[0139] A machine learning model for creating three-dimensional models is, for example, a machine learning model that learns using at least the following as training data: image data of an object (such as the ceiling, floor, or walls inside a building, or the building itself) taken by a model-creation imaging device, information about the length of the object, distance information from the model-creation imaging device to the object, and a three-dimensional model corresponding to the object, and takes the image data of the object as input to output a three-dimensional model of the object.

[0140] As such a three-dimensional model, for example, imaging data acquired by Matterport Inc.'s imaging device can be sent to Matterport Inc.'s (cloud) server, and the three-dimensional model obtained as the output of the machine learning model for creating three-dimensional models equipped on that server, or three-dimensional models owned by Matterport Inc., can be used (Matterport is a registered trademark).

[0141] The first type of three-dimensional model, namely three-dimensional models obtained using photogrammetry, NeRF, or Gaussian Splatting, or the three-dimensional models obtained using the machine learning models described above, does not require distance data (point cloud data) from a distance measuring sensor. However, the accuracy of the three-dimensional model may be improved by utilizing distance data (point cloud data) from a distance measuring sensor.

[0142] For example, with the machine learning model method described above, it is difficult to create a three-dimensional model from an image without feature points, but the accuracy of the three-dimensional model can be improved by adding point cloud data. An image without distinctive features would be, for example, an image of several doors of the same shape and color lined up. In this case, there is no difference between the image elements (doors), making it impossible to seamlessly connect multiple images.

[0143] Furthermore, the imaging data used for the first three-dimensional model may be moving images, that is, a series of images that are continuous in time. In other words, techniques for constructing a three-dimensional model from moving images may be used.

[0144] The second three-dimensional model is based on point cloud data acquired by sensors such as distance measuring sensors, or data obtained by converting this point cloud data into three dimensions, and imaging data from an imaging device. Examples of such three-dimensional data include mesh data, surface data, or surface data (geometry data). Because three-dimensional data stores coordinates in three-dimensional space, it is sometimes called coordinate data.

[0145] In this embodiment, the second three-dimensional model includes a three-dimensional model relating to space (room) (three-dimensional spatial model) and a three-dimensional model relating to an object (three-dimensional object model).

[0146] The following describes a method for acquiring a three-dimensional model (for recreating tourist facilities, etc., in a virtual space) using a device equipped with both an imaging device and a distance measuring sensor (such as an HMD). As mentioned above, the three-dimensional model is constructed from distance data (point cloud data) obtained from a distance measuring sensor and imaging data obtained from an imaging device such as a camera.

[0147] Examples of devices for acquiring such three-dimensional models include Meta Quest (META and META QUEST are registered trademarks).

[0148] Hereinafter, the device that acquires distance data (point cloud data) and imaging data will be referred to as the "data acquisition device". A data acquisition device is a type of computer terminal that includes not only input units such as distance measuring sensors and imaging devices, but also output units such as a control unit (processor), a storage unit, and a display unit.

[0149] The person who acquires distance data (point cloud data) and imaging data will be referred to as the "data acquirer" below. The data acquirer will acquire data on tourist facilities, etc., using data acquisition equipment.

[0150] To explain in more detail, the data collector moves around tourist facilities and other locations carrying (or wearing) a data acquisition device. For example, if a data acquirer points the data acquisition device north within a tourist facility, the device will acquire and store data from the north side of the facility. If the device is pointed south, it will acquire and store data from the south side of the facility. Similarly, if the device is pointed upwards, the processor will acquire data from the ceiling, and if it is pointed downwards, it will acquire data from the floor, and store these in the device's memory.

[0151] In this scenario, the range that the distance sensor can sense (for example, the range of infrared light from an infrared sensor) is limited by the presence of walls and other obstacles. Therefore, the data acquisition person moves around the space to acquire distance data (point cloud data) and image data for the entire tourist facility.

[0152] For example, in the case of an HMD (Head-Mounted Display), the HMD's display unit shows data elements (points, frames, meshes, or polygons, etc.) on the passthrough image, indicating where data has been successfully acquired. In other words, the data acquirer can confirm whether sufficient data has been acquired while acquiring the data.

[0153] Furthermore, if distance data (point cloud data) such as the boundary between a wall and the ceiling, the boundary between a wall and the floor, or the corner of a room is incorrectly recognized, the data acquirer can correct it while viewing the passthrough video. A detailed explanation is omitted.

[0154] At this stage, only spatial data (walls, ceiling, floor) has been acquired. Depending on the data acquisition device, it may also be possible to acquire data on objects in space and obtain independent three-dimensional object models.

[0155] Here, we will explain how to obtain a three-dimensional object model. For example, suppose there is only one statue (roughly rectangular) as an item in a room (rectangular) along the wall. The data acquirer can then use the controller (attached to the data acquisition device) to create a rectangular prism that includes the statue and set it as an item.

[0156] The data acquirer can determine the name of the item after the above settings are made, and the data acquisition device will acquire that information. The processor of the data acquisition device stores the various types of data being acquired (distance data (point cloud data), imaging data, etc.) in the storage unit.

[0157] Alternatively, the processor of the data acquisition device may create a three-dimensional model from this data, or a computer other than the data acquisition device may construct a three-dimensional model from this data.

[0158] In this embodiment, various types of data acquired by the data acquisition device (distance data (point cloud data), imaging data, and three-dimensional (object) models, etc.) are acquired by a three-dimensional model creation server (a server equipped with a machine learning model for three-dimensional model creation) separate from server 10, and stored in the storage unit of that server. Server 10 retrieves the three-dimensional model being built by this server dedicated to building three-dimensional models.

[0159] Return to the explanation of the tourism content creation process. The processor 122 creates tourism content by associating text information, image information, or audio information with a three-dimensional model of a tourist facility or the like.

[0160] Specifically, this process associates corresponding information (text information, image information, audio information, or a combination of these, such as video with audio) with a specific location (such as a tour tag) on ​​a three-dimensional model represented by coordinates in a virtual space.

[0161] As explained in the UI section, when a participant in the virtual space interacts with or selects the information (content), the processor 122 provides that information to the participant's terminal.

[0162] More specifically, when a participant object (avatar) in the virtual space approaches a point where the information is set (when it enters a certain distance), and the participant performs a predetermined operation (click, point, etc.) at that point, the processor 122 provides (sends) the set information to the participant's terminal. The predetermined operation in this embodiment is an action operation as described in the UI section.

[0163] The service provider configuration process and tourism content creation process described above are preparatory processes for providing tourism content services (tours).

[0164] <2-3. Processing of Tourism Content Provision> In the tourism content provision process, processor 122 provides the tourism content created in the tourism content creation process described above to participants and others.

[0165] The processing of providing tourism content is The process of providing introductory content (to participants). The process of receiving applications from participants for services (tours) that provide tourism content. (After obtaining the participant's application) A guide requirement determination process that determines whether a guide is necessary. Participation fee calculation process to calculate the participation fee for each participant. A fee adjustment process to change the participation fee if a participant requests a guide but is unable to provide one. The payment confirmation process for the aforementioned participation fee, An access permission granting process that grants access rights to the aforementioned tourist content to the aforementioned participants, A service provision process that accepts access to the tourism content by the aforementioned participants and provides the service (a service that provides tourism content (tour)), A participant viewpoint provision process that places a participant object corresponding to the participant in the virtual space and displays the video from the virtual camera of the participant object on the display unit of the participant terminal. A guide object corresponding to the guide is placed in the virtual space, and the video from the virtual camera of the guide object is displayed on the display unit of the guide terminal as part of the guide viewpoint provision process. A three-dimensional image change process is performed to change the image displayed on the participant's terminal by the participant's operation of the participant's terminal or by the guide's operation of the guide's terminal. It includes a distribution calculation process that calculates at least the distribution to rights holders from the participation fee (and the revenue included in it). Furthermore, the access permission granting unit, If a guide is required, grant the aforementioned access rights to the participants and the guide. The aforementioned access rights will be granted to participants (only) if a guide is not required.

[0166] Figure 7 is a flowchart showing the process of providing tourism content. Please note that this flowchart is for illustrative purposes only, and the processes related to application for participation, payment acceptance, service provision, and revenue sharing may be handled separately. Furthermore, although the flowchart below focuses on one participant, server 10 processes participation applications from multiple participants simultaneously.

[0167] Processor 122 begins processing the provision of tourism content upon receiving notification of participation applications from participants (prospective participants).

[0168] Processor 122 receives participant applications (participant application acceptance processing) and obtains participant information (step 21).

[0169] Although not shown in Figure 7, in this embodiment, the processor 122 provides (free) introductory content to participants (prospective participants) (referred to as "introductory content provision processing").

[0170] In other words, processor 122 accepts access from participants (to the URL corresponding to the introductory content) and provides the introductory content.

[0171] For example, after a guide or other person creates content, they can post the URL of the content on their own media, such as social media, to promote the tourist content. If participants like the content, they can proceed to register to participate from the featured content.

[0172] Then, the processor 122 determines from the information entered by the participant whether or not the participant requires a guide (whether a guide is needed ("needed" or "not needed")) (Step 22: Guide Needs Determination Process).

[0173] The processor 122 may store this necessity determination (a value corresponding to whether it is necessary or unnecessary) in the memory unit 14 and use this information for various decisions described later, such as cost calculation and granting access rights.

[0174] If a participant requires a guide (guide required), the processor 122 retrieves information about the guide registered with the tourism content provision service (tour) from the storage unit 14 (step 23). The processor 122 then selects an appropriate guide and sends a notification to that guide (step 24).

[0175] In this embodiment, the processor 122 sends a notification to all registered guides and, from among the guides who responded within a predetermined period, determines the guide using a predetermined algorithm. The specified algorithm is one that determines the guide based on factors such as the speed of response, the level of participant evaluation (review), the rate of response to notifications, or a combination of these factors.

[0176] Note that this is not the only way to select a guide. For example, you could assign registered guides in order and send notifications only to the guide assigned to the task.

[0177] Furthermore, if the guide themselves creates the tourism content, that creator becomes the guide. However, in this case, the guide who created the tourism content may recommend or select other guides.

[0178] Processor 122 determines whether the necessary guide has been determined (step 25), and if the participants have been determined (step 25 - guide determined), proceeds to the next step. If a participant does not require a guide (Step 22: No guide needed), proceed to the next step.

[0179] On the other hand, if a guide is not determined (Step 25: Guide not determined), for example, if there is no response from the guide, the processor 122 repeats the process in Step 23.

[0180] Next, processor 122 calculates and sets the final participation fee (step 26: participation fee calculation process). As mentioned above, the participation fee varies depending on whether there is a guide, whether there are third-party participants, and whether it is an individual or group application. Therefore, processor 122 retrieves the participant's application details and calculates the participation fee.

[0181] Whether or not a guide is required, the individual costs that make up the participation fee are stored in the storage unit 124 in advance (set by the administrator, etc.).

[0182] Furthermore, if circumstances arise such as the guide being unable to participate, processor 122 will (downward) adjust the participation fee. In other words, in step 26, the processor 122 specifically changes the participation fee (participation fee adjustment process) if a participant has requested a guide but no guide is available.

[0183] Processor 122 notifies participants of the final participation fee and accepts payment notifications (Step 27: Participation fee payment confirmation process). Following the flowchart, Processor 122 waits until it receives a deposit notification (Step 27: No deposit), and if it does receive a deposit notification (Step 27: Deposit received), it proceeds to the next step. However, this flowchart is for illustrative purposes only, and the processing related to payments may be handled separately.

[0184] Processor 122 grants participants and guides access rights to the tourism content (services) (processor 28). As described above, the process of granting participants access rights to the tourism content is called the "access rights granting process".

[0185] Here, in the access permission granting process, processor 122, If a guide is required (Step 22: Guide Required), grant the aforementioned access rights to the participants and the guide. If a guide is not required (Step 22: No guide needed), grant the aforementioned access rights to the participants (only).

[0186] If participants require a guide, the guide must also be granted access to the tourist content and participate accordingly. However, if participants do not require a guide, there is no need to grant the guide access.

[0187] Processor 122 also makes the tour public at a predetermined start time or a few minutes before (i.e., makes it possible for participants to access the sightseeing content) and provides the service (step 29). As mentioned above, the process of accepting access requests from participants to tourism content and providing the service is referred to as the "service publication and provision process."

[0188] When providing the service, processor 122 grants participants the right to operate the participant object, and guides the right to operate the guide object.

[0189] Furthermore, processor 122 causes the participant object's virtual camera image to be displayed on the participant terminal's display unit, and the guide object's virtual camera image to be displayed on the guide terminal's display unit (participant viewpoint provision processing / guide viewpoint provision processing).

[0190] Participants control their avatars while viewing video feeds from objects corresponding to themselves. In other words, the processor 122 changes the image displayed on the terminal's display unit in response to the terminal's operation by the participant (three-dimensional image change processing).

[0191] Furthermore, in certain cases, such as the guided mode described in the UI section, the processor 122 temporarily suspends the participants' control privileges, allowing only the guide to operate the system.

[0192] In this embodiment, the guide's viewpoint is shared with the participants. That is, the processor 122 displays the video from the guide object's viewpoint on the participant's terminal.

[0193] In other words, the guide operates their terminal to change the image displayed on the participant's terminal (three-dimensional image change processing).

[0194] After the service provision time ends, processor 122 terminates the operational privileges of participants and disconnects access to participant terminals and guide terminals.

[0195] After the service ends, processor 122 distributes the revenue. In other words, processor 122 calculates the distribution from the participation fee (and the revenue included in it) to the service provider, or at least the rights holder (distribution calculation process).

[0196] In this embodiment, since a travel agency is also involved, the processor 122 calculates at least the allocation to the rights holder and the allocation to the travel agency from the participation fee (and the revenue included in it).

[0197] In this embodiment, since the rights holder, guide, and travel agency are involved, the processor 12 calculates the revenue distribution to these parties. For example, processor 122 calculates, according to a predetermined formula, that 40% of the revenue (after deducting expenses from participation fees) will be distributed to the rights holder, 30% to the guide, 20% to the travel agency, and the remaining 10% to the administrator.

[0198] As another example, if a travel agency is not involved, processor 122 calculates that 50% will be allocated to the rights holder, 40% to the guide, and the remaining 10% to the administrator. As another example, if neither a guide nor a travel agency is involved, processor 122 calculates that 80% will be allocated to the rights holder and the remaining 20% ​​to the administrator. These are merely examples, and the allocation ratios can be changed as needed.

[0199] Further details will be explained below. In the application processing for participation, if the participants are a "group" as described above, processor 122 accepts applications from multiple participants through a single application by one participant (referred to as the "representative") (step 21). In this case, for a single application, processor 122 grants multiple access permissions according to the number of participants (step 28). Also, there is only one URL (the URL that each participant accesses) in this case. At this time, processor 122 will standardize the group's decision regarding whether a guide is necessary and whether or not a third party will participate. The processor 122 then creates an avatar (participant object) for each participant, and participants can control their avatars in the virtual space by inputting commands from their participant terminals.

[0200] In other words, processor 122 accepts the participation of multiple participants with a single application (bulk application processing). The processor 122 also grants each of the aforementioned participants access rights to the tourism content (referred to as the "bulk access rights granting process").

[0201] Furthermore, processor 122 changes the content of the services it provides (step 29) depending on the content of the application for participation.

[0202] In other words, the processor 122 accepts the participant's selection of either manual mode (referred to as the "first action mode") or auto mode (referred to as the "second action mode") (referred to as the "action mode acceptance process") (in this embodiment, at the stage of application for participation (step 21)).

[0203] Furthermore, when providing a service (a service that provides tourism content (tours)) (step 29), In the aforementioned manual mode (first action mode), the participant object is controlled according to the participant's input (e.g., movement, viewpoint change, or content delivery), In the case of the aforementioned auto mode (second behavior mode), regardless of the participant's input, the participant object is controlled in a manner predetermined by the administrator or the aforementioned guide, or by an operation performed by the administrator or the aforementioned guide (referred to as "behavior mode-specific service switching process").

[0204] Similarly, the processor 122 accepts the participant's selection of whether or not there is a third party ("third party presence / absence acceptance processing") (in this embodiment, at the stage of application for participation (step 21)).

[0205] At this time, the processor 122 (in this embodiment, at the application stage (step 21)) proposes a service (a service or tour providing tourism content) according to the participant's selection (referred to as "third-party selection proposal processing"). In other words, if a participant does not wish for a third party to participate, a tourism content provision service (tour) without a third party will be displayed as a suggestion. If a participant wishes to bring a third party, a tourism content provision service (tour) that includes third-party participation will be displayed as a suggestion.

[0206] Furthermore, there may be an option for third-party participation to be "either way or no," and if a participant selects this option, both a tour without a third party and a tour with a third party should be offered.

[0207] In addition, processor 122 controls the aforementioned "leading mode" when providing services. In other words, the processor 122, upon a predetermined operation by the guide (such as pressing the guide mode icon (not shown)), (temporarily) suspends the participant's authority to operate the participant object and grants that authority to the guide (referred to as the "authorization transfer process").

[0208] At this time, the processor 122 may individually grant the guide the control rights of one or more participants selected by the guide, or it may grant the guide the control rights of all participants at once. Furthermore, the participant object may be integrated with the guide object, or it may be kept separate from the guide object. In this embodiment, all participants' participant objects become one with the guide object. In other words, the position, viewpoint, etc., of the guide object are shared with all participants.

[0209] In addition, if there are objects such as walls, floors, or ceilings between two or more different participant objects or guide objects in the virtual space, the processor 122 reduces the exchange of audio information between those two or more objects ("audio obstacle processing"). When there are two participants, A and B, whether participant A can hear the audio from participant B's audio input (the output from participant A's audio output) depends on the distance between them. In other words, when the distance between the objects is short, one person can clearly hear the other's audio. Audio obstacle processing, on the other hand, is a process that, if there is an object such as a wall between participant A and participant B, makes the audio from participant B's audio input either inaudible or very quiet to participant A (by suppressing the output from participant A's audio output).

[0210] <2-4. Processing Quiz Questions> In the quiz question presentation process, processor 122 presents quiz questions to participants in a virtual space.

[0211] The quiz question generation process is a quiz setting process that accepts quiz settings related to tourist facilities, etc. A quiz presentation process that presents the aforementioned quiz to the participants, A quiz answer acquisition process that receives answers to the aforementioned quiz from participants. A quiz question correctness determination process that determines whether the aforementioned answer is correct or incorrect, and This includes a quiz result display process that displays the result of determining whether the result is correct or incorrect.

[0212] The quiz questions are associated with parts of the tourism content, such as tour tags. In this embodiment, if a participant has turned on the quiz enable / disable icon, and the participant's avatar is in a location where a quiz question is set, and the participant performs a predetermined operation, the processor 122 presents the participant with a quiz (see the quiz display screen in the UI section). The quiz questions are obtained from the quiz data table D18.

[0213] Processor 122 presents a quiz to the participants and waits for a certain period of time to accept their answers. Once it receives a participant's answer, it determines whether the answer is correct or incorrect by referring to the quiz data table D18, and displays the result.

[0214] Furthermore, processor 122 stores the user's score for each participant (the number of correct answers in the quiz, or the points assigned to each quiz (for example, points corresponding to the difficulty level of the quiz)). Processor 122 can provide services (such as discount tickets) based on the participants' scores.

[0215] With the configuration described above, the online tour provision system 1 of this embodiment holds and stores guide information. It then retrieves the user's application details, determines whether a guide is needed or not, selects an appropriate guide from the guides stored in the memory if necessary, and retrieves information such as the guide's availability to participate. It also determines the participation fee based on whether a guide is needed and distributes the revenue to those involved in providing the service. This ensures that a guide is available for each tour, and since the guide can participate online, they can work as a guide even from remote locations. Furthermore, rights holders of tourist facilities, guides, and travel agencies can securely and stably earn revenue online.

[0216] 3. Data Hereinafter, data handled by the online tour providing system 1 of the present embodiment will be described with reference to tables. The online tour providing system 1 of the present embodiment includes an online tour providing database D1 in the storage unit 14 (data storage unit 14b) of the server 10. The online tour providing database D1 includes a right holder data table D11, a tourist facility etc. data table D12, a guide data table D13, a participant data table D14, a travel agency data table D15, a tour data table D16, a tour tag data table D17, and a quiz data table D18.

[0217] Specific examples are given below, but the configuration of the data table and the data items included in the data table are merely examples and are not limited to the following examples.

[0218] The right holder data table D11 is a data table including data relating to right holders (right holder data).

[0219] [Table 1]

[0220] Table 1 exemplifies data included in the right holder data table D11. As shown in Table 1, the right holder data table D11 includes, as items, a unique right holder ID, a right holder name corresponding one-to-one to the right holder ID, a name of a tourist facility or the like for which the right holder holds rights, an address of the right holder, contact information of the right holder (e-mail address, telephone number, etc.), a transfer destination for the right holder's proceeds (bank name, branch name, account type, account number, etc.), and a payment date. Since examples of contact information and transfer destinations are the same, descriptions thereof are omitted below.

[0221] The tourist facility etc. data table D12 is a data table including data relating to tourist facilities and the like (tourist facility etc. data).

[0222] [Table 2]

[0223] Table 2 provides an example of the data contained in the tourist facilities data table D12. As shown in Table 2, the tourist facility data table D12 includes the following items: a unique tourist facility ID, the name of the tourist facility corresponding one-to-one with the tourist facility ID, the rights holder ID of the rights holder relating to the tourist facility, and the address of the tourist facility.

[0224] Guide data table D13 is a data table that contains data about guides (guide data).

[0225] [Table 3]

[0226] Table 3 provides an example of the data contained in the guide data table D13. As shown in Table 3, the guide data table D13 includes the following items: a unique guide ID, a guide name corresponding one-to-one with the guide ID, foreign languages ​​the guide can use, the names of tourist facilities etc. that the guide can guide (registered), the guide's address or residence, the guide's contact information, the guide's bank account details, and the date of payment to the guide. In Table 3, names or company names are represented by letters A through G.

[0227] The section on supported foreign languages ​​may also include the languages ​​available; that is, since the guide may not be a local, it is acceptable to include Japanese and other languages ​​as information. Furthermore, it may include linguistic characteristics such as dialects.

[0228] Participant data table D14 is a data table that contains data about participants (participant data).

[0229] [Table 4]

[0230] Table 4 exemplifies the data included in the participant data table D14. As shown in Table 4, the participant data table D14 includes, as items, a unique participant ID, a participant name that corresponds one-to-one to the participant ID, personal information other than the participant name (age (or age group), gender, address, contact information, etc.), points, participation type (individual or group), and information on tours the participant has joined in the past.

[0231] The points are points awarded by the online tour providing system 1. There may be a plurality of items related to points, and for example, points (scores) obtained from quizzes, points having monetary value, and the like may be included.

[0232] In addition to the above, the participant data table D14 may further include data related to participant behaviors, such as purchase data (at shops in a processing space, etc.), data on options added by the participant to the tour, data related to movement routes for each tour, data related to comments left on tour tags and the like (text data), image and audio data (or IDs associated therewith), or evaluations and comments for a certain tour, and the like. By acquiring these data, the processor 122 can analyze the participant's preferences and provide services that the participant is likely to prefer.

[0233] The travel agency data table D15 is a data table that includes data related to travel agencies (travel agency data).

[0234] [Table 5]

[0235] Table 5 exemplifies the data included in the travel agency data table D15. As shown in Table 5, the rights holder data table D15 includes the following items: a unique travel agency ID, the name of the travel agency corresponding one-to-one with the travel agency ID, the name of the tourist facility etc. for which the travel agency has a tour (registered), the travel agency's address, the travel agency's contact information, the travel agency's bank account details, and the payment date.

[0236] Tour data table D16 is a data table that contains data (tour data) related to (individual) tours.

[0237] [Table 6]

[0238] Table 6 provides an example of the data contained in tour data table D16. As shown in Table 6, the tour data table D16 includes the following items: a unique tour ID, the tour destination corresponding one-to-one with the tour ID, the ID of the tourist facilities etc. visited by the tour, the date and time the tour starts, the date and time the tour ends, the participation fee, the date and time recruitment begins, the date and time recruitment ends, the number of participants, the capacity (maximum number of participants), the minimum number of participants required, the status, and ancillary services.

[0239] The participation fee is the total cost paid by the participants, but the tour data table D16 may also include separate costs that make up the participation fee, such as costs with a guide, costs without a guide, per person costs when the participant is an individual, per person costs when the participant is a group, costs when a third party participates, or costs when a third party does not participate.

[0240] Tour destinations and tourist facility IDs do not necessarily have a one-to-one correspondence; a single tour may include multiple tourist facilities.

[0241] The status indicates the tour's "pre-registration," "registration in progress," "ongoing," and "completed" states. Processor 122 automatically updates the description based on the fire date and time, the registration date and time, and the current time obtained from an NTP server or similar source.

[0242] The "Additional Services" section lists the services and other related services included with the tour.

[0243] In addition to the above, tour data table D16 may also contain data on participants and guides, such as participant ID, guide ID, participant payment information (whether or not the participation fee has been paid), and actual participation / non-participation. Furthermore, the tour data table D16 may also contain data items related to tours other than those mentioned above, such as the tour creator, the date and time the tour was created, and the tour's expiration date (end date). Additionally, the tour data table D16 may also contain data items related to the type of tour (for example, the type, such as whether or not there is a guide as mentioned above).

[0244] Tour tag data table D17 is a data table that contains data related to tour tags (tour tag data).

[0245] [Table 7]

[0246] Table 7 provides an example of the data contained in the tour tag data table D17. As shown in Table 7, the tour tag data table D17 includes the following items: a unique location ID (tour tag ID), a location name (tour tag name) that corresponds one-to-one with the location ID, a tourist facility ID and its name, and the ID, type, and file name of the content associated with the location ID. In addition, the tour tag data table D17 may also contain coordinate data in a virtual space.

[0247] Regarding the content, a separate database may be maintained, such as a content data table with unique content IDs.

[0248] Quiz data table D18 is a data table that contains data related to quiz questions.

[0249] [Table 8]

[0250] Table 8 provides an example of the data contained in quiz data table D18. As shown in Table 8, the quiz data table D18 includes the following items: a unique quiz ID, a location ID for the location where the quiz is set, a quiz question and answer choices that correspond one-to-one with the quiz ID, the correct answer, and the difficulty level of the quiz question.

[0251] In addition to the above, the online tour database D1 contains data on coordinates in virtual space. For example, it is a data table that stores coordinates in a virtual space for a certain tourist content, along with the aforementioned "memory notes," that is, textual information, etc.

[0252] Furthermore, data related to these coordinates may include the virtual space movement paths of participants and guides. In other words, the processor 122 may store the paths taken by the avatars of participants and others, and make it possible to reproduce the movement paths with lines or other visual representations.

[0253] When participants are freely moving around in a virtual space, many participants are likely to visit popular spots. Therefore, by acquiring this kind of information, it is possible to accumulate information on popular spots and other relevant details for a given tour. For popular spots, content can be enhanced, while for unpopular spots, content and tour tags can be removed for improvement.

[0254] 4. Hardware Configuration As shown in Figure 1, the online tour provision system 1 in this embodiment comprises a server 10, a participant terminal 20, a guide terminal 30, a travel agency terminal 40, and a rights holder terminal 50. Furthermore, each of these devices is connected via a network N, which is, for example, the internet. Server 10 has software (application software) installed that includes the online tour provision program P1 and is necessary to operate the online tour provision system 1 according to this embodiment, and various processes are executed by the functions of this software. The following describes each piece of hardware.

[0255] <Server 10> Server 10 is a computer (information processing device) for running the online tour provision program P1. Although only one server 10 is shown in Figure 1, the number is not limited to one and may be implemented using multiple servers. For example, from the perspective of load balancing and availability, it might be possible to use multiple servers. Server 10 may utilize a computer provided by a cloud service provider, or the user may provide their own computer. In this embodiment in particular, the server that generates the three-dimensional model and the server that provides the tourism content are separate.

[0256] Figure 8 is a hardware configuration diagram of server 10. As shown in Figure 8, the server 10 comprises a control unit 12, a storage unit 14, and a communication control unit 16. The control unit 12 also comprises a processor 122, a ROM 124, a RAM 126, and a timing unit 128. The basic functions of each will be explained in detail later.

[0257] The control unit 12, which includes a processor 122, also functions as an online tour provision unit in the server 10 (not shown). The online tour provision unit executes the online tour provision program P1 to perform online tour provision processing. In this embodiment, the processor 122 is a CPU (Central Processing Unit).

[0258] Furthermore, one program may include other programs. For example, in this embodiment, the online tour provision program P1 includes a service provider setting program P11, a tourism content creation program P12, and a tourism content provision program P13, etc.

[0259] As shown in Figure 8, the storage unit 14 includes a program storage unit 14a and a data storage unit 14b, and stores programs and data necessary for various processes. For example, the program storage unit 14a stores not only the online tour provision program P1 according to this embodiment, but also control programs for controlling devices connected to the server 10, such as a communication control program for controlling the communication control unit 16.

[0260] As shown in Figure 8, the communication control unit 16 is a device that communicates between the server 10 and an external terminal, such as a terminal described later (e.g., participant terminal 20). As shown in Figure 1, the communication control unit 16 connects the server 10 to the network N.

[0261] In addition to the above, the server 10 may also include an input unit (e.g., a keyboard) for inputting commands and data, and an output unit (e.g., an audio output device) for outputting information in some form (not shown). Furthermore, it may include additional devices necessary for the use of this embodiment, or devices to improve convenience for the use of this embodiment.

[0262] <Data acquisition device> Now, let's discuss data acquisition devices. We'll use an HMD (Head-Mounted Display) as an example. The data acquisition device, like a typical computer, comprises a control unit including a processor, a storage unit, a communication control unit, an input unit, and an output unit.

[0263] The communication control unit is a device that communicates between the data acquisition device and an external terminal. Furthermore, the communication control unit enables wireless communication between the data acquisition device and the controller corresponding to the data acquisition device. For example, the data acquisition device and the controller communicate using Bluetooth®. In addition, the communication control unit is also used when the data acquisition device communicates with a network such as the Internet.

[0264] The input section of the data acquisition device includes a sensor and an imaging device. The data acquisition device is equipped with a distance measuring sensor. The distance measuring sensor is used to acquire point cloud data for recreating the interior of a building, etc. In this embodiment, the distance measuring sensor is an infrared depth sensor. However, the range-measuring sensor is not limited to this; it may also be a millimeter-wave radar or LiDAR (Light Detection and Ranging).

[0265] The data acquisition device here is equipped with a device for detecting position, orientation, and tilt. Examples of such devices include inertial measurement units (IMUs) using acceleration sensors, gyroscopes, and geomagnetic sensors. However, the devices are not limited to these, and may also include detection devices using infrared LEDs and infrared cameras, visible light LEDs and visible light cameras, or infrared laser emitters. A device that performs such a function may belong to either the input or output section, depending on its operation.

[0266] In addition to the above, the data acquisition device may also be equipped with environmental sensors such as a temperature sensor to measure ambient temperature and a humidity sensor to measure humidity.

[0267] The imaging device is a camera used to acquire images of the area surrounding the data acquisition device. The imaging device captures point cloud data obtained from the distance measuring sensor, as well as images to be superimposed on a 3D model of the point cloud data. Furthermore, the imaging device implements a hand tracking function. That is, the imaging device recognizes the user's hand, and the processor 111 reflects its position on the avatar's hand in the MR space. In addition, the imaging device may be used to track the HMD (inside-out method).

[0268] In addition to the above, HMDs are equipped with sensors and imaging devices that are also found in known head-mounted displays, such as a camera for capturing eye movements (near-infrared camera) to realize eye tracking functionality, and sensors for capturing head and neck movements (head tracking functionality).

[0269] The output section of the data acquisition device includes a display unit and a speaker (audio output device). The display unit is, for example, the display on an HMD (Head-Mounted Display). The display unit may, for example, show video to confirm whether data has been acquired correctly during data acquisition. In addition, it may display pass-through video, pointers, and information related to settings.

[0270] In addition to the above, the data acquisition device may include an audio input unit such as a microphone, an input unit for inputting commands and data (e.g., buttons), and an output unit for outputting information in some form (e.g., a vibration device). It may also include an input / output interface for connecting various devices. Furthermore, it may include additional devices necessary for the use of this embodiment, or devices to improve convenience for the use of this embodiment.

[0271] <Terminal (Participant Terminal 20)> The participant terminal 20 is a computer used by participants to access the online tour provision system 1. Participants use the online tour provision system 1 by accessing the server 10 using the participant terminal 20. Figure 9 is a hardware configuration diagram of the terminal (participant terminal 20). As shown in Figure 9, the participant terminal 20 comprises a control unit 22, a storage unit 24, a communication control unit 26, and an input / output unit 28. The control unit 22 also comprises a processor 222, ROM, RAM, and a timing unit. Explanations that overlap with previously explained content or relate to basic functions described later will be omitted.

[0272] In this embodiment, the participant terminal 20 is a desktop PC or HMD. However, the participant terminal 20 is not limited to these and may be a portable device such as a smartphone or tablet. Furthermore, devices worn by the user, such as HMDs, are referred to as "wearable devices."

[0273] Although simplified in Figure 9, if the participant terminal 20 is an HMD, the participant terminal 20 comprises a head-mounted device 20a and a controller 20b, which each comprises a control unit, a memory unit, a communication control unit, an input unit, and an output unit, respectively. The head-mounted device 20a and the controller 20b communicate with each other through the functions of their respective communication control units.

[0274] Furthermore, the input section of the head-mounted device 20a is equipped with sensors (such as a distance sensor, acceleration sensor, or gyro sensor) and an imaging device, while the output section is equipped with a display unit (usually placed in front of the wearer's eyes) and a speaker. The input section of the controller 20b is equipped with sensors (such as an accelerometer or gyroscope) and buttons (for operation input), and the output section is equipped with a vibration device (which is linked to the displayed image, etc.). Commercially available HMDs can be used as appropriate for this purpose.

[0275] The processor 122 determines whether the display unit of the terminal (participant terminal 20 or guide terminal 30) is a single display unit like a PC monitor, or a display unit that displays different information for the wearer's right and left eyes, such as an HMD (referred to as the "display unit determination process"). If the terminal (participant terminal 20 or guide terminal 30) is a wearable device such as an HMD that is capable of displaying different information for the right and left eyes, the processor 122 provides the display for the right eye and the display for the left eye (of the wearer (participant or guide)) to the display unit of the terminal (referred to as "content provision processing for wearable terminal"). HMDs and similar devices provide a high level of immersion by artificially creating binocular parallax by displaying different images to the wearer's right and left eyes.

[0276] The processor 122 also acquires input information from the controller 20b, which functions as an input device for a wearable terminal (operated by a participant, etc.), and changes an object (participant object or guide object) within the tourism content (referred to as "object change processing").

[0277] In this embodiment, the participant terminal 20 does not need to install any specific application program; participants can perform various processes by accessing a website provided by the administrator, meaning they can utilize the online tour provision system 1.

[0278] Various processes include output (screen display, audio output) based on information obtained from server 10, receiving user input, and various communications.

[0279] The guide terminal 30, the travel agency terminal 40, and the rights holder terminal 50 have the same hardware configuration as the participant terminal 20, so their descriptions will be omitted. The guide terminal 30 is a desktop PC, HMD, or portable terminal, while the travel agency terminal 40 and the rights holder terminal 50 are desktop PCs or portable terminals. In other words, the participant terminal 20 and guide terminal 30 that access the virtual space of the service providing tourism content may use HMDs or the like to provide a sense of immersion and presence.

[0280] (Explanation of the basic functions of a computer) The following describes the control unit (processor, ROM, RAM, timing unit), storage unit, communication control unit, input unit, and output unit. In any of the terminals in this embodiment, the connection configuration (network topology) between functional units is not particularly limited. For example, it may be a bus type, a star type, a mesh type, or the like.

[0281] The processor performs information processing and controls various devices according to programs stored in ROM or other memory units. In this embodiment, the processor is a CPU (Central Processing Unit).

[0282] However, the processor is not limited to a CPU. Various processors such as CPUs, DSPs (Digital Signal Units), GPUs (Graphics Processing Units), GPGPUs (General Purpose computing on GPUs), TPUs (Tensor Processing Units), or ASICs (Application Specific Integrated Circuits) may be used individually or in combination. For example, a processor that integrates a CPU and a GPU is called an APU (Accelerated Processing Unit), and such a processor can also be used.

[0283] ROM is read-only memory that contains various programs and data pre-stored for the processor to perform various control and calculations.

[0284] RAM is random-access memory used by the processor as working memory. Various areas can be reserved within this RAM for performing the various processes described in this embodiment.

[0285] The timing unit performs timing processing, including the acquisition of time information. If the computer is equipped with a communication control unit, it may acquire time information from an external source using NTP (Network Time Protocol).

[0286] A memory unit is a device for storing information such as programs and data. It is also called a storage unit. The memory unit can be either internal or external.

[0287] The storage unit includes a storage medium capable of reading and writing data, and a drive for reading and writing to the storage medium. Storage media include internal and external types, such as HD (hard disk), CD-ROM, and flash memory. Examples of drives include HDDs (hard disk drives) and SSDs (solid state drives).

[0288] The memory unit comprises a program storage unit and a data storage unit as functional units. The program storage unit stores control programs for controlling various devices, such as communication control programs for controlling communications.

[0289] The communication control unit is a device for facilitating communication between terminals and other devices. The communication control unit connects the terminals equipped with the communication control unit to the network N.

[0290] The communication method of the communication control unit is a known method, and either a wired or wireless method is applied depending on the equipment. For example, if the device is a desktop PC, both wired and wireless connections are possible, while if the device is a smartphone, a wireless communication method is possible.

[0291] For wired connections, communication methods defined in standards such as IEEE 802.3 (e.g., bus-type or star-type wired LANs) can be suitably used, but other communication methods such as those defined in IEEE 802.5 (e.g., ring-type wired LANs) may also be used.

[0292] For wireless communication, a communication method defined by, for example, IEEE 802.11 (e.g., Wi-Fi) can be suitably used. However, other methods such as IEEE 802.15 (e.g., Bluetooth®, BLE (Bluetooth® Low Energy)), IEEE 802.16 (e.g., WiMAX), ZigBee®, 920MHz band wireless (e.g., Wi-SUN), or optical communication methods such as infrared communication may also be used.

[0293] The input and output sections are devices responsible for input and output to and from the terminal, respectively. The input and output sections are sometimes collectively referred to as the input / output section. The input unit is a device that receives input from the user. Examples of such input units include keyboards, pointing devices such as mice, trackpads, tablets, or touch panels.

[0294] If the device is a tablet or smartphone and the input unit is a touch panel, the input unit is located on the surface of the display unit that displays images, such as a touchscreen. In this case, the input unit identifies the user's touch position corresponding to the various operation icons displayed on the display unit and accepts input from the user.

[0295] The output unit is a device for outputting, for example, images, audio, or documents. Output devices include, for example, display devices such as touchscreens and displays (liquid crystal displays and organic EL displays), audio output devices such as speakers, and document output devices such as printers.

[0296] (modified version) The present invention is not limited to the embodiments described above, but includes various modifications to the embodiments described above, without departing from the spirit of the invention. For example, in the "guided mode" of the above-described embodiment, the guide's field of view was shared with the participants, but this is not limited to this. For example, the participants' avatars may be near the guide, but the camera direction (viewpoint) of the participant object (avatar) may be freely moved by the participants.

[0297] In the auto mode of the embodiment described above, the user's location in the virtual space is linked to the guide video, but is not limited to this. In other words, participants may be allowed to move freely within the virtual space. In this case, for example, when a participant passes a certain point in the virtual space, the processor 122 may automatically play a guide video.

[0298] The above embodiment states that revenue is calculated and distributed from participation fees. Here, "distributing revenue from participation fees" does not mean that the expenses related to the distribution must be withdrawn from the funds received for participation fees. Rather, the expenses related to the distribution may be withdrawn from funds pooled in an account, or from an account separate from the deposit account (such as a withdrawal account). Furthermore, it may be done through one or more accounts. In other words, any payment acceptance and revenue distribution using a generally accepted method of monetary exchange is acceptable.

[0299] Furthermore, in the embodiment described above, revenue was distributed per tour, but revenue distribution does not have to be done per tour; revenue earned from multiple tours may be aggregated and then distributed. Furthermore, the payment system is not limited to commission; a fixed-rate system that is not dependent on the number of tours is also acceptable. The timing of revenue distribution, and whether it is based on a commission or a fixed rate, may vary from one rights holder, guide, and travel agency to another. Furthermore, for example, the distribution to rights holders or guides may be a fixed amount regardless of the number of tours.

[0300] Although only briefly mentioned in the embodiments described above, the exchange of money and other assets in payments and revenue sharing may be conducted through, for example, banks, credit card companies, or payment collection services.

[0301] In the embodiment described above, the URL corresponding to the tourism content and the URL corresponding to the introductory content were different, but they may be the same. In this case, the processor 122 grants participants access rights that are limited by time. For example, participants can enjoy the same paid tourism content for free for the first 5 minutes only.

[0302] In the guide setting process of the above-described embodiment, if the guide has set up a sub-guide, the processor 122 displays the sub-guide (separately from the guide) on the guide display unit UI-13 (not shown). This allows the guide to, for example, have a conversation with a co-guide in a virtual space (conversation mode). The co-guide is, for example, an expert familiar with the tourist facilities on the tour (such as a museum curator or university professor). The co-guide is sometimes referred to as a buddy.

[0303] In other words, if a sub-guide is set in the guide setting process, the processor 122 displays the sub-guide's image on the display unit (guide display unit) of the participant terminal and the guide terminal (referred to as the "sub-guide addition display process").

[0304] Aspects of the present invention, including this embodiment, have the following characteristics. The following corresponds to the claims at the time of filing the application. However, due to amendments to the claims after filing, the description of the claims may differ. (1) In the first aspect, a rights holder setting unit sets up rights holders who have rights to tourist destinations or tourist facilities ("tourist facilities, etc."), a guide setting unit sets up guides for the tourist facilities, etc., a tourist content creation unit creates tourist content by associating text information, image information, or audio information with a three-dimensional model of the tourist facilities, etc. constructed from at least imaging data of the tourist facilities, etc. by an imaging device, a participant application acceptance unit accepts applications from participants for services that provide the tourist content, a guide necessity determination unit determines whether a guide is necessary, a participant fee calculation unit calculates the participation fee for the participant, and provides the participant with information on the tourist content. The present invention provides an online tour provision system comprising: an access rights granting unit that grants access rights; a service publication and provisioning unit that accepts access to the tourism content by the participant and provides the service; a three-dimensional image changing unit that changes the image displayed on the display unit of the participant's terminal based on the participant's operation of the participant's terminal or the guide's operation of the guide's terminal; and an allocation calculation unit that calculates the allocation of the participation fee to at least the rights holder, wherein the access rights granting unit grants the access rights to the participant and the guide when a guide is required, and grants the access rights to the participant when a guide is not required. (2) In a second embodiment, the online tour provision system according to the first embodiment is provided, further comprising a travel agency setting unit for setting up travel agencies that introduce the service, and an allocation calculation unit for calculating at least the allocation to the rights holder from the participation fee, which is an allocation calculation unit for calculating at least the allocation to the rights holder and the allocation to the travel agency from the participation fee. In this case, integrating with the travel agency's system would allow more travel agencies to accept applications, creating an environment that makes it easier for participants to join, and also allow for a share of the revenue to be distributed to the travel agencies. (3) In a third embodiment, the online tour provision system according to the first embodiment is provided, further comprising a viewpoint switching unit that, when the service is provided, displays an image from a virtual camera equipped on another person's object on the display unit of the participant who performed the operation. In this case, participants can share their perspective with others (especially group members and guides), and quickly understand which sightseeing content others want them to see, giving them a sense of traveling together. Furthermore, they can easily see which content the guide is describing. While the online tour system 1 displays avatars of participants, it can be difficult to understand what someone is pointing to just by looking at their avatar; therefore, this feature is particularly effective for the purposes described above. (4) In a fourth embodiment, the online tour provision system described in the first embodiment is provided, further comprising: an application batch acceptance unit that accepts the participation of multiple participants with a single application; and an access rights batch granting unit that grants access rights to the tourism content to each of the multiple participants. In this case, processor 122 can accept group participation and allow multiple people to share and enjoy the same sightseeing content. (5) In a fifth embodiment, the online tour provision system according to the first embodiment is provided, further comprising: an action mode reception unit that accepts the participant's selection of a first action mode or a second action mode; and an action mode-specific service switching unit that, when providing the service, controls participant objects in accordance with the participant's input operations in the case of the first action mode, and controls participant objects in accordance with a manner predetermined by the administrator or guide, or by an operation of the administrator or guide, regardless of the participant's input operations in the case of the second action mode. In this case, the processor 122 can receive the participant's preferences and switch processing depending on whether the participant acts freely or follows predetermined actions when the service is provided. This allows for the provision of services tailored to the participant's hobbies, preferences, and economic situation, resulting in a more user-friendly service. (6) In the sixth aspect, the system further comprises an operation authority transfer unit that, when the service is provided, suspends the participant's operation authority over the participant object and grants said operation authority to the guide, The present invention provides an online tour provision system as described in the first embodiment. In this case, there is the advantage that the guide can ensure that participants see the tourist objects they want them to pay attention to. Participants can avoid the risk of missing the guide's explanations because they don't know where the guide is or other important information. (7) In the seventh embodiment, the online tour provision system described in the first embodiment is provided, further comprising: a quiz setting unit that accepts the setting of a quiz concerning the tourist facilities, etc.; a quiz question setting unit that presents the quiz to the participants; a quiz answer acquisition unit that accepts the answers to the quiz from the participants; a quiz question correctness determination unit that determines whether the answers are correct or incorrect; and a quiz result display unit that displays the results of the correctness determination. In this case, participants can gain a deeper understanding of tourist facilities and other related topics. Furthermore, processor 122 can award points to participants through quizzes, which encourages more active participation. This is an advantage compared to real-world tourism. (8) In the eighth aspect, the online tour provision system according to the first aspect is provided, further comprising: a display unit determination unit that determines whether the display unit of the participant terminal or guide terminal is a single display unit or a display unit that displays separately for the wearer's right eye and left eye; and, if the participant terminal or guide terminal is a terminal that displays separately for the right eye and left eye, a wearable terminal content provision unit that provides a display for the right eye and a display for the left eye to the display unit of the participant terminal or guide terminal. In this case, wearable devices enhance participants' immersion. (9) In the ninth aspect, the online tour provision system according to the first aspect is provided, further comprising an additional sub-guide display unit that displays the image of the sub-guide on the display units of the participant terminal and the guide terminal when a sub-guide is set in the guide setting unit. In this case, there is the advantage of being able to provide participants with more detailed information about tourist facilities and other attractions from multiple guides. Also, because it is an online tour, it is easier to schedule guides, which has the advantage of making it possible to arrange discussions between experts at a low cost. (10) In the tenth aspect, the computer is configured to provide: a rights holder setting means for setting rights holders who have rights to tourist destinations or tourist facilities ("tourist facilities, etc."); a guide setting means for setting guides for the tourist facilities, etc.; a tourism content creation means for creating tourism content by associating text information, image information, or audio information with a three-dimensional model of the tourist facilities, etc. constructed from imaging data of the tourist facilities, etc. by at least an imaging device; a participant application receiving means for receiving applications from participants for a service that provides the tourism content; a guide necessity determination means for determining whether a guide is necessary; a participation fee calculation means for calculating a participation fee for the participant; and a means for allowing the participant to access the tourism content. The present invention provides an online tour provision program that functions as an access permission granting means for granting access rights, a service publication and provision means for accepting access to the tourism content by the participant and providing the service, a three-dimensional image changing means for changing the image displayed on the display unit of the participant's terminal based on the participant's operation of the participant's terminal or the guide's operation of the guide's terminal, and a distribution calculation means for calculating at least the distribution of the participation fee to the rights holder, wherein the access permission granting means grants the access rights to the participant and the guide when a guide is required, and grants the access rights to the participant when a guide is not required. (11) In the 11th aspect, the processor performs the following steps: a rights holder setting step in which the processor stores in a memory unit the rights holders of a tourist destination or tourist facility ("tourist facility, etc."); a guide setting step in which the processor stores in a memory unit the guides of the tourist facility, etc.; a tourism content creation step in which the processor creates tourism content by associating text information, image information, or audio information with a three-dimensional model of the tourist facility, etc. constructed from at least imaging data of the tourist facility, etc. by an imaging device; a participant application acceptance step in which the processor accepts applications from participants for a service that provides the tourism content; a guide necessity determination step in which the processor determines whether or not a guide is necessary; a participant fee calculation step in which the processor calculates the participation fee for the participant; and a participant The method for providing an online tour comprises: an access rights granting step of granting access rights to tourism content; a service publication and provision step in which a communication control unit receives access to the tourism content from the participant and the processor provides the service; a three-dimensional image change step in which the processor changes the image displayed on the display unit of the participant's terminal based on the participant's operation of the participant's terminal or the guide's operation of the guide's terminal; and an allocation calculation step in which the processor calculates the allocation from the participation fee to at least the rights holder, wherein the access rights granting step is characterized in that the processor grants the access rights to the participant and the guide when a guide is required, and grants the access rights to the participant when a guide is not required. [Industrial applicability]

[0305] It can be applied not only to land but also to tourism experiences in normally accessible areas such as underwater, in the air, and in space. [Explanation of symbols]

[0306] 1. Online tour delivery system 10 servers 12 Control Unit 122 processors 124 ROM 126 RAM 128 Timing section 14 Storage section 14a Program storage section 14b Data storage section 16 Communication Control Unit 18 Input / output section 20. Participant's device (Participant's device) 22 Control Unit 24 Memory section 26 Communication Control Unit 28 Input / output section 282 Input section 284 Output section 284a Display section 30 Guide's terminal (Guide's terminal) 40. Travel agency terminal (travel agency terminal) 50 Rights holder's terminal (Right holder's terminal) UI-11 Main Screen UI-12 Participant display section UI-13 Guide Display Unit UI-14 Operation Icon Display Section UI-15 Tour Tag Icon UI-16 Content Display Section UI-17 Guide (Video) Display Unit UI-18 Quiz Screen P1 Online Tour Offering Program P11 Service Provider Configuration Program P12 Tourism Content Creation Program P13 Tourism Content Provision Program D1 Online Tour Database D11 Rights Holder Data Table D12 Data Table for Tourist Facilities, etc. D13 Guide Data Table D14 Participant Data Table D15 Travel Agency Data Table D16 Tour Data Table D17 Tour Tag Data Table D18 Quiz Data Table

Claims

1. A rights holder setting section for setting rights holders who have rights to tourist destinations or tourist facilities (hereinafter referred to as "tourist facilities, etc."), A guide setting unit for setting guides for the aforementioned tourist facilities, A tourism content creation unit that creates tourism content by associating text information, image information, or audio information with a three-dimensional model of a tourist facility, etc., constructed from imaging data of the tourist facility, etc., by an imaging device, The service that provides the aforementioned tourism content includes a participant application reception department that accepts applications from participants, A guide necessity determination unit that determines whether the aforementioned guide is necessary, A participation fee calculation unit that calculates the participation fee for the aforementioned participant, An access permission granting unit that grants access rights to the aforementioned tourism content to the aforementioned participants, A service provision unit that accepts access to the tourism content from the aforementioned participants and provides the aforementioned service, A three-dimensional image changing unit that changes the image displayed on the participant's terminal display based on the participant's operation of the participant's terminal or the guide's operation of the guide's terminal, A distribution calculation unit that calculates at least the distribution to the rights holders from the aforementioned participation fees, Equipped with, The access rights granting unit, If a guide is required, grant the aforementioned access rights to the participants and the guide. An online tour delivery system characterized by granting the aforementioned access rights to participants when a guide is not required.

2. Furthermore, it includes a travel agency setting unit for setting up travel agencies that introduce the aforementioned services, A distribution calculation unit that calculates the distribution to at least the rights holders from the aforementioned participation fees, The online tour provision system according to claim 1, characterized by being an allocation calculation unit that calculates at least the allocation to the rights holder and the allocation to the travel agency from the participation fee.

3. Furthermore, when providing the aforementioned service, Based on the actions of the aforementioned participant, a viewpoint switching unit is provided that displays an image from a virtual camera equipped with another person's object on the display unit of the participant who performed the action. The online tour provision system according to claim 1, characterized by comprising the above.

4. Furthermore, there is a single application processing department that accepts applications from multiple participants, A batch access permission granting unit grants access rights to the tourism content to each of the aforementioned multiple participants, The online tour provision system according to claim 1, characterized by comprising the above.

5. Furthermore, the system includes an action mode receiving unit that accepts the participant's selection of either a first action mode or a second action mode, When providing the aforementioned service, In the first behavior mode described above, the participant object is controlled in accordance with the participant's input operation. In the case of the second behavior mode, regardless of the participant's input, a service switching unit for each behavior mode controls the participant object in a manner predetermined by the administrator or the guide, or by operation by the administrator or the guide. The online tour provision system according to claim 1, characterized by comprising the above.

6. Furthermore, when providing the aforementioned service, An operation authority transfer unit that, through an operation by the guide, suspends the participant's operation authority to the participant object and grants said operation authority to the guide, The online tour provision system according to claim 1, characterized by comprising the above.

7. Furthermore, there is a quiz setting unit that accepts the setting of quizzes related to the aforementioned tourist facilities, A quiz-setting unit that presents the quiz to the aforementioned participants, A quiz answer acquisition unit that receives the quiz answers from the aforementioned participants, A quiz question correctness determination unit that determines whether the aforementioned answer is correct or incorrect, A quiz result display unit that displays the result of determining whether the result is correct or incorrect, The online tour provision system according to claim 1, characterized by comprising the above.

8. Furthermore, the display unit determination unit determines whether the display unit of the participant terminal or guide terminal is a single display unit or a display unit that displays different information on the right and left sides of the wearer's eye, If the participant terminal or guide terminal is a terminal that displays different information for the right eye and the left eye, a wearable terminal content provider unit provides the right-eye display and the left-eye display to the display unit of the participant terminal or guide terminal, The online tour provision system according to claim 1, characterized by comprising the above.

9. Furthermore, if a co-guide is set in the guide setting unit, the co-guide additional display unit displays the co-guide's video on the participant terminal and the guide terminal's display unit, The online tour provision system according to claim 1, characterized by comprising the above.

10. Computers, Means for establishing rights holders who have rights to tourist destinations or tourist facilities (hereinafter referred to as "tourist facilities, etc."), Guide setting means for setting guides for the aforementioned tourist facilities, etc. Tourism content creation means for creating tourism content by associating text information, image information, or audio information with a three-dimensional model of a tourist facility, etc., constructed from imaging data of the tourist facility, etc., by at least an imaging device. A means of accepting applications from participants for the service that provides the aforementioned tourism content, A means for determining whether the aforementioned guide is necessary, A means for calculating the participation fee for the aforementioned participant, Access permission granting means for granting access rights to the aforementioned tourist content to the aforementioned participants, A service provision means that accepts access to the tourism content by the aforementioned participants and provides the said service, A three-dimensional image changing means that changes the image displayed on the display unit of the participant terminal based on the participant's operation of the participant terminal or the guide's operation of the guide's terminal, and Distribution calculation means for calculating the distribution from the aforementioned participation fee to at least the aforementioned rights holder, To make it function as, The access permission granting means is If a guide is required, grant the aforementioned access rights to the participants and the guide. An online tour provision program characterized by granting the aforementioned access rights to participants when a guide is not required.

11. A rights holder setting step in which the processor causes the rights holders of a tourist destination or tourist facility (hereinafter referred to as "tourist facility, etc.") to be stored in the memory unit, The processor performs a guide setting step in which it stores the guides of the tourist facilities, etc., in the memory unit. A process for creating tourism content involves a processor associating text information, image information, or audio information with a three-dimensional model of a tourist facility, etc., constructed from imaging data of the tourist facility, etc., by at least an imaging device, to create tourism content. The processor takes a step of accepting applications from participants for the service that provides the aforementioned tourism content. A processor determines whether or not the aforementioned guide is necessary in a guide necessity determination step. A processor performs a participation fee calculation step to calculate the participation fee for the participant. The processor grants access rights to the participant to access the tourism content. A service provision step in which the communication control unit receives access to the tourism content by the participant and the processor provides the service, A three-dimensional image change step in which the processor changes the image displayed on the display unit of the participant terminal in response to the participant's operation of the participant terminal or the guide's operation of the guide's terminal. The processor performs an allocation calculation step of calculating at least the allocation to the rights holder from the participation fee. Composed of, The aforementioned access permission granting step is: When a guide is required, the processor grants the aforementioned access rights to the participants and the guide. An online tour delivery method characterized in that the processor grants the aforementioned access rights to participants when a guide is not required.

Images