ONLINE TOUR PROVIDING SYSTEM, ONLINE TOUR PROVIDING PROGRAM, AND ONLINE TOUR PROVIDING METHOD

Abstract

To provide a system that reproduces tourist facilities, etc. in a virtual space in order to provide a realistic online tourist experience, by allowing participants to select whether or not a guide will participate. [Solution] The online tour providing system 1 sets up rights holders and guides who have the rights to tourist destinations or tourist facilities (tourist facilities, etc.), sets up tourist content by recreating the tourist facilities, etc. in a virtual space and linking content to them, accepts participants for the service that provides the tourist content, and at this time accepts the participants' choice of whether or not they require a guide, and distributes the profits earned to at least the rights holders.

Description

[Technical field]

[0001] The present invention relates to an online tour providing system, an online tour providing program, and an online tour providing method. [Background technology]

[0002] Digital twins, which reproduce real-world spaces in virtual spaces, are becoming widely known. For example, the interior of a building can be photographed with an imaging device (such as a camera) to obtain the necessary data, and the building can then be constructed and reproduced in a virtual space.

[0003] For example, Non-Patent Document 1 reproduces a real space in a virtual space based on a huge amount of imaging data, and allows users to know the local situation by street view. Furthermore, Non-Patent Document 2 discloses digital twin technology and services using said technology. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Special Publication No. 2023-525561 [Non-patent literature]

[0005] [Non-Patent Document 1] Google Inc. (Google is a registered trademark), “Google Earth”, [online], [searched on January 9, 2025], Internet<URL:https: / / www.google.co.jp / earth / > [Non-Patent Document 2] Matterport Inc., “Matterport” (registered trademark), [online], [Retrieved January 9, 2025], Internet<URL:https: / / matterport.com / >

[0006] One promising application of digital twin technology is in the tourism industry. By showing videos of tourist spots online, viewers who are impressed by the videos can expect to actually visit the spots. Alternatively, if the footage itself has value, compensation may be paid for providing the footage.

[0007] And these business models also involve important processes associated with tourism, such as reservations, fee collection, and revenue distribution.

[0008] For example, Patent Document 1 discloses an online tourism system using drones, although it is different from a digital twin, and also discloses reservation and payment management, and revenue distribution.

[0009] For example, in tourist destinations that include tourist facilities such as historical buildings, guides (tour guides and tour conductors) who have in-depth knowledge of the history and historical value and can explain and introduce them play an important role. Participants can obtain information from information boards set up on-site, but their understanding will be enhanced if they receive explanations from a guide.

[0010] For example, the guide can change the explanation depending on the participants' reactions. If some participants look confused, the guide can explain again in a more understandable way, and if the participants react strongly, the guide can explain more about that part. Another advantage of the guide is that they can answer participants' questions in real time.

[0011] In other words, in order to widely disseminate tourism services using digital twin technology, it is essential to create attractive content and build a system that can provide smooth services to participants. Also, guides with in-depth knowledge can add value to a tourist destination by appealing to its attractions.

[0012] On the other hand, there are also participants who would like to explore tourist attractions and other places freely on their own, without relying on a guide. Therefore, it is more preferable that participants be able to select whether or not they require guidance from a guide. Such improvements in usability are particularly important in promoting tourism services in virtual space.

[0013] The above-mentioned prior art documents do not disclose a system in which the participation of a guide, and in particular a system in which participants can select whether or not they want the guide to participate, is required.

[0014] By the way, guides usually have to accompany tourists or live near the tourist spot. However, this places a restriction on those who have in-depth knowledge of the history and culture of a certain area (tourist destination) and are suitable to be guides to work as guides.

[0015] For example, let's say there is a person who lives in city A, but is interested in the history and culture of faraway city B, and has acquired enough knowledge through self-study to pass a certification exam held by an organization in city B. This person would like to work in a job conveying the appeal of city B to others as their main job or side job, but may not be able to do so because they live far away.

[0016] A shortage of guides is a problem in some tourist destinations, and a solution to this problem is needed. There is also a need to improve the content for promoting tourist destinations through the creative ideas of guides. Summary of the Invention [Problem to be solved by the invention]

[0017] The problem to be solved is that in a system that reproduces tourist facilities in a virtual space to provide a realistic tourist experience online, participants cannot choose the system parts that are essential for tourism, especially whether or not to have a guide participate. In addition, it is necessary to enrich the tourist content in order to expand the options available to participants. [Means for solving the problem]

[0018] The main features of the present invention are to set up rights holders and guides who have the rights to tourist destinations or tourist facilities (tourist facilities, etc.), to set up tourist content by recreating the tourist facilities, etc. in a virtual space and linking content to them, to accept participants of a service that provides the tourist content, and at this time to accept the participants' choice of whether or not they require a guide, and to distribute the profits earned to at least the rights holders.

[0019] The present invention has been made in consideration of the above problems, and employs, for example, the following means. That is, a right holder setting unit that sets the right holder who has the right of a tourist spot or a tourist facility (hereinafter referred to as "tourist facility, etc."); a guide setting unit for setting a guide for the tourist facility or the like; a tourism content creation unit that creates tourism content by associating character information, image information, or audio information with a three-dimensional model of the tourism facility or the like that is constructed from at least image data of the tourism facility or the like captured by an imaging device; a participation application receiving unit that receives applications from participants for the service of providing the sightseeing content; a guide necessity determination unit for determining whether a guide is necessary; a participation fee calculation unit for calculating a participation fee for the participant; an access authority granting unit that grants access authority to the sightseeing content to the participant; a service disclosure providing unit that accepts access to the sightseeing content from the participants and provides the service; a three-dimensional image changing unit that changes an image displayed on a display unit of the participant terminal by an operation of the participant terminal by the participant or an operation of the guide terminal by the guide; an allocation calculation unit that calculates an allocation to at least the right holder from the participation fee; Equipped with The access authority granting unit, granting the access authority to the participants and the guide when a guide is required; The present invention provides an online tour providing system, characterized in that the access authority is given to a participant when a guide is not required. Effect of the Invention

[0020] The online tour providing system of the present invention provides a realistic tourist experience by creating tourist content (i.e., recreating tourist facilities, etc. in a virtual space) by associating text information, image information, or audio information with a three-dimensional model of the tourist facility, etc. constructed from at least image data of the tourist facility, etc. taken by an imaging device, and has the advantage of being able to provide services that meet the needs of participants, such as allowing participants to decide whether or not they require a guide when applying for the tourist content.

[0021] Furthermore, as a result of guides being able to provide guidance online, people with in-depth knowledge of the above-mentioned tourist spots can participate in the tour of the tourist spots, thereby solving the problem of a shortage of guides. [Brief description of the drawings]

[0022] [Figure 1] FIG. 1 is a diagram (network diagram) showing an overview of an online tour providing system 1. [Diagram 2] FIG. 13 is a diagram showing a service (online tour) providing screen. [Diagram 3] FIG. 2 is a diagram showing a content display screen. [Figure 4] FIG. 13 is a diagram showing a screen in an auto mode. [Diagram 5] FIG. 13 is a diagram showing a quiz display screen. [Figure 6] 13 is a flowchart showing an overview of an online tour providing process. [Figure 7] 13 is a flowchart showing a sightseeing content providing process. [Figure 8] FIG. 2 is a hardware configuration diagram of the server 10. [Figure 9]FIG. 2 is a hardware configuration diagram of a terminal (such as a participant terminal 20). DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described with reference to the accompanying drawings. In the following embodiments, the same or corresponding parts are designated by the same reference numerals, and the description thereof may be omitted as appropriate. The drawings used below are used to explain the present embodiment, and may differ from the actual device configuration, user interface (UI), data configuration, and the like.

[0024] (Overview of the embodiment) The outline of this embodiment will be described with reference to FIG. FIG. 1 is a diagram (network diagram) showing an overview of a system (hereinafter referred to as an "online tour providing system 1") that performs processing according to an online tour providing program P1 of this embodiment. In the online tour providing system 1 of this embodiment, a (usually multiple) server 10 and terminals of participants, guides, travel agencies, and right holders (referred to as participant terminal 20, guide terminal 30, travel agency terminal 40, and right holder terminal 50, respectively) are communicatively connected via a network N. The terminals of participants and guides are, for example, PCs (personal computers) and HMDs (head mounted displays).

[0025] The server 10 constructs a three-dimensional model in a virtual space to reproduce tourist facilities, etc., and also creates tourist content by associating various information (text, images, audio, etc.) with the three-dimensional model, and provides participants with tours for enjoying this tourist content. Participants apply for participation by selecting the service (tour) they wish to participate in from the available options, the type of tour (whether or not a guide is present), and pay the participation fee. After completing payment, participants join the tour (connect online) at a set time, for example. Participants enjoy the tour by moving around tourist facilities in the virtual space through their avatars and viewing and viewing various information associated with the three-dimensional models. FIG. 2 is a diagram showing a state in which a participant is taking part in a tour (a diagram showing a service (online tour) providing screen) (when the participant terminal 20 is a PC). Within the virtual space, participants and / or guides (hereinafter referred to as "Participants, etc.") can move freely and converse as if they were moving in real space. Details will be described later. Revenues generated from participation fees are distributed to the rights holders of tourist facilities, participating guides, and intermediary travel agencies.

[0026] (Details of the embodiment) The online tour providing system 1 according to this embodiment will be described in detail below. The online tour providing system 1 includes a computer (server 10) equipped with an online tour providing program P1, and provides participants with a system that realizes a realistic and highly flexible online tour. That is, in the online tour providing system 1, information processing by the online tour providing program P1 is specifically realized using hardware resources. The following describes the components of the online tour providing system 1 in order: 1. user interface, 2. program processing, 3. data, and 4. hardware configuration.

[0027] (Definition of terms) Here we define some terms. The "administrator" is an administrator of the online tour providing system 1, and has special privileges over the online tour providing system 1. The administrator and the person who provides the tourist content providing service (the person who sets up the tour) through the online tour providing system 1 may be the same or different, and no strict distinction is made here.

[0028] A "right holder" is a person who has rights to a tourist destination or tourist facility. For example, it is the owner or manager of a building that displays exhibits or a structure (that has historical or other academic value), the right holder of land that is a natural asset, etc. A right holder can be an individual or a corporation. If there are multiple right holders, it is sufficient that at least some of them are the right holders. Furthermore, a right holder is not limited to a direct right holder, but can also be a person with authority such as being entrusted with management by a direct right holder, or an agent. For example, if the owner of a tourist destination or tourist facility is a country or local government, it can also be an organization that is entrusted with management. Furthermore, the right referred to here can be for tangible or intangible objects. For example, it can be a right related to land, buildings, objects, or services. For simplicity, we will refer to "tourist destinations or facilities" as "tourist facilities, etc." Tourist destinations and facilities are not separate concepts and can overlap. For example, a tourist facility may be the tourist destination itself, or a tourist destination may include multiple tourist facilities. Regarding the words "tourist attractions" or "tourist facilities," whether they refer to something in the real world or something in the virtual world is determined by the context. Usually, they refer to tourist attractions or facilities in the real world.

[0029] A "participant" is a person who participates in the tourism content providing service set in the online tour providing system 1 of this embodiment. The term "participant" includes a person who is in the application stage and plans to participate, and a person who is participating in the service by using the system. A "guide" is a person who guides participants in the tourism content provision service set up in the online tour provision system 1 of this embodiment. He may be called a tour guide, tour conductor, or host. A guide may also be a person who creates tourism content and sets up tours. In this case, the guide has privileges for the tours he sets up. As mentioned above, participants and / or guides may be referred to as "participants, etc."

[0030] "Imaging data" refers to image data (including still images and moving images) acquired by an imaging device. It may also include audio data recorded by an audio recorder provided in the imaging device. For example, in this embodiment, in addition to pass-through video (image data captured by the imaging device and displayed in real time on the display unit of the HMD), there is also image data for combining point cloud data and data that has been converted into three-dimensional data. The term "image" refers to either a still image or a moving image, and whether the image is a real-time image or a recorded image. The term "image" is sometimes used interchangeably with the term "image" (especially moving image). A "three-dimensional model" is a model created as three-dimensional, stereoscopic data in a virtual space created by a computer (information processing device).

[0031] "Tourist content" is an object in a virtual space that associates various information (text, images, audio, etc.) related to tourist facilities and the like (usually multiple pieces of information) with a three-dimensional model. In the following, the term "content" may be used as a general term. For example, the various information related to the tourist facilities mentioned above can be considered content. "Tour" is the Japanese transliteration of "tour," but here it specifically includes a service (or a group of services) that provides tourist content or a package of tourist content provision services. For example, it can be expressed as a tourism content that recreates "Shuri Castle" in a virtual space and a tour that includes that tourism content (such as a "Shuri Castle tour"). One tour may include multiple tourist attractions, such as an "Okinawa tour" that includes tourist attractions related to "Shuri Castle" and "Churaumi Aquarium."

[0032] A "tour tag" is a tag associated with a point such as a famous spot on a tour. In some cases, various information (text, image, audio, etc.) is associated as content, while in other cases, there is no corresponding tourist content (it simply indicates a location). For example, in the tourist content of "Shuri Castle," there are "Shureimon," "Main Hall," and "Shop." An "avatar" is a character (object) or icon that is placed in the virtual space as an alter ego of a participant or the like. A participant avatar and a guide avatar are also referred to as a participant object and a guide object, respectively. The object referred to here is one of the data in the virtual space, and a participant object and a guide object correspond to a participant and a guide, respectively. A participant or the like can operate an avatar (object) by operating his / her own terminal (input to an input unit). In the virtual space, each object of a participant or the like has a virtual camera. The image captured by this virtual camera becomes the field of view of the participant or the like, and the processor (122) displays this image on the display unit of the terminal of the participant or the like. In this embodiment, the processor 122 displays the avatar as a semi-transparent human figure in the virtual space, and one participant can see the avatar of another participant. The processor 122 can also change the viewpoint position by the operation of the participant or the like. In other words, a virtual camera can be set at the eye height of the avatar, or a virtual camera can be placed slightly behind and above the avatar, so that the viewpoint is one in which the participant's own avatar is viewed from slightly behind.

[0033] In the following description, when "XX processing" is mentioned, it means that the processor of the computer executes processing based on the "XX" program stored in the program storage unit. In this paragraph, the same word will be substituted for "XX". In other words, the "XX" program is a program that causes a computer to function as "XX" means by executing "XX" processing. In this case, the control unit equipped with the processor also functions as the "XX" unit (or "XX" device). In this case, the "XX" part means executing "XX" processing based on the "XX" program. Furthermore, the "XX" process is expressed as a procedure, namely, "XX" step (or "XX" procedure).

[0034] For example, the online tour providing program P1 is a program that causes a computer to function as an online tour providing means by executing an online tour providing process. In addition, at this time, the control unit 12 of the computer including the processor 122 functions as an online tour providing unit (or an online tour providing device).

[0035] In the online tour providing system 1, each terminal (computer) of the participant, guide, and travel agency is equipped with a processor, but when simply referring to a processor, it refers to the processor that performs processing according to the online tour providing program P1, which in this embodiment refers to the processor 122 of the server 10.

[0036] In the following, for simplicity, "processor 122 of server 10 receives a request from a terminal and returns data to be displayed on the browser of the terminal" may be described as "processor 122 displays (causes) it to be displayed on the browser of the terminal" or "processor 122 displays (causes) it to be displayed". Similarly, "the processor 122 of the server 10 causes the data to be stored in the data storage unit 14b of the storage unit 14" may be expressed as "the processor stores (causes) (the data)" or the like. In addition, it is the avatars of the participants and guides that move (etc.) in the virtual space, but for simplicity's sake, it may be written that the participants and guides move (etc.).

[0037] 1. User Interface (UI) First, the interface that the online tour providing system 1 of this embodiment displays on the participant terminal 20 or the like will be described with reference to the drawings. The interface described below is a simplified version of what the processor 122 causes to be displayed on the display unit (display) of the participant terminal 20 or the guide terminal 30.

[0038] In addition, only icons related to functions necessary for the explanation are displayed, and other well-known icons are omitted. For example, the exit button (x button) for closing (ending) the screen is omitted.

[0039] The online tour providing system 1 provides a function for participants and guides to meet online and tour tourist spots together (in a virtual space). At least one participant participates in the tour. Depending on the participant's choice, the guide may also join the tour. Participants can move around the virtual space freely or under the control of a guide or the like, and view and listen to the tourist content.

[0040] The online tour providing system 1 provides multiple types of tours. Participants select one of the multiple types and apply for it. The multiple types include at least a type based on participants and a type based on content.

[0041] In this embodiment, there are eight main types of participants. This can be divided into whether there is one participant or multiple participants (friends or other acquaintances), whether there is a guide or not, and whether or not the tour is conducted with a third party (2 ways x 2 ways x 2 ways). The third party here does not refer to a person who constitutes the "plurality" of participants who are acquaintances, but rather to a complete stranger.

[0042] The cost of participation varies depending on these types: for example, if there are multiple participants the cost per person is set (e.g. by the administrator) lower than if there is only one participant, if there is no guide the cost is lower than if there is a guide, and if there is a third party on the tour the cost is lower than if there is no guide.

[0043] When there is one participant, it is called a "standalone," and when there are multiple participants, such as friends, it is called a "group." In addition, when a guide is present it may be referred to as "with guide," and when not, it may be referred to as "without guide," and when a third party is present it may be referred to as "with third party," and when not, it may be referred to as "without third party."

[0044] In this embodiment, there are two main types of content: a "manual mode" in which participants are free to act within the virtual space, and an "auto mode" in which participants' actions are controlled by an administrator or guide.

[0045] In auto mode, participants do not control the movement of their own avatars, but instead move through the virtual space in a route and time determined in advance by the administrator, or the movement of the participant avatar is linked to the movement of the guide avatar. In addition, the administrator or guide may control the movement of the participant avatar at their own discretion.

[0046] Even in the manual mode, the processor 122 displays on the screen guidance icons such as arrows indicating the route.

[0047] The participation fee varies depending on the type of content. Auto mode has a lower participation fee than manual mode.

[0048] If no guide is present, participants can choose manual or automatic mode.

[0049] When a guide is present, the selection of manual or automatic mode by the participants is similar, but the guide can control the participants' actions.

[0050] Specifically, when the guide operates the guide terminal 30 (for example, by operating the guide mode icon (not shown) displayed on the screen) to activate the "escort mode" in which the guide obtains the participants' authority to act, the guide can gather the avatars of the participants taking part in the tour near the guide's (his own) avatar. The distance between the participant avatar and the guide avatar can be set arbitrarily. In this embodiment, the distance is zero (the participant avatar and the guide avatar are in the same position).

[0051] Furthermore, in this embodiment, the view of the guide is shared with the participants, and the participants view objects, scenery, etc. captured by the virtual camera of the guide object. In this embodiment, while the leader mode is active, the processor 122 displays a flag icon on the screens of the participants and the guide.

[0052] When the guide turns off "lead mode," the participant avatars can resume their actions from the location where they were collected (where the guide avatar is) or from the location they were before they were collected (free movement in manual mode, or continuing the tour in auto mode).

[0053] For example, in the case of one participant, no guide, no third party, manual mode, the participant tours tourist spots (in the virtual space) alone and is free to move around within the virtual space.

[0054] In addition, in the case of one participant, no guide, no third party, and auto mode, a guide video is prepared. The guide video is a video (movie image including audio) that contains an explanation by the guide. The processor 122 of the server 10 displays the guide video and controls the location of the participants in the virtual space.

[0055] For example, suppose there is a tourist facility that is connected to Room A, a corridor, and Room B. When the guide video is explaining Room A, the participant is also in Room A, and by changing the viewpoint at that location, the participant can view Room A in 360 degrees. Then, when the explanation in the guide video moves to Room B, the location of the participants, along with their viewpoint, smoothly moves from Room A to the hallway and back to Room B. In other words, participants can also view the hallway that is the route from Room A to Room B. The time it takes to move can be set appropriately by the administrator or guide.

[0056] Note that the URLs associated with the sightseeing contents may be different depending on the type, such as the type by the participants, the type by the contents, etc. In this case, it is possible to reduce the load on the server. A specific screen will be shown and explained below.

[0057] FIG. 2 is a diagram showing a service (online tour) providing screen. Participants can act within the virtual space by operating on the screen (clicking, pointing, etc.).

[0058] Participants can converse with each other within the virtual space, and at this time, processor 122 recognizes the locations of the participants and the walls, floors, or ceilings that separate the space, and controls the sound (volume) that the participants hear.

[0059] For example, if two or more participants are close to each other, they can converse, but if they are far apart, they cannot converse. In addition, even if two or more participants are close to each other, if there is a wall, floor, or ceiling between them, that is, if they are in different rooms or floors even if they are physically close, they cannot talk to each other. Details will be explained in the program processing section.

[0060] The service (online tour) providing screen includes a main screen UI-11, a participant display section UI-12, a guide display section UI-13, an operation icon display section UI-14 (UI-14A, UI-14B), and a tour tag icon UI-15.

[0061] If the participant or guide's terminal is a regular PC or the like and the display unit is a flat display, the image will be displayed flatly as shown in FIG. On the other hand, if the participants and guides use HMD devices, the images displayed are three-dimensional, and the HMD wearer can enjoy a sense of immersion because images of the tourist spots are displayed no matter where they look in 360° (up, down, front, back, left, right).

[0062] The main screen UI-11 displays images of participants and other objects (avatars) from a virtual camera. Specifically, the images are virtual tourist facilities and the like created based on data acquired in advance by photographing them with an imaging device or the like.

[0063] Here, the operation of the avatar in this embodiment will be described. Participants can move to a certain location by pressing (clicking, pointing, etc.) on a part of the floor a short distance ahead (displayed on the main screen UI-11). In other words, they can move their own avatar.

[0064] Also, participants can change their viewpoint without changing their position. If the participant's terminal is an HMD, when the participant moves his / her head, the sensor of the HMD detects the movement and acquires it as an input, and the processor 122 receives the input and changes the angle (yaw direction, pitch direction, or roll direction) of the virtual camera. In other words, if the participant wearing the HMD turns to the right, the image of the participant turning to the right is displayed on the display unit (main screen UI-11) of the terminal.

[0065] If the participants' terminals are desktop PCs, they can change the viewpoint by dragging and dropping on the main screen UI-11. For example, suppose you drag (mouse click, etc.) in the top right of the main screen UI-11, and then maintain that state (maintain the mouse click state) to move the cursor to the center. In this case, the image that was in the top right of the main screen UI-11 moves to the center of the main screen UI-11. In other words, the image of the participant looking to the top right is displayed on the main screen UI-11.

[0066] In addition, some of the buttons on the input devices, such as the buttons on the controller attached to the HMD or the buttons on the mouse and keyboard of a desktop PC, are so-called action buttons, and participants can perform certain actions within the virtual space by pressing specific buttons (hereinafter referred to as "action operations").

[0067] For example, when an avatar approaches a location (tour tag) where content is embedded, the processor 122 displays a tour tag icon UI-15 (described later) on the main screen UI-11. When a participant operates a specific button (action operation) in the location, the embedded content is played.

[0068] In other words, when a participant object is located within a certain distance from a location in the virtual space where content is embedded, processor 122 provides the participant with the content embedded at that location by accepting a specified operation from the participant (referred to as the "embedded content provision process").

[0069] It should be noted that known techniques can be used as appropriate for the above-mentioned movement, viewpoint operation, and action operation.

[0070] Returning to the explanation of Fig. 2, the participant display unit UI-12 displays images (usually video images) of the participants captured by an imaging device (camera) built into the participant terminal 20 or connected to the participant terminal. Similarly, the guide display unit UI-13 displays an image (usually a moving image) of the guide captured by an imaging device (camera) built into the guide terminal 30 or connected to the participant terminal.

[0071] The operation icon display unit UI-14 (UI-14A, UI-14B) provides participants with preset functions.

[0072] 2, in this embodiment, the operation icon display unit UI-14 includes an operation icon display unit UI-14A and an operation icon display unit UI-14B. Here, two operation icon display units are displayed separately, but these may be integrated into one, or conversely, may be divided into three or more. Each of these will be explained below.

[0073] Normally, the operation icon display unit has a plurality of operation icons, each having a shape and color corresponding to its function, but for simplicity, they are represented here by the alphabets A to G and the katakana characters A to E. Further, in the following description, "these are icons for participants to do ____ to ____," this is controlled by the processor 122 to produce such an effect.

[0074] The icons provided on the operation icon display unit UI-14A are A: guide display icon, B: participant display icon, C: tour tag list display icon, D: auto mode icon, E: viewpoint change icon, F: enlarged display icon, and G: mini map icon.

[0075] A: The guide display icon is an icon for participants to select whether or not to display the image of the guide on the screen. When selected once, the guide display section UI-13 disappears, and when selected again, the guide display section UI-13 appears. In this embodiment, even if the participants do not display the image of the guide, they can hear the guide's voice, etc. However, depending on the settings, it is also possible to mute the guide's voice when the image of the guide is not displayed.

[0076] B: The participant display icon is an icon that allows participants to select whether or not to display the participant's image on the screen. In addition to the operation method, it is similar to the guide display icon in that audio can be heard when the image is not displayed and the audio can be muted.

[0077] C: Tour tag list display icon is an icon that displays a list of the locations of tour tags in the tour that the participants are participating in. For example, the entrance, the garden, the first room, the second room, or the rooftop (in the above example, the Shureimon Gate, the main hall, or the shop).

[0078] When a participant selects a location from which a list of participants is displayed, the participant can move to that location. Specifically, the processor 122 performs a process of moving the avatar of the participant to the selected location (to the corresponding position in the virtual space). The speed of movement can be set by the administrator at will, allowing participants to move either instantly or slowly.

[0079] In this embodiment, participants can instantly move to a desired location by selecting a tour tag. Since participants can instantly move to a desired location, they can efficiently view tourist facilities. Another advantage is that participants and guides can move freely even at tourist facilities that have difficulties in terms of barrier-free access.

[0080] D: Auto mode icon is an icon for participants to switch to auto mode. When the auto mode icon is selected, participants can look around virtual tourist facilities (including tourist content) in order according to the route that is predefined for each tour (auto mode). At this time, the processor 122 controls the position information of the participants, so the participants do not need to move around by themselves. In this embodiment, even if a participant applies to participate in the event in manual mode, the participant can switch to automatic mode.

[0081] E: The viewpoint change icon is an icon for participants to obtain the viewpoint of other participants, etc. In other words, participants who select the viewpoint change icon can see what kind of image the other participants, etc. are looking at. Specifically, in response to an operation by a participant, etc., the processor 122 performs a process (referred to as a "viewpoint switching process") of displaying an image from a virtual camera equipped on the avatar (object) of another person on the display unit of the participant, etc. who performed the operation.

[0082] F: The magnification icon is an icon that allows participants to enlarge what is in their field of vision. In addition to enlarging nearby objects, it can also enlarge distant objects. For example, if something close to you is written in small letters that you cannot read, or if you cannot see a picture painted on a high ceiling, you can make it visible.

[0083] G: Minimap icon is an icon that allows participants to display a minimap on their screen. In this embodiment, the minimap displays the tour tag and the locations of participants, including yourself. Participants can move to the location of a tour tag by selecting the tour tag. The specific process by the processor 122 is the same as the process of the above-mentioned C: Tour Tag List Display Icon.

[0084] The icons provided on the operation icon display unit UI-14B are: A: camera movement icon, B: slide show icon, C: quiz enable / disable icon, and D: comment function icon.

[0085] A: The camera movement icon is an icon for changing (returning) the viewpoint from an object such as a participant in the virtual space.

[0086] In this embodiment, at least some of the tour tags automatically (forcibly) change the viewpoints of the participants, etc. For example, if the tour tag refers to writing in the floor, the participant may lose track of where to look, so the processor 122 causes the participant object's virtual camera to point downwards. That is, the processor 122 controls the virtual camera of the participant object at a specific predefined location.

[0087] Here, content is displayed, etc., either by the participant's operation or automatically, and after the display, etc. is finished, the participant, etc. can press a camera movement icon, etc., and the virtual camera will return to the position previously set by the administrator, etc., over a preset period of time.

[0088] In order to prevent the same content from being displayed repeatedly, this event is managed by a flag, etc. In other words, after the embedded content is viewed, etc., the processor 122 controls so that the viewpoint does not move automatically.

[0089] Additionally, the camera movement icon may have the role of resetting the position of the virtual camera when an abnormality occurs in the viewpoint due to an incorrect operation by a participant or the like.

[0090] A: The slide show icon is enabled in a location where multiple images are set as content, and causes multiple images to be displayed on the display unit when a participant or the like selects them.

[0091] C: The quiz enable / disable icon is an icon for enabling or disabling the quiz. When enabled, participants can enjoy the quiz questions described below in the location where the quiz is set as content.

[0092] D: The comment function icon is an icon that enables participants to leave comments (textual information) on virtual tourist facilities, etc. (such comments are referred to as "memory notes"). This is a function that allows participants to keep a memento of their visit to a virtual tourist attraction, for example. Specifically, the processor 122 takes participant input and associates that information with the location of the participant's object. In this embodiment, only character information is left, but audio information, image information, etc. may also be left.

[0093] The layout of elements displayed on the screens of participants, etc., such as the main screen UI-11, participant display section UI-12, guide display section UI-13, and operation icon display section UI-14, can be changed as appropriate (the same applies to the following screen elements).

[0094] Returning to Fig. 2, the tour tag icon UI-15 is an icon that displays the location where the content is embedded. In other words, the tour tag icon UI-15 indicates the location where text information, image information, or audio information that explains the location or items related to the location is set.

[0095] The tour tag icon UI-15 is usually set in multiple locations within the virtual space, and when a participant reaches that location (so that it can be displayed on the screen), they can access explanatory information by performing a specified operation (such as the action operation described above).

[0096] In other words, when a specified operation (action operation such as clicking or pointing) is performed on the tour tag icon UI-15, the processor 122 accepts the input and displays content such as text information, image information, or audio information associated with the tour tag on the display unit of the participant's terminal, etc.

[0097] More specifically, when a participant performs a predetermined operation (action operation) near the tour tag icon UI-15, the processor 122 displays the content in the content display unit UI-16 described next. The details of the process will be described later.

[0098] FIG. 3 is a diagram showing a content display screen. The processor 122 displays preset commentary information, such as text information, audio information, or image information (still images, moving images), on the content display unit UI-16. The content display section UI-16 in Fig. 3 displays a moving image with audio. When a participant clicks the play button (bottom right of the content display section UI-16), the processor 122 plays the video. The bottom left of the content display section UI-16 is a volume adjustment button. In FIG. 3, the operation icon display unit UI-14 is omitted.

[0099] FIG. 4 is a diagram showing the screen in the auto mode. The contents that have already been explained (screen elements such as the participant display section UI-12 and the content display screen, etc.) will be omitted from illustration and explanation because the contents are duplicated. In auto mode, the screen (image) displayed on the main screen UI-11 changes sequentially, and a guide gives commentary according to the location. (In some cases, participants can control the viewpoint, and in other cases, the viewpoint is predetermined.)

[0100] In the auto mode, the processor 122 displays a main screen UI-11 and a guide (moving image) display section UI-17, and displays a moving image and a moving image of the guide that are preset for each tour, respectively. Note that the guide (moving image) display section UI-17, which displays a moving image previously stored by the processor 122, is different from the guide display section UI-13, which displays the guide in real time.

[0101] Although the participant display section UI-12 is not shown in FIG. 4, when there are two or more participants, the processor 122 displays the participant display section UI-12.

[0102] FIG. 5 is a diagram showing a quiz display screen. In the virtual space, a quiz spot (not shown) is prepared together with the tour tag icon UI-15 or separately from the tour tag icon UI-15. When the user performs a predetermined operation (action operation) near the quiz spot, the processor 122 displays a quiz screen UI-18 shown in FIG. Through the quizzes given, participants can have fun while learning about the history and culture of tourist facilities and other places. In FIG. 5, the operation icon display unit UI-14 is omitted.

[0103] In this embodiment, rewards are given depending on the number of correct answers (or the rate of correct answers) to the quiz. The rewards may be points that can be used around the tourist spot or in other tourist content providing services (tours), for example. In the online tour providing system 1, participants can purchase destination-specific products or services online. When making this online purchase, the above points can be used, meaning that discounts such as reductions in prices can be received according to the number of points. Alternatively, if a participant has participated in multiple tours and accumulated points, they can use those points when applying for a new tour.

[0104] With the above configuration, participants and guides can join the tour from anywhere as long as they are in an environment with a network connection. In other words, participants and guides can enjoy sightseeing wherever they are and wherever the tourist spot is.

[0105] In particular, participants using HMDs can have an immersive and realistic sightseeing experience through the online tour provision system1 that utilizes MR (Mixed Reality / Augmented Reality). Furthermore, since guides can perform their work from home, they do not need to take on the burden of traveling to tourist spots, and can contribute to tourist information and ultimately to the development of tourist spots.

[0106] 2. Program Processing <Online tour provision processing> The program processing performed in the online tour providing system 1 of this embodiment will be described. In addition, the "steps" in the specification are represented as "S" in the drawings.

[0107] FIG. 6 is a flowchart showing an outline of the entire online tour providing process. First, the overall process will be explained using FIG. 6, and then each process will be explained. In FIG. 6 and FIG. 7 described later, it is assumed that participants have applied for a tourism content providing service (tour) through a travel agency and have requested the participation of a guide.

[0108] In FIG. 6, the dotted arrows do not necessarily represent processes, but rather represent contractual relationships between users, owners, or managers of each terminal (hereinafter referred to as "users, etc.").

[0109] A facility usage contract is concluded between a provider of services related to the online tour providing system 1 (such as a user of the server 10) and a right holder of a tourist facility, etc. (such as a user of the right holder terminal 50) (C1 in Figure 6). In other words, the rights holder grants permission to photograph tourist facilities etc. with imaging devices etc., to obtain the photographic data etc., to replicate those tourist facilities etc. in a virtual space, and to earn revenue from those tourist facilities etc. in the virtual space.

[0110] The server 10 acquires a three-dimensional model constructed from imaging data of tourist facilities and the like, and stores tourist content that associates the content with the three-dimensional model (step 1). In this embodiment, one tourist content corresponds to one URL.

[0111] A platform usage contract (and a fee collection agency contract) is concluded between a provider of services related to the online tour providing system 1 (such as a user of the server 10) and a travel agency (such as a user of the travel agency terminal 40) (C2 in Figure 6).

[0112] In this embodiment, a travel agency introduces a service (tour). A participant (a person who wishes to participate) of the service who has seen the service applies (online) to the travel agency (step 2), and the travel agency terminal 40 notifies the server 10 of the application (step 3).

[0113] The processor 122 of the server 10 acquires input from the participants regarding whether or not they need a guide (step 4). If a guide is requested, the guide (guide terminal 30) is notified (step 5). The participants respond (notify) that they will participate (step 6).

[0114] The server 10 calculates the participation fee and notifies the participant (participant terminal 20) (step 7), and the participant makes the payment (step 8). For simplicity, Fig. 6 shows the payment from the participant terminal 20 to the server 10, but in reality, the transaction also involves a settlement institution (such as a bank or credit card company). In other words, it is the settlement institution server that accepts operations related to the payment and issues notices of the payment. The same applies to revenue distribution.

[0115] The server 10 grants access rights to the participants (participant terminal 20) and the guide (guide terminal 30) (step 9), and publishes the service (tour) in a form that allows participation (access) at a predetermined scheduled time (step 10). The server 10 then accepts participation (access) from the participants (participant terminals 20) and the guide (guide terminal 30) and provides services (steps 11 and 12).

[0116] The server 10 calculates the profit from the participation fee and distributes the profit according to the predetermined fee, rate, etc. (step 13). In the example of Figure 6, profits are distributed to the rights holder as well as to the participating guides and intermediary travel agencies.

[0117] Next, each process will be described.

[0118] In this embodiment, the processor 122 performs online tour providing processing based on the online tour providing program P1. The online tour providing program P1 includes at least a service provider setting program P11, a tourism content creation program P12, a tourism content providing program P13, and a quiz question setting program P14, and the processor 122 executes a service provider setting process, a tourism content creation process, a tourism content providing process, and a quiz question setting process based on each of these programs. Furthermore, these programs cause the computer to function as a service provider setting means, a tourist content creating means, a tourist content providing means, and a quiz question answering means, by executing the respective processes.

[0119] <2-1. Service provider setting process> In the service provider setting process, the processor 122 sets a provider of a service (a service that provides tourist content). In this embodiment, the processor 122 sets a right holder who has rights regarding the tourist facilities and the like and opens the tourist facilities and the like to the public, a guide (and assistant guide) who provides guidance around the tourist facilities and the like, and a travel agency. That is, the processor 122 stores information on the right holder, the guide (including the assistant guide), and the travel agency in the storage unit 14.

[0120] The service provider setting process includes a rights holder setting process that sets the rights holder of a tourist destination or tourist facility (hereinafter referred to as "tourist facility, etc."), a guide setting process that sets a guide for the tourist facility, etc., and a travel agency setting process that sets a travel agency that introduces the service.

[0121] The processor 122 sets (registers) right holders who have rights to tourist facilities and the like, thereby enabling distribution of profits to the right holders. Similarly, the processor 122 sets (registers) a guide (and an assistant guide) who will guide the participants, thereby enabling distribution of profits to the guide (and the assistant guide). The information to be registered will be explained in the database section below.

[0122] Furthermore, the processor 122 sets (registers) travel agencies that introduce the service, thereby enabling profit distribution to the travel agencies.

[0123] <2-2. Tourism content creation process> In the tourist content creation process, the processor 122 obtains a three-dimensional model based on tourist facilities and the like in the real space, and reproduces the tourist facilities and the like in the virtual space. The processor 122 also associates information (contents) such as text information, image information, or audio information with the three-dimensional model in the virtual space, and creates tourist content. Participants can enjoy (view, etc.) the service that provides the tourism content online.

[0124] That is, the tourist content creation process creates tourist content by associating text information, image information, or audio information with a three-dimensional model of a tourist facility etc. constructed from image data of the tourist facility etc. captured by an imaging device.

[0125] Here, the person who creates the tourist content is the administrator or the guide. That is, the guide may provide information about content that has been created in advance by an administrator or the like, or the guide may create and provide the content himself.

[0126] When a guide creates tourist content, the guide may access the server 10 to create the tourist content, or may upload data to the server 10.

[0127] In other words, the above-mentioned tourist content creation process can create tourist content by associating text information, image information, or audio information (content) obtained from a guide terminal with a three-dimensional model of the tourist facility, etc. constructed from imaging data of the tourist facility, etc. taken by at least an imaging device.

[0128] When guides create content, they can provide unique content based on their knowledge. In other words, different guides can provide different content for the same tourist spot, so participants can choose the content that best suits them. Just as the advent of video sharing sites has given rise to many video channels, each of which has its own fan base, this allows content creators to create content freely based on their creativity. Current video sharing sites provide so-called two-dimensional content, whereas the online tour providing system 1 provides three-dimensional content.

[0129] In this embodiment, the administrator or guide can provide a part of the sightseeing content as introduction content to participants free of charge. The introductory content is promotional content that introduces the outline of the content to participants. The introductory content will be provided to participants free of charge.

[0130] That is, processor 122 acquires introduction content that includes at least a part of the sightseeing content and introduces the sightseeing content (this will be referred to as "introduction content creation processing"). The introduction content in this embodiment also associates text information, image information, or audio information (contents) with a three-dimensional model of a tourist facility or the like. In this embodiment, the URL corresponding to the introduction content is separate from the URL corresponding to the (paid) sightseeing content.

[0131] Next, the three-dimensional model will be described. Here, we will explain a three-dimensional model (first three-dimensional model) constructed from (usually multiple) pieces of imaging data from an imaging device, and a three-dimensional model (second three-dimensional model) constructed from imaging data from the imaging device and distance data from a distance measuring sensor.

[0132] The first three-dimensional model is created from a photograph or the like, and does not require distance data (or point cloud data) from a distance measuring sensor when creating the three-dimensional model. In addition, since the first three-dimensional model is based on imaging data, it includes information on the colors, patterns, and so on of objects such as buildings.

[0133] The second three-dimensional model is constructed from imaging data obtained by an imaging device and distance data (point cloud data) obtained by a distance measuring sensor. A three-dimensional model is constructed from the point cloud data and then combined with information from the imaging device, so that the second three-dimensional model includes information such as the color and pattern of objects such as buildings.

[0134] The first 3D model is for example based on a technique called photogrammetry, NeRF (Neural Radiance Fields), or Gaussian Splatting.

[0135] Photogrammetry is a method of obtaining the geometric characteristics of an object from photographic images, etc., but here it specifically refers to a technology that generates a three-dimensional model of the object from multiple two-dimensional images taken of the object from multiple angles.

[0136] NeRF is a neural network that builds a 3D model from multiple 2D images.

[0137] Gaussian Splatting involves obtaining point cloud data using Structure from Motion (SfM) technology, which creates three-dimensional point cloud data from imaging data such as photographs and videos, replacing the point cloud data with a 3D Gaussian, rasterizing the 3D Gaussian for comparison with the actual image, and optimizing it by adjusting the parameters of the 3D Gaussian.

[0138] In particular, in this embodiment, a three-dimensional model is used that is output by inputting image data captured by an imaging device into a "machine learning model for creating a three-dimensional model."

[0139] A machine learning model for creating three-dimensional models is, for example, a model that learns from at least image data of an object (such as the ceiling, floor, or wall inside a building, or the building itself) captured by an imaging device for model creation, information regarding the length of the object, information regarding the distance from the imaging device for model creation to the object, and a three-dimensional model corresponding to the object as learning data, and outputs a three-dimensional model of the object using the image data of the object as input.

[0140] As such a three-dimensional model, for example, imaging data acquired by an imaging device of Matterport Inc. can be sent to a (cloud) server of Matterport Inc., and a three-dimensional model obtained as the output of a machine learning model for creating three-dimensional models provided by the server, or a three-dimensional model owned by Matterport Inc. can be used (Matterport is a registered trademark).

[0141] The first 3D model, i.e., a 3D model based on photogrammetry, NeRF, or Gaussian Splatting, or a 3D model based on the machine learning model described above, does not require distance data (point cloud data) from a ranging sensor, but the accuracy of the 3D model may be improved by using distance data (point cloud data) from a ranging sensor.

[0142] For example, in the case of the method using the machine learning model described above, it is difficult to create a three-dimensional model from an image without feature points, but the accuracy of the three-dimensional model can be improved by adding point cloud data. An image without feature points is, for example, an image of a row of doors of the same shape, color, etc. In this case, there is no difference in the image elements (doors), and multiple images cannot be joined together well.

[0143] Furthermore, the imaging data used for the first three-dimensional model may be a moving image, that is, a series of images that are continuous in time series. In other words, a technique for constructing a three-dimensional model from a moving image may be used.

[0144] The second three-dimensional model is based on point cloud data acquired by a sensor such as a distance measuring sensor or three-dimensional data of this point cloud data, and imaging data from an imaging device. Examples of the three-dimensional data include mesh data, surfel data, and surface data (geometry data). The three-dimensional data is sometimes called coordinate data because it stores coordinates of a three-dimensional space.

[0145] In this embodiment, the second three-dimensional model includes a three-dimensional model related to a space (room) (three-dimensional space model) and a three-dimensional model related to an object (three-dimensional object model).

[0146] Below, we will explain a method of acquiring a three-dimensional model (for reproducing tourist facilities and the like in a virtual space) using a device (such as an HMD) that has both an imaging device and a distance measuring sensor. As described above, a three-dimensional model is constructed from distance data (point cloud data) obtained by a distance measuring sensor and imaging data obtained by an imaging device such as a camera.

[0147] An example of a device for acquiring such a three-dimensional model is Meta Quest (META and META QUEST are registered trademarks).

[0148] Hereinafter, a device that acquires distance data (point cloud data) and imaging data will be referred to as a "data acquisition device." The data acquisition device is a type of computer terminal, and includes not only input sections such as a distance measuring sensor and an image capturing device, but also output sections such as a control section (processor), a memory section, and a display section.

[0149] An entity that acquires distance data (point cloud data) and imaging data is hereinafter referred to as a "data acquirer." The data acquirer acquires data on tourist facilities, etc. using a data acquisition device.

[0150] More specifically, a data collector moves around a tourist facility or the like carrying (or wearing) a data collection device. For example, if the data collector points the data acquisition device north inside a tourist facility, the data acquisition device will acquire and store data on the north side of the tourist facility, and if the data acquisition device points south, the data acquisition device will acquire and store data on the south side of the tourist facility. Similarly, if the data acquisition device is pointed upwards, the processor of the data acquisition device will acquire data on the ceiling, and if pointed downwards, the processor of the data acquisition device will acquire data on the floor, and store the data in the memory unit of the device.

[0151] Here, the presence of walls and other objects limits the range that a distance measuring sensor can sense (for example, the range that an infrared sensor's infrared light can reach), so the data collector moves around the space to collect distance data (point cloud data) and image data for the entire tourist facility.

[0152] Taking an HMD as an example, the display unit of the HMD displays data elements (points, frames, meshes, polygons, etc.) in the pass-through image and indicates the areas where data has been acquired. In other words, the data acquirer can check whether the data has been acquired sufficiently while acquiring the data.

[0153] In addition, if distance data (point cloud data) of, for example, the boundary between a wall and a ceiling, the boundary between a wall and a floor, or the corners of a room is recognized incorrectly, the data acquirer can correct it while watching the pass-through video. Detailed explanation will be omitted.

[0154] At this stage, only data on the space (walls, ceiling, floor) has been acquired. Depending on the data acquisition device, data of the object in space may also be acquired to obtain an independent three-dimensional object model.

[0155] Here, the acquisition of a three-dimensional article model will be described. For example, suppose there is only one statue (roughly a rectangular parallelepiped) as an object near a wall in a (rectangular parallelepiped) room. A data acquirer can operate a controller (attached to a data acquisition device) to create a rectangular parallelepiped that includes the statue and set it as an object.

[0156] After the above settings, the data acquirer can determine the name of the item, and the data acquisition device acquires that information. The processor of the data acquisition device stores the various acquired data (distance data (point cloud data), imaging data, etc.) in a storage unit.

[0157] In addition, a processor in the data acquisition device may create a three-dimensional model from these data, or a computer other than the data acquisition device may construct the three-dimensional model from these data.

[0158] In this embodiment, the various data acquired by the data acquisition device (distance data (point cloud data), imaging data, and three-dimensional (object) models, etc.) are acquired by a three-dimensional model creation server (a server equipped with a machine learning model for creating three-dimensional models) separate from server 10, and stored in the memory unit of that server. The server 10 acquires the three-dimensional model that is being constructed by the three-dimensional model construction server.

[0159] Returning to the explanation of the sightseeing content creation process. The processor 122 creates tourist content by associating text information, image information, or audio information with three-dimensional models of tourist facilities and the like.

[0160] Specifically, it is a process of associating corresponding information (text information, image information, audio information, or a combination of these (such as video with audio)) with a specific position on a three-dimensional model (such as a tour tag) represented by coordinates in virtual space.

[0161] As described in the section on the UI, when a participant in the virtual space touches or selects the information (content), the processor 122 provides the information to the participant terminal.

[0162] More specifically, when a participant object (avatar) in the virtual space approaches (comes within a certain distance from) the point where the information is set, and the participant performs a specified operation (clicking, pointing, etc.) at that point, processor 122 provides (sends) the set information to the participant terminal. The above-mentioned predetermined operation in this embodiment is the action operation described in the UI section.

[0163] The above-mentioned service provider setting process and sightseeing content creation process are processes that are preparations for providing a sightseeing content provision service (tour).

[0164] <2-3. Tourism content provision processing> In the tourism content provision process, processor 122 provides the tourism content created in the tourism content creation process described above to participants, etc.

[0165] Tourism content provision process: An introduction content provision process for providing introduction content (to participants); Participation application acceptance processing to accept applications from participants for services (tours) that provide tourism content; A guide necessity determination process for determining whether or not a guide is necessary (upon receipt of the participant's application); A participation fee calculation process for calculating the participation fee for the participant; Participation fee modification process to change the participation fee when a guide is unable to attend despite the participant's request for a guide; A participation fee payment confirmation process to confirm the payment of the participation fee; an access authority granting process for granting access authority to the tourism content to the participant; a service publishing and providing process for accepting access to the tourism content by the participants and providing the service (a service (tour) for providing the tourism content); a participant viewpoint providing process for arranging a participant object corresponding to the participant in the virtual space and displaying an image from a virtual camera of the participant object on a display unit of a participant terminal; a guide viewpoint providing process for arranging a guide object corresponding to the guide in the virtual space and displaying an image captured by a virtual camera of the guide object on a display unit of a guide terminal; a three-dimensional image changing process for changing an image displayed on a display unit of the participant terminal by an operation of the participant terminal by the participant or an operation of the guide terminal by the guide; It is equipped with an allocation calculation process that calculates at least the allocation to the right holder from the participation fee (or the revenue included in the participation fee). Moreover, the access authority granting unit granting the access authority to the participants and the guide when a guide is required; When a guide is not required, the above access rights are granted to participants (only).

[0166] FIG. 7 is a flowchart showing the sightseeing content provision process. It should be noted that this flowchart is for illustrative purposes only, and the process relating to the participation application, the process relating to the receipt of payment, the process relating to the provision of services, and the process relating to the distribution of profits may each be performed separately. Also, although the following flow chart focuses on one particular participant, the server 10 processes participation applications from multiple participants simultaneously.

[0167] The processor 122 starts the sightseeing content provision process by accepting a notification of an application for participation from a participant (a person wishing to participate).

[0168] Processor 122 accepts an application from a participant (participation application acceptance process) and obtains participant information (step 21).

[0169] Although not shown in FIG. 7, in this embodiment, the processor 122 provides (free) introduction content to participants (those who wish to participate) (hereinafter referred to as "introduction content providing process").

[0170] That is, the processor 122 accepts access (to a URL corresponding to the introduction content) from a participant and provides the introduction content.

[0171] For example, after a guide creates content, he or she can post the URL of the content on their own media, such as social media, and promote the tourist content. If participants like the content, they can proceed to apply for participation from the introduction content.

[0172] Then, the processor 122 determines whether or not the participant needs a guide (necessity of a guide ("necessary" or "not necessary")) from the information input by the participant (step 22: guide necessity determination process).

[0173] Processor 122 may store this necessity determination (a value corresponding to necessary or unnecessary) in storage unit 14, and use the information for various determinations, described later, such as cost calculation and granting of access authority.

[0174] If the participant requires a guide (guide required), the processor 122 obtains information on a guide registered with the tourism content provision service (tour) from the storage unit 14 (step 23). Processor 122 then selects an appropriate guide and sends a notification to that guide (step 24).

[0175] In this embodiment, the processor 122 sends a notification to all registered guides, and from among those who respond within a predetermined period, selects the guide using a predetermined algorithm. The predetermined algorithm is, for example, an algorithm that determines the guide based on the fastest response, the highest evaluation (review) from participants, the highest response rate to notifications, or a combination of these.

[0176] However, the selection of the guide is not limited to this. For example, the guides may be assigned in order of registration, and the notification may be sent only to the guide who has been assigned the assigned guide.

[0177] In addition, if the guide himself creates the tourist content, the creator becomes the guide. However, in this case, the guide who created the travel content may recommend and select other guides.

[0178] The processor 122 determines whether the necessary guide has been determined (step 25), and if the participants have been determined (step 25--guide determination), the process proceeds to the next step. If the participant does not require a guide (Step 22: No guide required), proceed to the next step.

[0179] On the other hand, if the guide has not been decided (step 25--guide not decided), for example, if there is no reply from the guide, the processor 122 performs the process of step 23 again.

[0180] Next, the processor 122 calculates and sets the final participation fee (step 26: participation fee calculation process). As described above, the participation fee varies depending on whether a guide is provided, whether a third party is participating, or whether the application is an individual or group application, so the processor 122 obtains the application details of the participant and calculates the participation fee.

[0181] The individual costs constituting the participation fee, such as when a guide is required or not, are stored in advance (set by an administrator, etc.) in the storage unit 124.

[0182] In addition, if circumstances arise such as the guide being unable to participate, the processor 122 adjusts (downward) the participation fee. That is, in step 26, the processor 122 changes the participation fee in particular in the case where the guide is unable to attend despite the participant's request for a guide (participation fee correction process).

[0183] The processor 122 notifies the participant of the final participation fee and accepts notification of payment (step 27 - participation fee payment confirmation process). To explain this according to the flowchart, the processor 122 waits until it receives a notification of a deposit (step 27--no deposit), and if it receives a notification of a deposit (step 27--deposit), it proceeds to the next step. However, this flowchart is for explanatory purposes only, and the process relating to deposits may be performed separately.

[0184] The processor 122 grants the participants and guides access rights to the sightseeing content (services) (processor 28). As described above, the process of granting participants access rights to the sightseeing content is referred to as the "access right granting process."

[0185] In this case, in the access authority granting process, the processor 122: If a guide is required (step 22: guide required), grant the above access rights to the participants and the guide; If a guide is not required (step 22: guide not required), the above access rights are granted to participants (only).

[0186] If a participant requires a guide, the guide must also have access to the tourist content in order to participate; however, if a participant does not require a guide, there is no need to grant the guide access rights.

[0187] The processor 122 also publishes the tour (i.e., makes the sightseeing content accessible to participants, etc.) at a predetermined start date and time or before that date (such as a few minutes before), and provides the service (step 29). As described above, the process of accepting access to sightseeing content from participants and providing services is referred to as the "service disclosure and provision process."

[0188] When providing a service, the processor 122 grants the participant the right to operate a participant object, and the guide the right to operate a guide object.

[0189] The processor 122 also displays the image from the virtual camera of the participant object on the display unit of the participant terminal, and displays the image from the virtual camera of the guide object on the display unit of the guide terminal (participant viewpoint providing process / guide viewpoint providing process).

[0190] Participants operate avatars while viewing images from objects corresponding to them. That is, in response to an operation of a terminal by a participant or the like, the processor 122 changes the image displayed on the display unit of the terminal (three-dimensional image change processing).

[0191] In addition, in certain cases, such as the guide mode described in the UI section, the processor 122 temporarily suspends the operation authority of the participants, allowing only the guide to operate.

[0192] In this embodiment, the guide's viewpoint is shared with the participants. That is, the processor 122 displays an image from the viewpoint of the guide object on the participant terminal.

[0193] In other words, the image displayed on the display unit of the participant terminal is changed by the guide operating the guide terminal (three-dimensional image change process).

[0194] After the service provision time has ended, the processor 122 terminates the operating authority of the participants, etc., and also cuts off access to the participant terminals and the guide terminal.

[0195] After the provision of the service is completed, the processor 122 distributes the profits. That is, the processor 122 calculates the distribution to the service provider, at least the right holder, from the participation fee (profits included in the participation fee) (distribution calculation process).

[0196] In this embodiment, a travel agency is also involved, so the processor 122 calculates at least the allocation to the rights holder and the allocation to the travel agency from (the revenue included in) the participation fee.

[0197] In this embodiment, the rights holder, the guide, and the travel agency are involved, so the processor 12 calculates the distribution of revenue to these parties. For example, the processor 122 calculates according to a predetermined formula that 40% of the revenue after deducting expenses from the participation fee should be allocated to the rights holder, 30% to the guide, 20% to the travel agency, and the remaining 10% to the administrator.

[0198] As another example, if no travel agency is involved, the processor 122 calculates an allocation of 50% to the rights holder, 40% to the guide, and the remaining 10% to the administrator, As another example, when neither a guide nor a travel agency is involved, the processor 122 calculates that 80% will be allocated to the right holder and the remaining 20% ​​will be allocated to the manager. These are merely examples, and the allocation ratios can be changed as appropriate.

[0199] The details will be explained below in more detail. In the participation application acceptance process, when the participants are the above-mentioned "group," processor 122 accepts applications from multiple participants through an application by one participant (hereinafter referred to as the "representative") (step 21). In this case, for one application, the processor 122 grants a plurality of access rights according to the number of participants (step 28). In addition, there is only one URL (the URL accessed by each participant). At this time, the processor 122 unifies within the group whether a guide is required and whether a third party is to participate. The processor 122 then creates an avatar (participant object) for each participant, and the participants can operate their own avatars in the virtual space by inputting operations from their participant terminals.

[0200] That is, the processor 122 accepts the participation of multiple participants through one application (collectively accepting applications). Processor 122 also grants access rights to the sightseeing content to each of the multiple participants (hereinafter referred to as "collective access rights granting process").

[0201] Furthermore, the processor 122 varies the content of the service to be provided (step 29) depending on the content of the participation application.

[0202] That is, processor 122 accepts (in this embodiment, at the participation application stage (step 21)) the participant's selection of manual mode (referred to as the "first action mode") or auto mode (referred to as the "second action mode") (referred to as the "action mode acceptance process").

[0203] In addition, the processor 122 performs the following steps when providing a service (a service (tour) that provides sightseeing content) (step 29): In the case of the manual mode (first behavior mode), the participant object is controlled in response to the participant's input operation (e.g., movement, viewpoint movement, content provision, etc.); In the case of the auto mode (second behavior mode), regardless of the participant's input operation, the participant object is controlled in a manner determined in advance by the administrator or the guide, or by the operation of the administrator or the guide (referred to as "service switching processing by behavior mode").

[0204] Similarly, processor 122 (in this embodiment, at the participation application stage (step 21)) accepts the participant's selection of whether or not a third party is present (this will be referred to as "third party presence / absence acceptance process").

[0205] At this time, the processor 122 proposes a service (a service / tour that provides tourist content) according to the selection of the participant (in this embodiment, at the stage of application for participation (step 21)) (this is referred to as "proposal processing based on third-party selection"). In other words, if a participant does not wish for a third party to participate, a tourism content provision service (tour) without a third party is displayed as a suggestion, If the participant wishes for a third party to participate, a tourism content provision service (tour) in which the third party also participates is displayed as a proposal.

[0206] In addition, there may be an option to select "either" regarding the participation of a third party, and if the participant selects this option, both a tour without a third party and a tour with a third party will be proposed.

[0207] In addition, the processor 122 controls the above-mentioned "escort mode" when the service is provided. That is, in response to a specified operation by the guide (such as pressing the guide mode icon (not shown)), processor 122 (temporarily) suspends the participants' authority to operate the participant object and grants that authority to the guide (referred to as the "operation authority transfer process").

[0208] At this time, the processor 122 may grant the operation authority of one or more participants selected by the guide to the guide individually, or may grant the operation authority of all the participants to the guide collectively. Furthermore, the participant object may be integrated with the guide object, or may be independent of the guide object. In this embodiment, the participant objects of all participants are integrated into the guide object, meaning that the position, viewpoint, etc. of the guide object are shared by all participants.

[0209] In addition, when there is an object such as a wall, floor, or ceiling between two or more different participant objects or guide objects in the virtual space, the processor 122 reduces the exchange of audio information between the two or more objects ("audio obstacle processing"). In the case of participant A and participant B, whether or not participant A can hear the sound from the audio input section of participant B (output from the audio output section of participant A) is attenuated according to the distance. In other words, if the distance between the objects is close, one can clearly hear the other's sound. On the other hand, audio obstacle processing is a process that, for example, when there is an object such as a wall between participant A and participant B, makes the audio from participant B's audio input section inaudible or faintly audible to participant A (suppresses the output from participant A's audio output section).

[0210] <2-4. Quiz question processing> In the quiz question setting process, the processor 122 sets quiz questions to participants in the virtual space.

[0211] The quiz question setting process includes a quiz setting process for accepting settings for quizzes related to tourist facilities, etc. a quiz-asking process for asking the quiz to participants; a quiz answer acquisition process for accepting answers to the quiz from participants; A quiz question correctness determination process for determining whether the answer is correct or incorrect; and and a quiz result display process for displaying the result of the judgment of whether the quiz is correct or incorrect.

[0212] Quiz questions are associated with parts of the sightseeing content, such as tour tags. In this embodiment, when a participant has turned on the quiz enable / disable icon, the participant avatar is in a place where a quiz question is set, and the participant performs a predetermined operation, the processor 122 asks the participant a quiz (see the quiz display screen in the UI section). The quiz questions are obtained from the quiz data table D18.

[0213] Processor 122 asks a quiz to the participant and waits for a certain period of time to accept an answer to the quiz. When the answer of the participant is acquired, processor 122 judges whether the answer is correct or incorrect by referring to quiz data table D18 or the like, and displays the result of the judgment.

[0214] In addition, the processor 122 stores the user's score for each participant (the number of correct answers to the quiz, or a score determined for each quiz (eg, a score according to the difficulty level of the quiz)). The processor 122 can provide services (such as providing discount tickets) according to the participant's points.

[0215] With the above configuration, the online tour providing system 1 of this embodiment holds and stores guide information. It then acquires the user's application details to determine whether a guide is necessary or not, and if necessary, selects an appropriate guide from the guides stored in the storage unit and acquires information such as whether the guide can participate. It also determines the participation fee depending on whether a guide is necessary or not, and distributes the profits to those involved in providing the service. This allows guides to be secured for each tour, and because guides can participate online, they can work even if they are in a remote location. Furthermore, rights holders of tourist facilities, guides, travel agencies, etc. can earn a stable income online with peace of mind.

[0216] 3. Data The data handled by the online tour providing system 1 of this embodiment will be described below with reference to tables. The online tour providing system 1 of the present embodiment includes an online tour providing database D1 in the memory unit 14 (data storage unit 14b) of the server 10. The online tour provider database D1 includes a rights holder data table D11, a tourist facility data table D12, a guide data table D13, a participant data table D14, a travel agency data table D15, a tour data table D16, a tour tag data table D17, and a quiz data table D18.

[0217] Specific examples are given below, but the data table configurations and data items included in the data tables are merely examples and are not limited to the following examples.

[0218] The right holder data table D11 is a data table that includes data relating to the right holder (right holder data).

[0219] [Table 1]

[0220] Table 1 shows an example of data contained in the right holder data table D11. As shown in Table 1, the rights holder data table D11 includes the following items: a unique rights holder ID, a rights holder name which corresponds one-to-one to the rights holder ID, the name of the tourist facility etc. over which the rights holder holds rights, the rights holder's address, the rights holder's contact information (email address, telephone number, etc.), the transfer destination of the rights holder's revenues (bank name, branch name, account type, account number, etc.), and the payment date. Examples of contact details and transfer details are similar and will not be repeated below.

[0221] The tourist facility data table D12 is a data table including data on tourist facilities (tourist facility data).

[0222] [Table 2]

[0223] Table 2 shows an example of data stored in the tourist facility data table D12. As shown in Table 2, the tourist facility data table D12 includes the following items: a unique tourist facility ID, a name of the tourist facility that corresponds one-to-one to the tourist facility ID, a rights holder ID of the rights holder related to the tourist facility, and the address of the tourist facility.

[0224] The guide data table D13 is a data table that includes data relating to guides (guide data).

[0225] [Table 3]

[0226] Table 3 shows an example of data contained in the guide data table D13. As shown in Table 3, the guide data table D13 has the following items: a unique guide ID, a guide name that corresponds one-to-one to the guide ID, foreign languages ​​that the guide can speak, the names of tourist facilities etc. that the guide can provide information on (are registered with), the address or residence of the guide, contact information for the guide, the bank details for the guide, and the date of payment to the guide. In Table 3, names and titles are represented by the letters A to G.

[0227] The corresponding foreign language may be a language that can be used. In other words, since the guide is not necessarily a Japanese national, any language that can be used, including Japanese, may be acquired as information. It may also include language features such as dialects.

[0228] The participant data table D14 is a data table that includes data on participants (participant data).

[0229] [Table 4]

[0230] Table 4 shows an example of data contained in the participant data table D14. As shown in Table 4, participant data table D14 includes items such as a unique participant ID, participant name that corresponds one-to-one to the participant ID, personal information other than the participant name (age (or generation), gender, address, contact information, etc.), points, type of participation (individual or group), and information on tours participated in the past.

[0231] The points are given by the online tour providing system 1. There may be a plurality of items related to points, and may include, for example, points (scores) from quizzes and points having monetary value.

[0232] In addition to the above, the participant data table D14 may also include data on the participant's behavior, such as purchasing data (at a shop in the processing space, etc.), data on options that the participant added to the tour, data on the route taken for each tour, data on comments left on tour tags, etc. (text data), image and audio data (or IDs associated with these), or ratings and comments on a particular tour. By obtaining this data, the processor 122 can analyze the tastes and preferences of the participants and provide services that the participants are likely to prefer.

[0233] The travel agency data table D15 is a data table that includes data related to travel agencies (travel agency data).

[0234] [Table 5]

[0235] Table 5 shows an example of data contained in the travel agency data table D15. As shown in Table 5, the rights holder data table D15 includes the following items: a unique travel agency ID, a travel agency name which corresponds one-to-one to the travel agency ID, the names of tourist facilities etc. for which the travel agency has tours (registered), the address of the travel agency, contact information for the travel agency, the bank details of the travel agency, and the payment date.

[0236] The tour data table D16 is a data table that includes data (tour data) relating to (individual) tours.

[0237] [Table 6]

[0238] Table 6 shows an example of data contained in the tour data table D16. As shown in Table 6, the tour data table D16 has the following items: a unique tour ID, a tour destination that corresponds one-to-one to the tour ID, the ID of tourist facilities etc. that the tour will visit, the tour start date and time, the tour end date and time, the participation fee, the recruitment start date and time, the recruitment end date and time, the number of participants, the capacity (maximum number of participants), the minimum number of participants for the tour to be held, the status, and additional services.

[0239] The participation fee is the total cost paid by the participant, however, the tour data table D16 may also include other costs that make up the participation fee, such as the cost with a guide, the cost without a guide, the cost per person when the participant is an individual, the cost per person when the participant is a group, the cost when a third party participates, or the cost when no third party participates, etc.

[0240] Tour destinations and tourist facility IDs do not necessarily correspond one-to-one, and one tour may include multiple tourist facilities.

[0241] The status indicates whether the tour is “not yet open for booking”, “open for booking”, “ongoing”, or “completed.” The processor 122 automatically changes the description based on the date and time of the fire, the date and time of booking, and the current time obtained from an NTP server or the like.

[0242] The additional services section describes the services that are additional to the tour.

[0243] In addition to the above, the tour data table D16 may also hold information about participants and guides, such as participant ID, guide ID, information about participant payments (whether the participation fee has been paid or not), and actual participation or non-participation. Furthermore, the tour data table D16 may include information about the tour other than those mentioned above, such as data items such as the creator of the tour, the date and time the tour was created, and the expiration date (period of operation) of the tour. The tour data table D16 may also include data items such as information about the type of tour (such as the type of tour, such as whether or not a guide is present as mentioned above).

[0244] The tour tag data table D17 is a data table that includes data relating to tour tags (tour tag data).

[0245] [Table 7]

[0246] Table 7 shows an example of data contained in the tour tag data table D17. As shown in Table 7, the tour tag data table D17 has the following items: a unique location ID (tour tag ID), a location name (tour tag name) that corresponds one-to-one to the location ID, the tourist facility ID of the tourist facility etc. that includes the tour tag and the name of the tourist facility etc., and the ID, type, and file name of the content associated with the location ID. In addition, the tour tag data table D17 may include coordinate data in a virtual space.

[0247] As for the contents, a separate database may be provided in the form of a contents data table having a unique contents ID.

[0248] The quiz data table D18 is a data table that includes data related to quiz questions.

[0249] [Table 8]

[0250] Table 8 shows an example of data contained in the quiz data table D18. As shown in Table 8, the quiz data table D18 has, as items, a unique quiz ID, a location ID of the location where the quiz is set, quiz questions and options corresponding one-to-one to the quiz ID, correct answers, and difficulty levels of the quiz questions.

[0251] In addition to the above, the online tour providing database D1 includes data related to coordinates in the virtual space. For example, it is a data table that stores a set of coordinates in a virtual space for a certain sightseeing content and the above-mentioned "memory memo," that is, text information, etc.

[0252] Furthermore, as data related to the coordinates, the movement paths of the participants and guides in the virtual space may be stored. In other words, the processor 122 may store the paths traveled by the avatars of the participants, etc., so that the movement paths can be reproduced as lines, etc.

[0253] When participants are moving freely in the virtual space, many of them will likely stop by popular spots. By acquiring such information, it is possible to accumulate information on popular spots in a given tour. Improvements can be made by adding more content to popular spots, and removing content or tour tags from unpopular spots.

[0254] 4. Hardware Configuration As shown in Fig. 1, the online tour providing system 1 in this embodiment includes a server 10, a participant terminal 20, a guide terminal 30, a travel agency terminal 40, and a right holder terminal 50. These devices are connected to each other via a network N. The network N is, for example, the Internet. The server 10 includes an online tour providing program P1, and software (application software) for operating the online tour providing system 1 according to the present embodiment is installed on the server 10, and various processes are executed by the functions of the software. Each piece of hardware will be described below.

[0255] <Server 10> The server 10 is a computer (information processing device) for executing an online tour providing program P1. Although only one server 10 is shown in FIG. 1, the number is not limited to one, and may be realized by a plurality of servers. For example, from the standpoint of load balancing and availability, it is possible to use multiple servers. The server 10 may be a computer of a cloud service provider, or a computer provided by the user. In particular, in this embodiment, the server that generates the three-dimensional model and the server that provides the tourist content are separate.

[0256] FIG. 8 is a diagram showing the hardware configuration of the server 10. As shown in FIG. 8, the server 10 includes a control unit 12, a storage unit 14, and a communication control unit 16. The control unit 12 also includes a processor 122, a ROM 124, a RAM 126, and a clock unit 128. The basic functions of each will be explained later.

[0257] The control unit 12 including the processor 122 also functions as an online tour providing unit (not shown) in the server 10. The online tour providing unit executes the online tour providing program P1 to perform online tour providing processing. In this embodiment, the processor 122 is a CPU (Central Processing Unit).

[0258] Moreover, one program may include another program. For example, in this embodiment, the online tour providing program P1 includes a service provider setting program P11, a tourism content creating program P12, and a tourism content providing program P13.

[0259] As shown in FIG. 8, the storage unit 14 includes a program storage unit 14a and a data storage unit 14b, and stores programs and data necessary for various processes. For example, the program storage unit 14a stores, in addition to the online tour providing program P1 of this embodiment, a control program for controlling devices connected to the server 10, such as a communication control program for controlling the communication control unit 16.

[0260] 8, the communication control unit 16 is a device that performs communication between the server 10 and an external terminal, for example, a terminal (such as a participant terminal 20) described later. The communication control unit 16 connects the server 10 to a network N, as shown in FIG.

[0261] In addition to the above, the server 10 may also include an input unit (e.g., a keyboard) for inputting commands and data, an output unit (e.g., a voice output device) for outputting information in some form, etc. (not shown). The server 10 may also include devices additionally required for the use of this embodiment, or devices for improving the convenience of the use of this embodiment.

[0262] <Data acquisition device> Here, we will explain the data acquisition device, taking an HMD as an example. Like a general computer, the data acquisition device includes a control unit including a processor, a storage unit, a communication control unit, an input unit, and an output unit.

[0263] The communication control unit is a device that performs communication between the data acquisition device and an external terminal. Furthermore, the communication control unit allows the data acquisition device and the controller corresponding to the data acquisition device to perform wireless communication. For example, the data acquisition device and the controller communicate with each other using Bluetooth (registered trademark). In addition, the communication control unit is also used when the data acquisition device communicates with a network such as the Internet.

[0264] The input section of the data acquisition device includes a sensor and an imaging device. The data acquisition device includes a distance measurement sensor as a sensor. The distance measurement sensor is a sensor for acquiring point cloud data for reproducing the interior of a building, etc. In this embodiment, the distance measurement sensor is an infrared depth sensor. However, the distance measurement sensor is not limited to this, and may be a millimeter wave radar or LiDAR (Light Detection and Ranging), etc.

[0265] Here, the data acquisition device includes a device for detecting the position, orientation, and tilt. Such a device may be, for example, an inertial measurement unit (IMU) using an acceleration sensor, a gyro sensor, or a geomagnetic sensor. However, the device is not limited to these, and may further include a detection device using an infrared LED and an infrared camera, a visible LED and a visible camera, or an infrared laser emitter. A device that realizes such a function may belong to the input section or the output section depending on its operation.

[0266] In addition to the above, the data acquisition device may include environmental sensors such as a temperature sensor for measuring the surrounding temperature and a humidity sensor for measuring humidity.

[0267] The imaging device is a camera for capturing images of the surroundings of the data acquisition device. The imaging device captures images to be overlaid on point cloud data captured by a distance measuring sensor or on a three-dimensional representation of the point cloud data. The image capture device also realizes a hand tracking function, that is, the image capture device recognizes the user's hands, and the processor 111 reflects the positions of the hands of the avatar in the MR space. Additionally, the imaging device may be used to track the HMD (inside-out method).

[0268] In addition to the above, in the case of an HMD, the input section is equipped with sensors and imaging devices that are equipped in known head-mounted displays, such as an eye-photographing camera (near-infrared camera) to realize an eye tracking function, and a sensor to realize a function of capturing head and neck movements (head tracking function).

[0269] The output unit of the data acquisition device includes a display unit and a speaker (audio output device). The display unit is, for example, a display provided in the HMD. For example, the display unit displays an image for confirming whether data is being properly acquired when acquiring data. In addition, the display unit displays a pass-through image, a pointer, information related to settings, etc.

[0270] In addition to the above, the data acquisition device may include a voice input unit such as a microphone, an input unit (e.g., a button) for inputting commands and data, an output unit (e.g., a vibration device) for outputting information in some form, and the like. It may also include an input / output interface for connecting various devices. In addition, it may include devices additionally required for the use of this embodiment and devices for improving the convenience of the use of this embodiment.

[0271] <Terminal (Participant Terminal 20)> The participant terminal 20 is a computer that enables the participant to use the online tour providing system 1. The participant uses the participant terminal 20 to access the server 10 and thereby uses the online tour providing system 1. Fig. 9 is a hardware configuration diagram of a terminal (participant terminal 20). As shown in Fig. 9, the participant terminal 20 includes a control unit 22, a storage unit 24, a communication control unit 26, and an input / output unit 28. The control unit 22 also includes a processor 222, a ROM, a RAM, and a timer unit. Explanations of contents that overlap with those already explained and of basic functions that will be described later will be omitted.

[0272] In this embodiment, the participant terminal 20 is a desktop PC or an HMD. However, the participant terminal 20 is not limited to this, and may be a mobile terminal such as a smartphone or a tablet. Additionally, a device that is worn by a user, such as an HMD, is called a "wearable device."

[0273] 9, when the participant terminal 20 is an HMD, the participant terminal 20 includes a head-mounted device 20a and a controller 20b, each of which includes a control unit, a storage unit, a communication control unit, an input unit, and an output unit. The head-mounted device 20a and the controller 20b communicate with each other using the functions of their respective communication control units.

[0274] The input section of the head worn device 20a includes sensors (such as a distance measuring sensor, an acceleration sensor, or a gyro sensor) and an imaging device, and the output section includes a display section and a speaker (usually disposed in front of the wearer's eyes). The input section of the controller 20b includes a sensor (such as an acceleration sensor or a gyro sensor) and buttons (for inputting operations), and the output section includes a vibration device (linked to a displayed image, etc.). As such an HMD, a commercially available one can be used as appropriate.

[0275] The processor 122 determines whether the display unit of the terminal (participant terminal 20 or guide terminal 30) is a single display unit such as a PC monitor, or a display unit that displays separate images for the wearer's right and left eyes, such as an HMD (this is referred to as the "display unit determination process"). If the terminal (participant terminal 20 or guide terminal 30) is a terminal that allows separate displays for the right and left eyes, such as a wearable terminal such as an HMD, the processor 122 provides a display for the right eye and a display for the left eye (of the wearer (participant or guide)) to the display unit of the terminal (referred to as "content provision processing for wearable terminals"). HMDs and other devices provide a high level of immersion by displaying separate images to the wearer's right and left eyes, artificially creating binocular disparity.

[0276] The processor 122 also acquires input information from the controller 20b, which functions as an input device for the wearable terminal (operated by participants, etc.), and changes objects (participant objects or guide objects) within the sightseeing content (referred to as "object change processing").

[0277] In this embodiment, the participant terminal 20 does not need to install a specific application program, and participants can perform various processes by accessing the website provided by the administrator, i.e., they can use the online tour providing system 1.

[0278] The various processes include output (screen display, audio output) based on information acquired from the server 10, reception of user input, and various communications.

[0279] The guide terminal 30, the travel agency terminal 40, and the right holder terminal 50 have the same hardware configuration as the participant terminal 20, so a description thereof will be omitted. The guide terminal 30 is a desktop PC, an HMD, a portable terminal, or the like, and the travel agency terminal 40 and the right holder terminal 50 are each a desktop PC or a portable terminal, or the like. That is, the participant terminal 20 and the guide terminal 30 that access the virtual space of the service that provides tourist content may use an HMD or the like that provides a strong sense of immersion and realism.

[0280] (Explanation of the basic functions of a computer) The control unit (processor, ROM, RAM, and timer unit), storage unit, communication control unit, input unit, and output unit will be described below. In any of the terminals of this embodiment, the connection form (network topology) between the functional units is not particularly limited, and may be, for example, a bus type, a star type, a mesh type, or the like.

[0281] The processor processes information and controls various devices according to programs stored in a ROM, a storage unit, etc. In this embodiment, the processor is a CPU (Central Processing Unit).

[0282] However, the processor is not limited to a CPU. Various processors, such as a CPU, a digital signal unit (DSP), a graphics processing unit (GPU), a general purpose computing on a GPU (GPGPU), a tensor processing unit (TPU), or an application specific integrated circuit (ASIC), may be used alone or in combination. For example, a processor that integrates a CPU and a GPU is called an APU (Accelerated Processing Unit), and such a processor may be used.

[0283] ROM is a read-only memory that pre-stores various programs and data that the processor uses to perform various controls and calculations.

[0284] The RAM is a random access memory used by the processor as a working memory, and various areas can be allocated in the RAM to perform various processes in this embodiment.

[0285] The clock unit performs clock processing related to obtaining time information, etc. If the computer has a communication control unit, time information may be obtained from outside using NTP (Network Time Protocol).

[0286] A memory unit is a device for storing information such as programs and data. A memory unit is also called storage. The memory unit may be built-in or external.

[0287] The storage unit includes a storage medium capable of reading and writing data, and a drive for reading and writing data from and to the storage medium. Examples of storage media include internal and external types, such as HD (hard disk), CD-ROM, and flash memory. Examples of drives include HDDs (hard disk drives) and SSDs (solid state drives).

[0288] The storage unit includes a program storage unit and a data storage unit as functional units. The program storage section stores control programs for controlling various devices, such as a communication control program for controlling communication.

[0289] The communication control unit is a device for performing communication between terminals etc. The communication control unit connects the terminal equipped with the communication control unit to the network N.

[0290] The communication method of the communication control unit is a known method, and a wired method or a wireless method is applied depending on the device. For example, if the terminal is a desktop PC, both wired and wireless communication is possible, and if the terminal is a smartphone, a wireless communication method is possible.

[0291] If it is wired, a communication method defined by IEEE802.3 (e.g., bus-type or star-type wired LAN) can be preferably used, but other communication methods such as those defined by IEEE802.5 (e.g., ring-type wired LAN) may also be used.

[0292] For wireless communication, a communication method defined by IEEE802.11 (e.g., Wi-Fi) can be suitably used, but other communication methods such as IEEE802.15 (e.g., Bluetooth (registered trademark), BLE (Bluetooth (registered trademark) Low Energy), etc.), IEEE802.16 (e.g., WiMAX), ZigBee (registered trademark), 920 MHz band wireless (Wi-SUN, etc.), or communication methods defined for optical communication such as infrared communication may also be used.

[0293] The input unit and the output unit are devices that handle input and output to and from the terminal, respectively. The input unit and the output unit may be collectively referred to as an input / output unit. The input unit is a device that accepts input from a user. Examples of such an input unit include a keyboard, a mouse as a pointing device, a track pad, a tablet, and a touch panel.

[0294] When the terminal is a tablet, smartphone, or the like, and the input unit is a touch panel, the input unit is disposed on the surface of a display unit that displays images, such as a touch screen. In this case, the input unit identifies the user's touch position corresponding to various operation icons displayed on the display unit, and accepts input by the user.

[0295] The output unit is, for example, a device for outputting images, sounds, forms, and the like. Examples of the output unit include display devices such as a touch screen or a display (a liquid crystal display or an organic EL display), audio output devices such as a speaker, and form output devices such as a printer.

[0296] (Modification) The present invention is not limited to the above-described embodiment, and includes various modifications to the above-described embodiment without departing from the spirit of the present invention. For example, in the "guiding mode" of the above embodiment, the view of the guide is shared with the participants, but this is not limited to the above. For example, the avatars of the participants are located near the guide, but the participants may freely move the camera direction (viewpoint) of the participant object (avatar).

[0297] In the auto mode of the above-described embodiment, the user's location in the virtual space is linked to the guide video, but this is not limiting. In other words, the participant may be allowed to move freely within the virtual space. In this case, for example, the processor 122 may automatically play a guide video when the participant passes a certain point in the virtual space.

[0298] In the above embodiment, it is described that profits are calculated and allocated from the participation fee. Here, "profits are allocated from the participation fee" does not mean that the expenses related to the allocation must be withdrawn from the deposit related to the participation fee, but rather the expenses related to the allocation may be withdrawn from money pooled in an account, or the expenses related to the allocation may be withdrawn from an account (such as a withdrawal account) other than the deposit account. In addition, the amount may be paid via one or more accounts. In other words, it is sufficient that the payment is accepted and the profits are allocated using a method of money transfer generally accepted in society.

[0299] Furthermore, in the above embodiment, the profits are allocated for each tour, but the profits do not have to be allocated for each tour. The profits obtained from multiple tours may be aggregated and then allocated. Furthermore, the system is not limited to a commission-based system, and may be a flat-rate system that is not dependent on the number of tours. The timing of revenue distribution, whether it is a commission system or a flat rate system, etc. may differ depending on the rights holder, the guide, and the travel agency. Also, for example, distribution to rights holders and guides may be a fixed amount that is not dependent on the number of tours.

[0300] Although only briefly mentioned in the above embodiment, monetary transactions such as payments and revenue distribution may be conducted through, for example, a bank, a credit card company, or a fee collection agency.

[0301] In the above embodiment, the URL corresponding to the sightseeing content and the URL corresponding to the introduction content are different, but they may be the same. In this case, the processor 122 grants the participant access authority limited by time. For example, the participant can enjoy the same sightseeing content for free for the first five minutes.

[0302] In the guide setting process of the above-described embodiment, when a guide sets an assistant guide, the processor 122 displays the assistant guide (separate from the guide) in the guide display unit UI-13 (not shown). This allows the guide to talk with the assistant guide in the virtual space (talk mode). The assistant guide is, for example, an expert (such as a curator or a university professor) who is familiar with the tourist attractions on the tour. The assistant guide is sometimes called a buddy.

[0303] In other words, when an assistant guide is set in the guide setting process, the processor 122 displays an image of the assistant guide on the display unit (guide display unit) of the participant terminal and the guide terminal (this is referred to as the "assistant guide additional display process").

[0304] In other words, aspects of the present invention including this embodiment have the following features. The following corresponds to the claims at the time of filing of this application. However, due to amendments to the claims after filing, the claims may differ from the description of the amended claims. (1) In the first aspect, the present invention includes a right holder setting unit that sets a right holder having rights to a tourist destination or a tourist facility (hereinafter referred to as "tourist facility, etc."), a guide setting unit that sets a guide for the tourist facility, etc., a tourist content creation unit that creates tourist content by associating text information, image information, or audio information with a three-dimensional model of the tourist facility, etc. constructed from at least image data of the tourist facility, etc. taken by an imaging device, a participation application acceptance unit that accepts applications from participants for a service that provides the tourist content, a guide necessity determination unit that determines whether the guide is necessary, a participation fee calculation unit that calculates a participation fee for the participant, and a participation fee calculation unit that calculates an access fee for the participant to the tourist content. The present invention provides an online tour providing system comprising: an access authority granting unit that grants access authority; a service disclosure providing unit that accepts access to the tourist content by the participant and provides the service; a three-dimensional image changing unit that changes an image displayed on a display unit of the participant terminal by the participant operating the participant terminal or the guide operating the guide terminal; and an allocation calculation unit that calculates an allocation of the participation fee to at least the right holder, wherein the access authority granting unit grants the access authority to the participant and the guide when a guide is required, and grants the access authority to the participant when a guide is not required. (2) In a second aspect, there is provided an online tour providing system as described in the first aspect, further comprising a travel agency setting unit which sets a travel agency which will introduce the service, and an allocation calculation unit which calculates an allocation from the participation fee to at least the right holder is an allocation calculation unit which calculates an allocation from the participation fee to at least the right holder and an allocation to the travel agency. In this case, integrating with the travel agency's system will allow travel agencies to more easily apply to participate, providing an environment in which participants can easily get involved and also allowing revenue to be distributed to the travel agency. (3) In a third aspect, there is provided an online tour providing system as described in the first aspect, further comprising a viewpoint switching unit which, when the service is provided, displays an image from a virtual camera provided on another person's object on the display unit of the participant or other person who performed the operation, in response to the operation of the participant or other person. In this case, the viewpoint can be shared with others (particularly group members and guides) and the tourist content that others want you to see can be immediately known, giving the feeling of traveling together and allowing you to easily know what content the guide is explaining. The online tour providing system 1 displays avatars of participants, etc., but since it is sometimes difficult to know where someone else is pointing just by looking at their avatar, this function is effective for the above purpose. (4) In a fourth aspect, there is provided an online tour providing system as described in the first aspect, further comprising a batch application acceptance unit that accepts participation of multiple participants through a single application, and a batch access authority granting unit that grants access authority to the tourism content to each of the multiple participants. In this case, the processor 122 can accept group participation, and multiple people can share and enjoy the same sightseeing content. (5) In a fifth aspect, there is provided an online tour providing system as described in the first aspect, further comprising: a behavior mode receiving unit that receives a selection by the participant of a first behavior mode or a second behavior mode; and a behavior mode-specific service switching unit that, when the service is provided, controls the participant object in accordance with an input operation of the participant in the case of the first behavior mode, and controls the participant object in a manner previously determined by an administrator or the guide, or by the operation of the administrator or the guide, regardless of the input operation of the participant in the case of the second behavior mode. In this case, the processor 122 can accept the participant's preference and switch the processing to determine whether the participant will act freely or act in a predetermined manner when the service is provided. This allows the service to be provided according to the participant's hobbies, preferences, and economic situation, making it more user-friendly. (6) In the sixth aspect, the present invention is characterized in that the present invention further includes an operation authority transfer unit that, when the service is provided, suspends the operation authority of the participant object held by the participant through an operation by the guide and grants the operation authority to the guide. According to a first aspect, there is provided an online tour providing system. In this case, there is an advantage that the participants can be sure to see the tourist object that the guide wants them to pay attention to, and the participants can avoid the risk of missing the guide's explanation because they are not aware of the guide's location. (7) In a seventh aspect, there is provided an online tour providing system as described in the first aspect, further comprising a quiz setting unit that accepts settings for quizzes related to the tourist facilities, etc., a quiz question setting unit that asks the quizzes to the participants, a quiz answer acquisition unit that accepts answers to the quizzes from the participants, a quiz question correctness determination unit that determines whether the answers are correct, and a quiz result display unit that displays the correctness determination results. In this case, the participants can gain a deeper understanding of the tourist facilities, etc. Also, since the processor 122 can give points to the participants for the quiz, there is an advantage in that the participants can participate in the quiz more actively. This is an advantage over real sightseeing. (8) In an eighth aspect, there is provided an online tour providing system as described in the first aspect, further comprising a display unit determination unit which determines whether the display unit of the participant terminal or guide terminal is a single display unit or a display unit which provides separate displays for the right and left eyes of the wearer, and a wearable terminal content providing unit which provides a right-eye display and a left-eye display to the display unit of the participant terminal or guide terminal if the participant terminal or guide terminal is a terminal which provides separate displays for the right and left eyes. In this case, the wearable device enhances the participant's sense of immersion. (9) In a ninth aspect, there is provided an online tour providing system as described in the first aspect, further comprising a secondary guide additional display unit which displays an image of a secondary guide on the display unit of the participant terminal and the guide terminal when a secondary guide is set in the guide setting unit. In this case, there is an advantage that multiple guides can provide participants with more detailed information about tourist facilities, etc. Also, since it is an online tour, it is easy to arrange the schedule for guides, so there is an advantage that discussions between experts can be held at low cost. (10) In the tenth aspect, a computer includes a right holder setting means for setting a right holder having rights to a tourist destination or a tourist facility (hereinafter referred to as "tourist facility, etc."), a guide setting means for setting a guide for the tourist facility, etc., a tourism content creation means for creating tourism content by associating text information, image information, or audio information with a three-dimensional model of the tourist facility, etc. constructed from at least image data of the tourist facility, etc. taken by an imaging device, a participation application acceptance means for accepting an application from a participant for a service providing the tourism content, a guide necessity determination means for determining whether or not the guide is necessary, a participation fee calculation means for calculating a participation fee for the participant, and a participation fee calculation means for allowing the participant to access the tourism content. the participant and the guide when a guide is required, and the access right granting means grants the access right to the participant and the guide when a guide is required, and the access right granting means grants the access right to the participant and the guide when a guide is not required, and the access right granting means functions as a service providing means for accepting access to the tourist content by the participant and providing the service, a three-dimensional image changing means for changing an image displayed on a display unit of the participant terminal by the participant operating the participant terminal or the guide operating the guide terminal, and an allocation calculation means for calculating an allocation of the participation fee to at least the right holder, wherein the access right granting means grants the access right to the participant and the guide when a guide is required, and grants the access right to the participant when a guide is not required. (11) In the eleventh aspect, the present invention includes a right holder setting step in which a processor stores a right holder who has rights to a tourist destination or a tourist facility (hereinafter referred to as "tourist facility, etc.") in a memory unit, a guide setting step in which the processor stores a guide for the tourist facility, etc. in the memory unit, a tourism content creation step in which the processor creates tourism content by associating text information, image information, or audio information with a three-dimensional model of the tourist facility, etc. constructed from at least imaging data of the tourist facility, etc. taken by an imaging device, a participation application acceptance step in which the processor accepts an application from a participant for a service that provides the tourism content, a guide necessity determination step in which the processor determines whether the guide is necessary, a participation fee calculation step in which the processor calculates a participation fee for the participant, and a participation fee calculation step in which the processor calculates the participation fee for the participant. The method for providing an online tour comprises an access authority granting step of granting access authority to tourism content, a service disclosure provision step in which a processor provides the service by accepting access to the tourism content by the participant via a communication control unit, a three-dimensional image changing step in which the processor changes the image displayed on the display unit of the participant terminal in response to operation of the participant terminal by the participant or operation of the guide terminal by the guide, and an allocation calculation step in which the processor calculates an allocation of the participation fee to at least the right holder, wherein the access authority granting step is characterized in that when a guide is required, the processor grants the access authority to the participant and the guide, and when a guide is not required, the processor grants the access authority to the participant. [Industrial Applicability]

[0305] It can be applied not only to terrestrial areas, but also to tourist experiences in normally accessible areas such as underwater, in the sky, and in space. [Explanation of symbols]

[0306] 1. Online tour provision system 10 Server 12 Control section 122 processors 124 ROM 126 RAM 128 Timing section 14 Storage section 14a Program storage section 14b Data storage section 16 Communication control section 18 Input / output section 20 Participant's terminal (Participant's terminal) 22 Control section 24 Memory section 26 Communication control section 28 Input / output section 282 Input section 284 Output section 284a Display section 30 Guide's terminal (Guide's terminal) 40 Travel agency terminal (travel agency terminal) 50 Right holder's terminal (right holder's terminal) UI-11 Main Screen UI-12 Participant display section UI-13 Guide display unit UI-14 Operation icon display section UI-15 Tour Tag Icon UI-16 Content display section UI-17 Guide (video) display unit UI-18 Quiz screen P1 Online Tour Offering Program P11 Service provider setting program P12 Tourism Content Creation Program P13 Tourism content provision program D1 Online tour database D11 Rights holder data table D12 Tourism facilities data table D13 Guide Data Table D14 Participant Data Table D15 Travel agency data table D16 Tour Data Table D17 Tour Tag Data Table D18 Quiz Data Table

Claims

1. a right holder setting unit for setting a right holder having rights to a tourist destination or a tourist facility (referred to as "tourist facility, etc."); a guide setting unit for setting a guide for the tourist facility or the like; a tourism content creation unit that creates tourism content by associating character information, image information, or audio information with a three-dimensional model of the tourism facility or the like that is constructed from at least image data of the tourism facility or the like captured by an imaging device; a participation application receiving unit that receives applications from participants for the service of providing the sightseeing content; a guide necessity determination unit for determining whether a guide is necessary; a participation fee calculation unit for calculating a participation fee for the participant; an access authority granting unit that grants access authority to the sightseeing content to the participant; a service disclosure providing unit that accepts access to the sightseeing content from the participants and provides the service; a three-dimensional image changing unit that changes an image displayed on a display unit of the participant terminal by an operation of the participant terminal by the participant or an operation of the guide terminal by the guide; an allocation calculation unit that calculates an allocation to at least the right holder from the participation fee; Equipped with The access authority granting unit is granting the access authority to the participants and the guide when a guide is required; An online tour providing system, characterized in that the access authority is given to a participant when a guide is not required.

2. A travel agency setting unit is further provided for setting a travel agency that introduces the service, an allocation calculation unit that calculates an allocation to at least the right holder from the participation fee, 2. The online tour providing system according to claim 1, further comprising an allocation calculation unit that calculates an allocation from the participation fee to at least the right holder and an allocation to the travel agency.

3. Furthermore, when providing the above services, a viewpoint switching unit that displays an image from a virtual camera provided on an object of another person on a display unit of the participant who performed the operation; The online tour providing system according to claim 1 , comprising:

4. Furthermore, a batch application reception unit that accepts the participation of multiple participants by one application; an access authority collective granting unit that grants access authority to the sightseeing content to each of the plurality of participants; The online tour providing system according to claim 1 , comprising:

5. a behavior mode receiving unit that receives a selection of the first behavior mode or the second behavior mode by the participant; When providing the above services, In the case of the first action mode, a participant object is controlled in response to an input operation of the participant; a service switching unit for each action mode that controls a participant object in a manner previously determined by a manager or the guide or by an operation of the manager or the guide in the second action mode, regardless of an input operation of the participant; The online tour providing system according to claim 1 , comprising:

6. Furthermore, when providing the above services, an operation authority transfer unit that, by an operation by the guide, suspends the operation authority of the participant object held by the participant and grants the operation authority to the guide; The online tour providing system according to claim 1 , comprising:

7. Furthermore, a quiz setting unit that accepts settings of quizzes regarding the tourist facilities, etc. a quiz setting unit that sets the quiz to the participants; a quiz answer acquisition unit that accepts answers to the quiz from the participants; a quiz question correctness determining unit for determining whether the answer is correct; a quiz result display unit that displays the result of the judgment of correctness; The online tour providing system according to claim 1 , comprising:

8. a display unit determination unit that determines whether the display unit of the participant terminal or the guide terminal is a single display unit or a display unit that displays information separately for the right eye side and the left eye side of the wearer; a wearable terminal content providing unit that provides a right-eye display and a left-eye display to a display unit of the participant terminal or the guide terminal when the participant terminal or the guide terminal is a terminal that provides separate displays to the right eye and the left eye; The online tour providing system according to claim 1 , comprising:

9. and an auxiliary guide additional display unit for displaying an image of the auxiliary guide on the display unit of the participant terminal and the guide terminal when the auxiliary guide is set by the guide setting unit. The online tour providing system according to claim 1 , comprising:

10. Computer, A right holder setting means for setting a right holder having rights to a tourist destination or a tourist facility (hereinafter referred to as "tourist facility, etc."); a guide setting means for setting a guide for the tourist facility, etc.; A tourism content creation means for creating tourism content by associating character information, image information, or audio information with a three-dimensional model of the tourism facility or the like constructed from at least image data of the tourism facility or the like captured by an imaging device. a participation application receiving means for receiving an application from a participant for the service of providing the sightseeing content; a guide necessity determination means for determining whether a guide is necessary; participation fee calculation means for calculating a participation fee for said participant; an access authority granting means for granting access authority to the sightseeing content to the participant; a service publishing and providing means for accepting access to the sightseeing content by the participants and providing the service; a three-dimensional image changing means for changing an image displayed on a display unit of the participant terminal in response to an operation of the participant terminal by the participant or an operation of the guide terminal by the guide; and an allocation calculation means for calculating an allocation to at least the right holder from the participation fee; Function as a The access authority granting means includes: granting the access authority to the participants and the guide when a guide is required; An online tour providing program, characterized in that the access authority is given to a participant when a guide is not required.

11. a right holder setting step in which the processor stores in the storage unit a right holder who has the right of a tourist destination or a tourist facility (hereinafter referred to as "tourist facility, etc."); a guide setting step in which the processor stores a guide for the tourist facility or the like in a storage unit; a tourism content creation step in which the processor creates tourism content by associating character information, image information, or audio information with a three-dimensional model of the tourism facility etc. constructed from at least imaging data of the tourism facility etc. captured by an imaging device; a participation application receiving step in which a processor receives an application from a participant for a service that provides the sightseeing content; a guide necessity determination step in which the processor determines whether the guide is necessary; a participation fee calculation step in which a processor calculates a participation fee for the participant; an access permission granting step in which a processor grants access permission to the tourism content to the participant; a service publishing and providing step in which a processor provides the service by accepting access to the sightseeing content by the participant through a communication control unit; a three-dimensional image changing step in which a processor changes an image displayed on a display unit of the participant terminal in response to an operation of the participant terminal by the participant or an operation of the guide terminal by the guide; an allocation calculation step in which a processor calculates an allocation to at least the right holder from the participation fee; It is composed of The access authority granting step includes: When a guide is required, the processor grants the access authority to the participant and the guide; A method for providing an online tour, characterized in that a processor grants the access authority to a participant when a guide is not required.

Images