Virtual space exhibition system

The virtual exhibition system addresses the lack of realism and communication in existing systems by using visibility information to synchronize user and guide views and control user movement, enhancing the guidance and user experience in virtual exhibitions.

JP2024157590AActive Publication Date: 2024-11-08ISHIDA TAISEISHA INC

Patent Information

Application Number
JP2023071995
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-04-26
Publication Date
2024-11-08
Estimated Expiration
2043-04-26

AI Technical Summary

Technical Problem

Existing virtual exhibition systems lack realism and effective communication between guides and users, as they rely on pre-prepared text and audio explanations, and staff cannot determine the visitor's focus within the virtual space, leading to suboptimal guidance.

Method used

A virtual exhibition system that uses visibility information to display the virtual space and exhibition to users, allowing guides to view the same image and adjust guidance based on the user's viewpoint, line of sight, and angle of view, with synchronized image display between user and guide devices, and controlled user movement within the virtual space.

Benefits of technology

Enhances realism and facilitates targeted guidance by aligning explanations with user interest, improving communication and user experience in virtual exhibitions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024157590000001_ABST
    Figure 2024157590000001_ABST
Patent Text Reader

Abstract

To support smooth communication between a guide and a user, in virtual exhibition in virtual space.SOLUTION: A virtual exhibition system in which an exhibitor can exhibit a virtual booth IM1 is constructed in virtual space IM0. A user can view exhibition while moving on each booth. Positions on which the user can move are restricted to specified positions in the booth as view positions VP1, VP2. A user image in which the exhibition etc. of the booth is three-dimensionally shown is displayed, in a user terminal, by performing perspective projection of the virtual space, at each view position. Moreover, the user image is displayed as it is, at an exhibitor terminal. In this manner, because an operator of the exhibitor can have conversation with the user, based on what the user is looking at, guidance along interest of the user can be performed.SELECTED DRAWING: Figure 4
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a virtual space exhibition system that exhibits a virtual space to a user. [Background technology]

[0002] Recently, technology for holding exhibitions and other events in virtual spaces has been attracting attention. Patent Document 1 discloses a system that presents data in a virtual space to a viewing user in a viewable state using a three-dimensional viewer such as a head-mounted display. The viewing user can view virtual exhibits while moving around in the virtual space. If the viewing user makes a request, an explanation of the virtual exhibit is displayed in text or output as audio. Patent Document 2 discloses a technology that allows staff at an online exhibition to guide visitors via video calls. Staff can also share a screen with visitors and give explanations. Patent Document 3 discloses a virtual tourism system that uses a virtual space for sightseeing. Users can sightsee by moving an avatar in the virtual space and can also converse with other users. A guide can also guide multiple users on a virtual sightseeing tour. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-133109 A [Patent Document 2] JP 2022-164988 A [Patent Document 3] JP 2023-48271 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the prior art, in a virtual exhibition in a virtual space, explanations of exhibits are provided to visitors using pre-prepared text and audio, which causes a lack of realism. In Patent Documents 2 and 3, it is possible to converse with visitors, but even in this case, the staff cannot grasp which part of the exhibit the visitor is looking at when asking a question, so communication is not as smooth as in a real exhibition. In addition, if the presenter wants to show the visitors an exhibit, he or she must verbally explain where the exhibit is located, making it difficult to guide the visitors. This problem is not limited to cases where an exhibition is held in a virtual space, but is a common problem when a virtual space is exhibited to users for various purposes such as sightseeing or other nature observation, visiting art galleries, museums, factories and other facilities, etc. The present invention aims to solve this problem and facilitate communication between guides and users when exhibiting a virtual space. [Means for solving the problem]

[0005] The present invention provides, as a first aspect, A virtual exhibition system that performs an electronically constructed virtual exhibition in a virtual space, comprising: a virtual data storage unit that stores virtual data representing the virtual space and the virtual exhibition; a visibility information input unit for inputting visibility information that specifies a visibility in the virtual space of a user viewing the virtual exhibit; a user display control unit that uses the virtual data based on the visibility information to display an image of the virtual exhibition on a user output device used by the user; a guide display control unit that causes a guide output device used by a guide of the virtual exhibition to display the same image as the image to be displayed on the user image display device; a guide information input unit that receives, from the guide, guide information to be provided to the user; The virtual exhibition system may further include an output control unit that causes the user output device to output the guidance information.

[0006] In a first aspect of the present invention, by displaying images of a virtual space and a virtual exhibit, the user can be made to recognize that he or she is viewing a virtual exhibit within the virtual space. The same images displayed to the user can also be displayed to a guide of the virtual exhibit. In the prior art, it was possible for the guide to share images prepared by the guide with the user, but this method did not allow the guide to know what the user was interested in. In contrast, in the present invention, the guide can guide the user while checking what scene the user is looking at in the virtual space, making it possible to provide guidance that is in line with the user's interests. In the present invention, the field of view information may be the position of the viewpoint, the line of sight, the angle of view, etc. in the virtual space. Only a part of these may be input. In addition, a part of the information may be a fixed value set in advance. In the present invention, the virtual space and virtual exhibit may be two-dimensional, but by making them three-dimensional, a more realistic exhibit can be created. The virtual space and virtual exhibit may be provided using CG (computer graphics), or may be provided by attaching textures of still images or videos created using CG, or photographs or videos of real exhibits. The virtual space and virtual exhibit may be provided in various other ways. If real-life video or CG video (animation) is used as texture, it is possible to reproduce the movement in the virtual space, further improving the sense of realism. In particular, when the present invention is used for tourist information, the use of video has the advantage of being able to express the flow of a waterfall or river, or the way trees flutter in the wind. Hereinafter, in this specification, when photographs and CG are mentioned, actual videos and animations may be used instead of or in addition to them. The virtual exhibit may be an exhibition that explains the contents of a product or service, or it may be a variety of contents such as scenery of a tourist spot, an art museum, or an exhibition. The user output device may be a display of a personal computer, tablet, smartphone, or the like, or may utilize goggles or glasses worn on the head. In particular, when goggles or glasses are used, there is an advantage that the sense of realism is further improved. The same applies to the guide output device. Furthermore, the user output device and the guide output device may be capable of outputting not only images but also sounds. The guidance information provided to the user can take various forms, such as text, images, and audio.

[0007] In the present invention, The user display control unit generates a two-dimensional image of the virtual exhibit based on the visibility information, The guide display control unit may perform the display by utilizing the two-dimensional image generated by the user display control unit, without generating an image based on the visibility information.

[0008] There are two possible methods for sharing the image viewed by the user with the guide. The first method is to generate the images to be displayed on the user output device and the guidance output device separately, and to share the parameters for generating the images between the two. The second method is to use the image data generated for the user output device as is on the guidance output device as well. The above embodiment corresponds to the second method. This has the disadvantage of transferring relatively large amounts of image data, but has the advantage of reducing the load of generating images to be displayed on the guidance output device and making it relatively easy and reliable to synchronize the two.

[0009] In the present invention, the user display control unit is capable of displaying a moving image on the user output device using moving image data designated by the guide, The guide display control unit may intermittently acquire playback position information indicating at what time in the video data the user display control unit is playing, and at the time of acquisition, display a video on the guide output device using the video data specified by the guide, while synchronizing with the playback position information.

[0010] When playing back moving images, it is preferable to play them separately on the user output device and the guide output device, since there is no need for perspective projection into the virtual space. However, in this case, since they are played separately, the synchronization between the two is not necessarily maintained, and there may be a discrepancy in the playback position. On the other hand, if one tries to always maintain the synchronization between the two, it is necessary to constantly monitor the playback position of the moving image on the user output device, which imposes a large load on the playback. According to the above aspect, by intermittently acquiring the playback position information, it is possible to realize synchronization between the user output device and the guidance output device to a degree that does not cause discomfort while suppressing the load. In the above aspect, the timing for intermittently acquiring the playback position information may be determined arbitrarily within a range that does not cause discomfort, depending on the occurrence of a discrepancy when the video is played on the user output device and the guidance output device.

[0011] In a second aspect, the present invention provides a method for producing a composition comprising the steps of: A virtual exhibition system that performs an electronically constructed virtual exhibition in a virtual space, comprising: a virtual data storage unit that stores virtual data representing the virtual space and the virtual exhibition; a position setting unit that sets a position of a user in the virtual space in response to an instruction from the user viewing the virtual exhibition; a visibility information input unit for inputting visibility information that specifies the visibility of a user viewing the virtual exhibit at the position in the virtual space; a user display control unit that uses the virtual data based on the visibility information to display an image of the virtual exhibition on a user output device used by the user; The virtual exhibition system may be provided with a push-type guidance unit that controls the position setting by the position setting unit and / or the field of view information based on instructions from a guide of the virtual exhibition or on data prepared in advance.

[0012] In a second aspect of the present invention, the user can set a position in the virtual space by himself / herself, that is, the user can move in the virtual space at his / her own will. The user may be allowed to move freely in the virtual space, just as one would look around various booths at a real exhibition. Also, the user may be allowed to move to the virtual exhibit corresponding to a desired exhibitor by selecting the exhibitor from a list or the like. In the second aspect, the position of the user and / or the visibility information can be controlled based on the guide's instructions or on data prepared in advance. In other words, the instructions of the guide, etc. can be prioritized over the user's instructions. This makes it possible to guide the user while moving in the virtual space, just as a guide guides the user in the real space. Furthermore, by controlling the visibility information, the user can be guided in a desired direction. If these guidance procedures can be performed based on data prepared in advance, automatic guidance can also be realized. In the above aspect, the operating state in which the guide's instructions are prioritized may be always valid, or may be valid only when consent is obtained from the user. Also, the guide may be configured to freely move the user's position, or may be limited to a predetermined guide point prepared in advance in the virtual space. The latter method makes it possible to easily move the user to a desired position. Also, the user may feel some discomfort if he or she is allowed to move freely against his or her will, even in a virtual space, but the latter method makes it possible to reduce such discomfort.

[0013] In either the first or second aspect of the present invention, the virtual data storage unit stores guide image data for displaying an image representing the guide, The user display control unit is capable of displaying an image of the guide based on the guide image data, The display position of the image of the guide may be restricted to a predetermined position set in advance within the virtual space.

[0014] By displaying the image of the guide in this way, a sense of realism can be given to the user. It is preferable that the image of the guide is the same as that of the virtual exhibition. In other words, if the virtual exhibition is performed using two-dimensional images, it is preferable that the image of the guide is also two-dimensional, and if the virtual exhibition is performed using three-dimensional images, it is preferable that the image of the guide is also three-dimensional. The image of the guide can be of various types, such as a human appearance, an animal, or a fictional creature. The guide image data may be created using CG, or may use photographs, videos, or the like. Although the position of the guide can be moved according to the position of the user, in the above embodiment, the position of the guide in the virtual space is restricted to a predetermined position. This has the advantage that it is easy to avoid the inconvenience of the guide's position interfering with the virtual exhibit. The position of the guide in the virtual space may be defined by one or more points, or may be defined by an area. The image of the guide may be displayed at all times, or may be displayed in response to a request from the user or an operation by the guide.

[0015] In the above aspect, the predetermined position is set so that the image of the guide can be displayed within the field of view of the user regardless of the position of the user, Only one image of the guide may be displayed in the field of view.

[0016] By doing so, the image of the guide is displayed to the user no matter where the user is, so the sense of realism can be maintained. The guide only needs to be within the user's field of vision, and does not necessarily need to be displayed in front of the user. Also, the guide does not need to be in the user's field of vision at all times, and it is sufficient that the guide can be seen by looking in any direction from each position. Although multiple guides may be displayed, displaying only one will give the user the feeling that he or she is communicating with the displayed guide, further enhancing the sense of realism. There are, for example, two possible methods for displaying only one guide. The first method is to determine the position of the guide in advance so that multiple guides do not come within the user's field of view. The second method is to select and display only one guide when multiple guides come within the user's field of view. Either method may be adopted. In the above embodiment, the user's "field of view" may be limited to a predetermined distance range from the user's position. This allows guides to be displayed in the distance, while limiting the number of guides displayed nearby to one.

[0017] In either the first or second aspect of the present invention, The position of the user in the virtual space may be restricted to a preset default position.

[0018] In this way, a virtual exhibition can be provided efficiently, and the user can view it efficiently. In the above aspect, the visibility information at the specified position may be arbitrary, or the visibility information may be restricted.

[0019] In either the first or second aspect of the present invention, The virtual space may be individually allocated to one or a particular group of users.

[0020] In the above embodiment, the data constituting the virtual space may be used by multiple users. "Individually" means that no users other than the user himself or the group to which the user belongs exist in the same virtual space. This makes it possible to prevent communication between the user and the guide from being leaked to other users. A technology known as the Metaverse allows an unspecified number of users to gather and communicate within a virtual space, but the above embodiment does not use this method; instead, a unique virtual space is created for each user or specific group. It should be noted that such a unique virtual space and a virtual space in which unspecified users coexist may be selected at will.

[0021] In either the first or second aspect of the present invention, a material distribution unit that, in response to an instruction from the user, causes the user to acquire material data prepared by the guide as a distribution material in a state in which the user can freely use the material; The information providing unit may provide the guide with information about which of the users has acquired the material data.

[0022] This makes it possible to analyze which handouts users are interested in. The results of this analysis can be used, for example, to select users who are more likely to close a deal. They can also be used to improve the content of handouts and for other purposes.

[0023] In either the first or second aspect of the present invention, The virtual space may further include an information providing unit that provides the guide with information about the time that the user has stayed in the virtual space.

[0024] This makes it possible to analyze the extent to which the virtual space has attracted the interest of users. This analysis can also be used to select users who are likely to close business deals, improve the content of virtual exhibits, and for other purposes.

[0025] In either the first or second aspect of the present invention, the virtual data storage unit stores a plurality of virtual data; The user display control unit may perform display using virtual data selected by the user from the plurality of virtual data.

[0026] In this way, it becomes possible to view several virtual exhibits in a virtual space, just as one views several booths at an exhibition. The multiple virtual exhibits may be provided by different entities or may be provided by the same entity.

[0027] The present invention does not necessarily have to have all of the above-mentioned features, and some of them may be omitted or combined as appropriate. The above-mentioned virtual exhibition may be configured as a virtual exhibition method executed by a computer, or as a computer program for causing a computer to execute such a method. Furthermore, the present invention may be configured as a computer-readable recording medium on which such a computer program is recorded. [Brief description of the drawings]

[0028] [Figure 1] FIG. 1 is an explanatory diagram showing a configuration of a virtual exhibition system in an embodiment. [Diagram 2] FIG. 2 is an explanatory diagram showing a data structure. [Diagram 3] FIG. 11 is an explanatory diagram showing a method for generating a displayed image. [Figure 4] FIG. 1 is an explanatory diagram showing an example image (1) displayed to a user. [Diagram 5] FIG. 13 is an explanatory diagram showing an example image (2) displayed to the user. [Figure 6] FIG. 13 is an explanatory diagram showing how the guide's position is set. [Figure 7] 13 is a flowchart of a virtual space selection process. [Figure 8] 13 is a flowchart of a user image generation process. [Figure 9]13 is a flowchart of a viewpoint change process. [Figure 10] 13 is a flowchart of a video playback process. [Figure 11] 13 is a flowchart of an information providing process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0029] Hereinafter, an embodiment of the present invention will be described in the following sections, taking as an example a case where an exhibition is held in a virtual space. A. System Configuration: B. How to display the image: C. Guide position: D. Virtual space selection processing: E. User image generation process: F.Video playback processing: G. Processing of Data: H. Effects and Modifications:

[0030] A. System Configuration: 1 is an explanatory diagram showing the configuration of a virtual exhibition system according to an embodiment of the present invention, in which a plurality of exhibitors set up virtual booths in a virtual space and provide users with virtual exhibitions of products or services. The virtual exhibition system is composed of a server 100 that provides its main functions, exhibitor terminals 200 used by exhibitors at each booth, and user terminals 300 used by users visiting the booths, all connected via a network NE such as the Internet. The exhibitor terminals 200 and the user terminals 300 may be various devices that can be connected to a network NE, such as personal computers, tablets, and smartphones. The functions of the server 100 may be provided by a plurality of devices. A plurality of exhibitor terminals 200 and user terminals 300 can also be connected.

[0031] The exhibitor terminal 200 and the user terminal are provided with the functions shown in the figure. The transmitting / receiving unit 211 and the transmitting / receiving unit 311 have a function of transmitting and receiving information via the network NE. The information includes text, images, videos, sounds, etc. Also included are various instructions from exhibitors and users to the virtual exhibition system. The input / output unit 212 and the input / output unit 312 control devices that input and output information. In this embodiment, the exhibitor's operator wears goggles 201, and the user also wears goggles 301. These goggles 201, 301 display various images provided by the virtual exhibition system and output audio. They also have built-in microphones, allowing audio input. When the operator makes input to the virtual exhibition system, he uses devices such as a keyboard and mouse of the exhibitor terminal 200. When the user makes input, he uses the touch panel of the user terminal 300. Input and output are not limited to the above example, and various forms can be taken. The application 213 and the application 313 provide functions such as performing the above-mentioned input and output based on information transmitted from the virtual exhibition system. They may be configured as software specific to the virtual exhibition system, or may use general-purpose software such as a browser.

[0032] The server 100 is provided with the functions shown in the figure. The virtual data storage unit 101 stores various data (hereinafter, sometimes collectively referred to as "virtual data") for realizing a virtual space and a virtual exhibition. In this embodiment, the virtual data representing the entire venue where the virtual exhibition is held is prepared by the organizer of the virtual exhibition or the operator of the virtual exhibition system. The virtual data representing each booth is prepared by the exhibitor. The user data storage unit 102 stores information about users who visit each booth. The exhibitor data storage unit 103 stores information about the exhibitors who will be exhibiting at each booth. The material data storage unit 104 stores data such as materials and videos to be distributed to users at each booth. The specific contents and structures of these data will be described later.

[0033] The virtual space allocation unit 111 allocates a virtual space to a user. In this embodiment, when a user visits a booth, the virtual space of that booth is set as a space unique to that user. This allows the user to communicate with the exhibitor's operator as if that user were the only person present at that booth. The virtual space allocation unit 111 realizes this function. The data management unit 112 manages the input and output of data in the virtual data storage unit 101 , the user data storage unit 102 , the exhibitor data storage unit 103 , and the material data storage unit 104 . The transmitting / receiving unit 113 transmits and receives information to and from the exhibitor terminal 200 and the user terminal 300 via the network NE.

[0034] The exhibitor display control unit 121 generates and transmits image data to be displayed on the exhibitor terminal 200. Information for generating an image may be transmitted to the exhibitor terminal 200 to cause the exhibitor terminal 200 to generate the image data. The guide information input unit 122 inputs guide information for introducing products or services from the exhibitor terminal 200. The operator can input the guide information in the form of characters, voice, etc. at the exhibitor terminal 200. The input guide information is transmitted to the user terminal 300, thereby realizing communication between the operator and the user. The information providing unit 123 provides various information regarding the performance of the exhibit to the exhibitor. For example, the information includes the time that the user stayed at the booth, the number of users who visited the booth, the type of materials that the user acquired at the booth, etc. The information may also include the results of an analysis performed based on this information. The push-type guidance unit 124 has a function that allows the operator to set the user's position, line of sight, etc. This allows the operator to guide the user around the booth as if he or she were taking the user with him or her. Also, if the operator is unable to respond, the server 100 may automatically guide the user according to previously prepared data.

[0035] The user display control unit 131 generates and transmits image data (hereinafter referred to as "user image") to be displayed on the user terminal 300. Information for generating an image may be transmitted to the user terminal 300 to cause the user terminal 300 to generate the image data. The position setting unit 132 sets the position of the user in the virtual space. In this embodiment, the user can move within the virtual space by setting the position. The visibility information input unit 133 inputs visibility information from the user terminal 300. The visibility information is information such as the viewpoint position, line of sight direction, and angle of view for generating a user image. The material distribution unit 134 transmits data of materials distributed in the booth in accordance with instructions from the user. The data of the materials is downloaded to the user terminal 300 or the like.

[0036] The virtual exhibition system is not limited to the above configuration and may have various configurations. It is not necessary to have all the functions shown in the figure, and some of them may be omitted. In addition, the virtual exhibition system may have functions other than those shown in the figure. In this embodiment, each of the functions described above is implemented as software, but it is also possible to implement a part or all of them as hardware.

[0037] FIG. 2 is an explanatory diagram showing a data structure. In the exhibitor data storage unit 103, an exhibitor ID, which is an identifier, is assigned to each exhibitor, and exhibitor information such as exhibitor name, contact information, virtual data, invitees, and visitors is stored. The virtual data information indicates the location of the 3D model, photos, videos, textures, and other data that constitute the exhibitor's booth. In this embodiment, the virtual data is stored in the virtual data storage unit 101 with a virtual data ID, which is an identifier, so the virtual data ID can be stored as the virtual data information. The guest information is information about users who are invited to a booth by an exhibitor. As will be described later, each user is assigned a user ID, which is an identifier, so the guest information can be stored as the user ID. Visitor information is information about users who visited the booth. As with invitees, the user ID is stored.

[0038] The user data storage unit 102 stores user information such as name, contact information, visit log, material log, and group for each user ID. A visit log is a history of booths visited. It can record the time when you visited a booth and the time when you left. Also, a booth can be identified by its exhibitor ID. The material log is a history of materials acquired through downloading. It can be recorded using a unique ID attached to the material. The group information stores the user IDs of the user and the other users who make up the group. In this embodiment, when a booth is visited, the booth is assigned as a virtual space specific to the user, but by specifying a group, it is also possible to assign the virtual space specific to the group. It is preferable that the group information be editable by the user as appropriate.

[0039] The virtual data storage unit 101 stores virtual data that constitutes a virtual space and a virtual exhibition. "Venue data" is data prepared by the organizer of the virtual exhibition or the operator of the virtual exhibition system. It stores 3D models, photos, videos, textures, etc. that represent the virtual space of the entire venue where the virtual exhibition will be held. The virtual data for each exhibitor's booth is stored with a virtual data ID for each booth. If an exhibitor has multiple booths, multiple virtual data IDs will be assigned to each exhibitor. The virtual data information includes exhibitor IDs, booth data, exhibit data, viewpoint data, guide data, viewing logs, and the like. The booth data includes data such as a 3D model, photos, videos, textures, etc. The booth may be represented using photos and videos of the actual booth. Exhibit data includes 3D data, 2D image data, video data, etc. of exhibits in the booth. Exhibits may be provided as CG, or by attaching photographs or videos of the exhibits. The viewpoint data stores the positions to which the user can move within the booth. In this embodiment, as described below, the user is not allowed to move completely freely within the booth, but can selectively move to a preset location. The viewpoint data is data that represents the positions to which the user can move. The guide data is three-dimensional image data, so-called avatar data, that represents a person who will guide the user in the booth. In this embodiment, the guide does not move freely in the booth, but is fixed and displayed at a specified position. The guide data also stores information on the position where the guide is displayed. The viewing log is a history of users who visited the booth. The visit time and the departure time are recorded for each user. The content of the communication between the user and the operator may also be recorded. In addition, data may be prepared to automatically guide users around the booth when an operator is not present, such as voice guidance data, data for controlling the user's viewpoint and line of sight, data for instructing the user to enlarge panels in the booth, play videos, etc.

[0040] The virtual space allocation data is data generated by the virtual space allocation unit 111 when a user visits a booth. As shown in the figure, a virtual data ID that identifies the booth is associated with an exhibitor ID that identifies the exhibitor, and a user ID that identifies the user. When users are visiting as a group, multiple user IDs in the group are associated. Questions from the user are conveyed only to the operator of the associated exhibitor ID, and conversely, the operator's responses are conveyed only to the associated user. This allows users to communicate privately with the operator without worrying about information being leaked to other users. Virtual space allocation data is generated for each user or group. Virtual data IDs may be duplicated. This allows a booth exhibit to be provided to multiple users, while still providing each user with a unique virtual space. When multiple users are visiting a booth, multiple virtual space allocation data are set for one exhibitor ID. In such a case, the operator will select one of the visiting users and communicate with that user. The conversation flag is a flag that indicates whether or not a conversation with the operator is possible. When the operator selects one of the visiting users, the conversation flag in the virtual space allocation data for that user is turned on, and the conversation flags in the other virtual space allocation data are turned off. In this way, the above-mentioned closed communication can be realized even when multiple users are visiting a booth. When multiple exhibitor terminals 200 are prepared, multiple operators may share the responsibility of responding to users. In such a case, the flag may be omitted.

[0041] B. How to display the image: FIG. 3 is an explanatory diagram showing a method of generating a displayed image. In this embodiment, the virtual space representing the venue is constructed by pasting the texture of photos and videos representing the venue onto the inner surface of a spherical virtual space S. Also, as shown in the figure, the exhibitor has constructed virtual data for booth B using a 3D model or the like, and the booth B is placed in the virtual space. Booth B may also be constructed by pasting photos and videos taken of an actual booth. The user image can be generated by perspective projection in the virtual space according to visibility information such as the viewpoint VP, line of sight direction VL, and angle of view VA. The viewpoint VP and line of sight direction VL can be determined using three-dimensional coordinates in the virtual space. The angle of view VA may be fixed or may be adjustable by the user. For ease of illustration and explanation, in FIG. 3 the viewpoint VP is shown as being located outside the virtual space S of the venue, but in reality the viewpoint VP is located at the center of the virtual space S.

[0042] FIG. 4 is an explanatory diagram showing an example image (1) displayed to the user. An image IM0 of the venue and an image IM1 depicting a booth in three dimensions are displayed. Also displayed is a guide G guiding the user around the booth. The person PM in the booth is a fixed human-shaped model drawn to convey a sense of realism. Exhibits hung on the wall are panels shown as still images, videos, etc. VP1 drawn on the floor represents the viewpoint position to which the user can move. A floor plan W1 of the booth is displayed in the upper left of the user image. An enlarged view of the floor plan W1 is shown at the top of the figure. In this embodiment, the user cannot move freely in the booth, but selectively moves to a preset viewpoint position. The floor plan W1 illustrates the viewpoint positions and field of view to which the user can move. The viewpoint positions to which the user can move are indicated by a camera icon. The user image shown in FIG. 4 shows a state seen from the viewpoint position VP0 as in the field of view VA. The circular viewpoint position VP1 shown on the floor in the user image corresponds to the viewpoint position VP1 shown in the lower right of the floor plan W1. When the user selects one of the camera icons by clicking on it, the viewpoint position moves. The user may select a circular viewpoint position VP1 drawn on the floor in the user image. The image when viewpoint position VP2 is selected is shown in Figure 5.

[0043] 5 is an explanatory diagram showing an example image (2) displayed to the user. From the plan view W1 shown in the upper left, it can be seen that the image is of the state in which the viewpoint position VP2 in FIG. 4 is selected and the user is looking toward the wall. The wall is painted with displays, and a video MV is automatically played silently, creating a sense of realism as an exhibition. When the user clicks on icon IC1, the video MV is enlarged and played with sound. The video may be played over the entire user image. Next to it, a panel is displayed. When users click on the icon IC2 under the panel, the panel is enlarged. When users click on the icon IC3, the document can be downloaded.

[0044] An icon may be prepared for users to apply for guidance from exhibitors. When the user clicks on this icon, the user can receive guidance in text, voice, etc. through closed communication with the operator. When the user applies for guidance and the operator responds, an image of the guide G shown in Figure 4 may be displayed. Conversely, the operator may offer guidance to the user. An icon for calling the user may be provided on the screen of the exhibitor terminal 200. When the operator operates the icon to call the user, an icon indicating that the user is being called may be displayed on the user's screen, and the user may respond to this icon to be connected to the operator. When the user is connected to the operator, an image of the guide G shown in FIG. 4 may be displayed.

[0045] C. Guide position: Fig. 6 is an explanatory diagram showing how to set the position of the guide. In this embodiment, the position where the guide is displayed is set in advance within the booth. This has the advantage that it is easy to avoid problems such as the guide interfering with the exhibits in the booth. The position of the guide can be determined arbitrarily, but in this embodiment, it is set so that one guide always appears on the user image. As shown in the figure, the user can move between viewpoint positions P1 to P5. Also, the user can view fields V1 to V5 from the viewpoint positions P1 to P5, respectively. The guide positions G1 to G5 may be set at any positions within each of the fields of view V1 to V5. Since the fields of view V1 and V2 partially overlap, the guide may be placed at position G12, and positions G1 and G2 may be omitted. With the above settings, at the viewpoints P1 to P4, only one guide is displayed in the fields of view V1 to V4. When the user communicates with the operator, displaying the guide in this way can give the user a sense of realism. Even if the guide is always displayed at positions G1 to G4, the user can only view one guide, so no discomfort is felt. It is acceptable if the guide temporarily disappears from the user image as the user moves his or her line of sight within the fields of view V1 to V4.

[0046] For the viewpoint V5, only one guide is displayed at position G5 within the field of view V5, but guides at positions G1 to G4 may be displayed in the distance. Since such distant guides do not cause discomfort to the user at the viewpoint position P5, there is no problem in displaying them as they are. Also, if there is a possibility that multiple guides will be displayed within the field of view, it is possible to display the guide only at position G5 within the field of view V5, and not display the guides at the other positions G1 to G4.

[0047] D. Virtual space selection processing: A flowchart for realizing a virtual exhibition will be described below. 7 is a flowchart of the virtual space selection process. This process generates virtual space allocation data (see FIG. 2) when a user visits a booth. This process is mainly executed by the virtual space allocation unit 111 (see FIG. 1), and in terms of hardware, it is mainly executed by the server 100. In this embodiment, the user can visit a booth by selecting an exhibitor or booth from the exhibitor list or booth list. The user can also visit a booth by moving around various booths in the virtual space and moving into the desired booth. In the virtual space selection process, the server 100 checks whether or not an exhibitor or booth has been designated by the user (step S10). If a designation has been made, the server 100 generates virtual space allocation data accordingly (step S13). If not specified, the server 100 inputs the user's position information in the virtual space (step S11), and judges whether or not the user is in any of the exhibitor's booths (step S12). If the user is within an exhibitor's booth, it is determined that the user is visiting that booth, and virtual space allocation data is generated (step S13).If the user is not within an exhibitor's booth, the process from step S10 onwards is repeated. By the above process, a unique virtual space can be assigned to each user. If the users have set up groups, the virtual space assignment data can be generated assuming that all users in the group will visit the booth when instructed by any user (step S10) or when any user is in the exhibitor's booth (step S12).

[0048] In the above description, it is assumed that the user moves freely through the virtual space and visits booths, just as if the user were to visit booths while touring the venue at a real exhibition. This embodiment is not necessarily limited to a situation in which the user can move freely through the virtual space. In such a case, step S11 (input of user location information) and step S12 (determination of whether or not the user is within an exhibitor's booth) in FIG. 7 may be omitted. When the user has not specified a booth (step S10), the server 100 may generate virtual space allocation data for a predetermined location.

[0049] E. User image generation process: Fig. 8 is a flowchart of the user image generation process. This process generates a user image by performing the perspective projection shown in Fig. 3. This process is mainly executed by the user display control unit 131 (see Fig. 1), and in terms of hardware, this process is mainly executed by the server 100. When the process starts, the server 100 inputs the user's position information and visibility information (step S21). Then, the position of the guide is set (step S22). The position of the guide is set in advance in the booth as described in FIG. 6. In step S22, a position to be displayed is selected from these positions. If the guide is to be displayed at all of the set positions, the process of step S22 may be omitted. The server 100 generates a user image by perspective projection (step S23). The generated user image is transmitted to the user terminal. Moreover, when the operator is giving guidance to the user (step S24), the generated user image is also transmitted to the exhibitor terminal (step S25). In this way, the operator can see the same image as the user is viewing. In a real space, the guide can guide the user according to the user's interests depending on where the user is looking, but this is difficult in a virtual space. However, according to this embodiment, the operator can check the user's image, so the operator can infer the user's interests from the image and provide guidance. It should be noted that the process of determining whether or not the exhibition is being guided (step S24) may be omitted, and the user image may be transmitted to the exhibitor terminal at all times. The above process is repeatedly executed while the virtual exhibition system is being used.

[0050] The user image displayed on the exhibitor terminal may be generated in the same manner as the user image, based on the user's position information and visibility information. However, with this method, due to a difference in calculation timing, etc., synchronization between the image displayed on the exhibitor terminal and the user image may not be achieved. In this embodiment, as described above, the generated user image is also used on the exhibitor terminal, thereby easily and reliably achieving synchronization of the user image displayed on the user terminal and the exhibitor terminal.

[0051] 9 is a flowchart of the viewpoint change process. This process is performed when the user moves around the booth or moves his / her line of sight. This process is mainly executed by the push-type information unit 124 (see FIG. 1), and in terms of hardware, this process is mainly executed by the server 100. When the process starts, the server 100 checks whether the user agrees to the operator guiding the user (step S31). For example, a screen may be displayed to the user asking whether the user agrees and to have the user respond. Also, if the operator verbally confirms with the user and obtains consent, the operator may instruct the server 100 to that effect. Furthermore, if the user is at a specified viewpoint position in the booth, it may be treated as unconditional consent. If the user does not agree (step S31), the server 100 inputs an instruction for position information and visibility information from the user terminal (step S33). This instruction can be input, for example, by using a touch panel of the user terminal. Alternatively, the instruction can be input by the movement of goggles worn by the user. On the other hand, if the user agrees to the guidance (step S31), the server 100 inputs instructions for location information and visibility information from the exhibitor terminal (step S32). These instructions can be input using, for example, the keyboard or mouse of the exhibitor terminal. Alternatively, the instructions may be input by the movement of goggles worn by the operator. In this case, the operator will decide the user's location information and visibility information regardless of the user's wishes. The operator may be able to specify only one of the location information or visibility information. The server sets the user's location information and visibility information depending on whether the user has given consent, and then ends the process. This information is used to generate the user image (Figure 4).

[0052] According to the viewpoint change process described above, if the user does not consent to being guided by the operator, the user can freely browse inside the booth by selecting the viewpoint position (see FIG. 4) and visibility information of his / her own will. If the user consents to being guided by the operator, the operator can move around the booth and guide the user as if he / she were taking the user along. For example, when the operator guides the user to the exhibition location of the information he / she is looking for, this has the advantage that the guidance can be done more simply and quickly than by giving verbal guidance to the location in the virtual space. It is also useful when the operator proactively guides the user to the exhibition contents based on a conversation with the user.

[0053] In the above process, when an operator is not available, the server 100 may automatically provide guidance based on instruction data prepared in advance. For example, in the above process, instead of inputting the position information and visibility information based on the operator's instruction (step S32), the user's position information and visibility information may be input based on instruction data prepared in advance in the virtual data storage unit 101 or the like. For example, by preparing instruction data for the position information and visibility information in chronological order, the user can be moved sequentially and guidance can be provided while changing the viewing direction. When providing guidance, audio data prepared in advance may be played back. Also, the display of the panel may be enlarged, a video may be played back, and documents may be downloaded automatically.

[0054] F.Video playback processing: Fig. 10 is a flowchart of the video playback process. This process is executed when the user clicks on the video playback icon IC1 in Fig. 5. This process is executed mainly by the exhibitor display control unit 121 (see Fig. 1), and in terms of hardware, it is executed mainly by the server 100. When video playback is instructed, the server 100 plays the instructed video data (step S41). The played video data can be viewed on the user terminal. In this embodiment, the video data is played by streaming, and when the server 100 sequentially transmits the video data to the user terminal 300, the user terminal 300 receives it and plays it sequentially while storing it in a buffer. The video data may be played on a display MV in the user image shown in FIG. 5, or a window for video playback may be displayed on the user terminal 300. In step S41, the server 100 also plays the video on the exhibitor terminal 200 in conjunction with the playback of the video on the user terminal 300. The video is also played on the exhibitor terminal 200 in a streaming manner, just like the user terminal 300. In order to synchronize the image displayed on the exhibitor terminal 200 with the image displayed on the user terminal 300, it is possible to consider a method of transmitting the image data being played on the user terminal 300 to the exhibitor terminal 200 via the network NE. However, this method has the disadvantage that complete synchronization is difficult to achieve due to the large amount of traffic and delays associated with communication. Therefore, this embodiment does not adopt this method, and each is played in a streaming manner.

[0055] However, when playing back by streaming, there is a risk of the timing of playback being out of sync between the exhibitor terminal 200 and the user terminal 300. Therefore, in this embodiment, synchronization between the two is attempted intermittently at a predetermined timing. The synchronization timing is set to one second in this embodiment, but it can be arbitrarily determined within a range that realizes synchronization without discomfort, taking into consideration the magnitude of the delay that occurs in the playback of the video. When it is determined that it is time for synchronization (step S42), the server 100 performs a process of synchronizing the exhibitor's video playback position with the user's video playback position (step S43). The exhibitor side is synchronized with the user side in order to prevent the user from feeling uncomfortable about the video being viewed. The server 100 repeats the above process until the playback of the video ends or until the user instructs the server 100 to end the process (step S44).

[0056] An overview of the synchronization process (step S43) is shown in Fig. 10(b). The upper part shows each frame of the video played on the user terminal in chronological order. As shown in the figure, frames F1 to F11 are played in sequence. Frames F5 and F10 represent frames being played at the synchronization timing. The bottom shows each frame of the video being played on the exhibitor's terminal in chronological order. In the example shown in the figure, there is a lag between the playback timing of the user terminal and the video, and while frame F5 is being played on the user terminal, frame F5 has not yet been played on the exhibitor's terminal. Therefore, as a synchronization process, the next frame F6 is played in sync. In this case, frame F5 is omitted from being played on the exhibitor's terminal. After that, a lag occurs between the playback timings of the two, and this time, frame F10 is being played on the user terminal, while frame F11 is already being played on the exhibitor terminal. Therefore, as a synchronization process, the next frame F11 is played in sync. In this case, frame F11 is played again on the exhibitor terminal. In this way, the playback position of the video on the user terminal is acquired at the synchronization timing, and this is reflected in the playback position of the video on the exhibitor terminal. By doing this, it is possible to synchronize the playback to a degree that does not cause discomfort, while allowing for a slight delay in the playback of the videos on both terminals.

[0057] G. Processing of Data: 11 is a flowchart of the information provision process. This process provides exhibitors with information such as the history of visits to their booths. This process is mainly executed by the information provision unit 123 (see FIG. 1), and in terms of hardware, it is mainly executed by the server 100. When the process starts, the server 100 creates a list of visitors who visited the exhibitor's booth (step S51). Then, information on the visit of all visitors is compiled (step S52). For example, the time spent at the booth, whether or not materials were downloaded and the type of materials downloaded, the time an operator guided the visitors, etc. can be compiled. Other information may also be compiled.

[0058] The server calculates interest points for each user based on this information (step S53). The interest points are an index that indicates the user's interest in the exhibitor's products or services. For example, each of the various pieces of information collected in step S52 may be converted into points using a predetermined calculation formula or the like, and the interest points may be calculated by adding them up. Alternatively, the interest points may be calculated using a learning model obtained by performing machine learning using the various pieces of information collected in step S52 and the results of business negotiations with users as learning data. The calculation of the interest points may be omitted, or other analyses may be performed.

[0059] The server outputs the tabulated results thus obtained (step S54). An output example is shown in the figure. In this example, the tabulated results are output in the form of a list, but the output can be in various other formats. The tabulated results may also be sorted by some key before being output. The collected data can be used by exhibitors for their marketing and sales activities, and can also be used to improve booth configurations and materials.

[0060] H. Effects and Modifications: According to the virtual exhibition system of the embodiment described above, it is possible to guide the user while checking the user's image, and it is possible to realize guidance that is in line with the user's interests. In addition, it is also possible to realize effective guidance by having the operator control the user's position and line of sight. It is not necessary to have all of the various features described above, and some may be omitted or combined. Furthermore, the present invention can be configured in various modified forms other than the system for holding an exhibition in a virtual space. For example, if the virtual exhibit is a landscape of a tourist spot instead of a booth, the system can be configured as a sightseeing system in a virtual space. In addition, it is possible to provide various kinds of virtual exhibits such as art galleries and museums. [Industrial Applicability]

[0061] The present invention can be used to exhibit a virtual space to a user. [Explanation of symbols]

[0062] 100 Servers 101 Virtual Data Storage Unit 102 User data storage unit 103 Exhibitor data storage unit 104 Document Data Storage Unit 111 Virtual Space Allocation Department 112 Data Management Department 113 Transmitter / receiver 121 Exhibitor display control unit 122 Guidance information input section 123 Information Provision Department 124 Push-type information unit 131 User display control unit 132 Position setting section 133 Visibility information input unit 134 Document Distribution Department 200 Exhibitor terminals 201 Goggles 211 Transmitter / receiver 212 Input / output section 213 Applications 300 User terminals 301 Goggles 311 Transmitter / receiver 312 Input / output section 313 Applications

Claims

1. A virtual exhibition system that performs an electronically constructed virtual exhibition in a virtual space, comprising: a virtual data storage unit that stores virtual data representing the virtual space and the virtual exhibition; a visibility information input unit for inputting visibility information that specifies a visibility in the virtual space of a user viewing the virtual exhibit; a user display control unit that uses the virtual data based on the visibility information to display an image of the virtual exhibition on a user output device used by the user; a guide display control unit that causes a guide output device used by a guide of the virtual exhibition to display the same image as the image to be displayed on the user image display device; a guide information input unit that receives, from the guide, guide information to be provided to the user; An output control unit that outputs the guidance information to the user output device.

2. 2. The virtual exhibition system according to claim 1, The user display control unit generates a two-dimensional image of the virtual exhibit based on the visibility information, A virtual exhibition system in which the guide display control unit performs the display using the two-dimensional image generated by the user display control unit, without generating an image based on the field of view information.

3. 2. The virtual exhibition system according to claim 1, the user display control unit is capable of displaying a moving image on the user output device using moving image data designated by the guide, The guide display control unit intermittently acquires playback position information indicating what time of the video data is being played by the user display control unit, and displays a video on the guide output device using the video data specified by the guide, while synchronizing with the playback position information at the time of acquisition.This is a virtual exhibition system.

4. A virtual exhibition system that performs an electronically constructed virtual exhibition in a virtual space, comprising: a virtual data storage unit that stores virtual data representing the virtual space and the virtual exhibition; a position setting unit that sets a position of a user in the virtual space in response to an instruction from the user viewing the virtual exhibition; a visibility information input unit for inputting visibility information that specifies the visibility of a user viewing the virtual exhibit at the position in the virtual space; a user display control unit that uses the virtual data based on the visibility information to display an image of the virtual exhibition on a user output device used by the user; A virtual exhibition system comprising a push-type guidance unit that controls the position setting by the position setting unit and / or the visibility information based on instructions from a guide of the virtual exhibition or on data prepared in advance.

5. A virtual exhibition system according to claim 1 or 4, the virtual data storage unit stores guide image data for displaying an image representing the guide, The user display control unit is capable of displaying an image of the guide based on the guide image data, A virtual exhibition system in which the display position of the image of the guide is restricted to a predetermined position set in advance within the virtual space.

6. 6. The virtual exhibition system according to claim 5, the predetermined position is set so that the image of the guide can be displayed within the field of view of the user regardless of the position of the user, A virtual exhibition system in which only one image of the guide is displayed in the field of view.

7. A virtual exhibition system according to claim 1 or 4, A virtual exhibition system in which the user's position within the virtual space is restricted to a preset position.

8. A virtual exhibition system according to claim 1 or 4, A virtual exhibition system in which the virtual space is individually assigned to one user or to a specific group of users.

9. A virtual exhibition system according to claim 1 or 4, a material distribution unit that, in response to an instruction from the user, causes the user to acquire material data prepared by the guide as a distribution material in a state in which the user can freely use the material; and an information providing unit that provides the guide with information about which of the users has acquired the material data.

10. A virtual exhibition system according to claim 1 or 4, and an information providing unit that provides the guide with the time that the user stayed in the virtual space.

11. A virtual exhibition system according to claim 1 or 4, the virtual data storage unit stores a plurality of virtual data; The user display control unit is a virtual exhibition system that performs display using virtual data selected by the user from the plurality of virtual data.

12. A virtual exhibition method for performing an electronically constructed virtual exhibition in a virtual space, comprising: The steps that the computer implements are: storing virtual data representing the virtual space and the virtual exhibit; inputting visibility information that identifies a visibility in the virtual space of a user viewing the virtual exhibit; displaying an image of the virtual exhibition on a user output device used by the user using the virtual data based on the visibility information; displaying the same image as the image to be displayed on the image display device for users on a guide output device used by a guide of the virtual exhibition; receiving guidance information to be provided to the user from the guide; and outputting the guidance information to the user output device.

13. A virtual exhibition method for performing an electronically constructed virtual exhibition in a virtual space, comprising: The steps that the computer implements are: storing virtual data representing the virtual space and the virtual exhibit; setting a position of a user within the virtual space in response to an instruction from the user viewing the virtual exhibit; inputting visibility information that identifies a visibility of a user viewing the virtual exhibit at the location in the virtual space; displaying an image of the virtual exhibition on a user output device used by the user using the virtual data based on the visibility information; and when an instruction is given by a guide of the virtual exhibition, setting the position according to the instruction from the guide regardless of an instruction from the user.

Citation Information

Patent Citations

  • Spatial information management device

    JP2020126455A

  • Processing apparatus, processing method, and processing program

    JP2022113022A

  • Virtual exhibition system

    JP2022133109A

  • Method of distributing content, program of distributing content, and system of distributing content

    JP2022164988A

  • Virtual sightseeing system and communication terminal

    JP2023048271A

Cited By

  • Virtual exhibition system

    JP2025172292A

  • Exhibition support system and service processing method for the exhibition support system

    JP2026164287A

  • Exhibition support system and method for processing questionnaires for the exhibition support system

    JP2026164290A

  • Virtual display system

    JP7713259B1