Presentation programs, presentation methods, presentation devices, and presentation systems

Abstract

We provide a presentation program that can enhance audience engagement during presentations. [Solution] The program causes a computer to perform a display step, which displays scenes of a presentation video composed of multiple scenes arranged in chronological order in a virtual space using a head-mounted display, based on first presentation data, and a switching step, which switches between scenes. At least one of the multiple scenes includes multiple elements with different representation formats. The first presentation data includes multiple scene information, each defining one of the multiple scenes. The scene information includes element information for each element that controls the display of the element.

Description

[Technical Field]

[0001] The present disclosure relates to a presentation program, a presentation method, a presentation apparatus, and a presentation system. [Background Art]

[0002] The display control apparatus described in Patent Document 1 includes a first display control unit, a second display control unit, and an operation input information acquisition unit. The first display control unit displays, at a first position in a virtual reality space, a page currently displayed in real space among a first material that is presentation material used by a presenter when giving a presentation. The second display control unit performs control to display a second material at a second position different from the first position in the virtual reality space. The operation input information acquisition unit acquires first operation input information, which is operation input information regarding display of the second material, input by an audience experiencing the virtual reality space. The second display control unit changes a display mode of the second material based on the first operation input information. [Prior Art Document] [Patent Document]

[0003] [Patent Document 1] Japanese Patent No. 7226836 [Summary of Invention] [Problem to be Solved by Invention]

[0004] However, the display control apparatus described in Patent Document 1 merely displays two-dimensional material in the virtual reality space. Therefore, there are cases where the audience does not have a sufficient sense of immersion in the presentation.

[0005] Accordingly, an object of the present disclosure is to provide a presentation program, a presentation method, a presentation apparatus, and a presentation system that can improve the audience's sense of immersion in a presentation. [Means for solving the problem]

[0006] According to this disclosure, a presentation program can be provided in which a computer performs a display step of displaying scenes of a presentation video, which is composed of multiple scenes arranged in chronological order, in a virtual space using a head-mounted display based on first presentation data, and a switching step of switching between the scenes, wherein at least one of the multiple scenes includes multiple elements with different representation formats, the first presentation data includes multiple scene information defining each of the multiple scenes, and the scene information includes element information for each element that controls the display of the element.

[0007] According to this disclosure, a presentation method can be provided that includes a display step of displaying scenes of a presentation video, which is composed of a plurality of scenes arranged in chronological order, in a virtual space using a head-mounted display based on first presentation data, and a switching step of switching between the scenes, wherein at least one of the plurality of scenes includes a plurality of elements with different representation formats, the first presentation data includes a plurality of scene information defining each of the plurality of scenes, and the scene information includes information for each element that controls the display of the element.

[0008] According to this disclosure, a presentation device can be provided comprising: a display control unit that displays scenes of a presentation video composed of multiple scenes arranged in chronological order in a virtual space using a display unit of a head-mounted display based on first presentation data; and a switching control unit that switches between the scenes, wherein at least one of the multiple scenes includes multiple elements with different representation formats, the first presentation data includes multiple scene information defining each of the multiple scenes, and the scene information includes information for controlling the display of each element.

[0009] According to this disclosure, a presentation system can be provided comprising: a storage unit for storing first presentation data and second presentation data; a communication unit for transmitting the first presentation data to a head-mounted display and the second presentation data to a management terminal, wherein the head-mounted display displays scenes of a presentation video composed of a plurality of scenes arranged in chronological order in a virtual space based on the first presentation data, at least one of the plurality of scenes includes a plurality of elements with different representation formats, the first presentation data includes a plurality of scene information defining each of the plurality of scenes, the scene information includes information for controlling the display of each element, the second presentation data includes information for managing the presentation, and the management terminal operates based on the second presentation data and transmits switching instruction information to the head-mounted display instructing it to switch scenes. [Effects of the Invention]

[0010] According to this disclosure, it is possible to improve audience engagement with presentations. [Brief explanation of the drawing]

[0011] [Figure 1] This figure shows an example configuration of a presentation system according to one embodiment of the present invention. [Figure 2] This is a conceptual diagram showing the transitions in the presentation video according to the same embodiment. [Figure 3] This is a perspective view showing an example of a scene according to the same embodiment. [Figure 4] This figure shows an example of a scene according to the same embodiment. [Figure 5] This figure shows another example of a scene according to the same embodiment. [Figure 6] This figure shows yet another example of a scene according to the same embodiment. [Figure 7] (a) and (b) are diagrams showing yet another example of a scene according to the same embodiment. [Figure 8] This is a block diagram showing an example configuration of a database server according to the same embodiment. [Figure 9] This block diagram shows an example configuration of a management terminal according to the same embodiment. [Figure 10] This is a block diagram showing an example configuration of a head-mounted display according to the same embodiment. [Figure 11] (a) is a diagram showing an example of the configuration of the first presentation data according to the same embodiment. (b) is a diagram showing an example of scene information according to the same embodiment. (c) is a diagram showing another example of scene information according to the same embodiment. [Figure 12] This figure shows an example of the transition of scene information according to the same embodiment. [Figure 13] This is a flowchart showing an example of a presentation method according to the present invention. [Figure 14] This flowchart shows an example of the presentation management process and presentation process according to the same embodiment. [Modes for carrying out the invention]

[0012] Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. In the present specification and drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and duplicate explanations are omitted.

[0013] FIG. 1 is a diagram showing a configuration example of a presentation system SYS according to an embodiment of the present invention. As shown in FIG. 1, the presentation system SYS includes a presentation data creation device 100, a database server 200, a management terminal 300, and one or more head-mounted displays 400. Hereinafter, a head-mounted display may be referred to as "HMD". The head-mounted display 400 corresponds to an example of the "presentation device" in the present disclosure. The database server 200 corresponds to an example of the "server" in the present disclosure.

[0014] The presentation data creation device 100, the database server 200, the management terminal 300, and the HMD 400 are connected to a network NW. The network NW includes, for example, one or more of the Internet, a LAN (Local Area Network), a public telephone network, a closed network, and a short-range wireless network.

[0015] The presentation data creation device 100 creates presentation data. The database server 200 stores the presentation data. The management terminal 300 is operated by a presenter. The HMD 400 is worn by the audience of the presentation. For example, a certain HMD 400 and the management terminal 300 may belong to group G1, and another HMD 400 may belong to another group G2. Group G1 is a group in which the HMD 400 is arranged at the same location as the management terminal 300. Group G2 is a group arranged at a location remote from (for example, a remote location from) the location where the management terminal 300 is arranged.

[0016] The management terminal 300 and HMD400 download presentation data from the database server 200. The management terminal 300 manages the presentation based on the presenter's operations and the presentation data. The HMD400 projects a virtual space into the audience's field of view. Specifically, the HMD400 displays the presentation video IM in the virtual space based on the presentation data.

[0017] Figure 2 is a conceptual diagram of a presentation video (IM). As shown in Figure 2, the presentation video (IM) is composed of multiple scenes (SC) arranged in chronological order. At least one of the multiple scenes (SC) contains multiple elements (EL) with different representation formats (Figure 3). The representation format indicates the representation form. A scene (SC) is a three-dimensional spatial image.

[0018] Figure 3 is a perspective view showing an example of a scene SC of a presentation video IM in the virtual space VS. In Figure 3, the presenter P1 and audience P2 are shown with dashed lines for ease of understanding, although they are not actually displayed on the HMD400.

[0019] As shown in Figure 3, Scene SC includes multiple elements EL with different representation formats. Therefore, according to this embodiment, audience immersion in the presentation can be improved compared to when only two-dimensional materials are displayed in the virtual space VS.

[0020] In the example shown in Figure 3, the multiple element EL includes a first element 510, a first element 610, a second element 520, and a third element 530.

[0021] The first elements 510 and 610 are two-dimensional objects (planar images). The first elements 510 and 610 may include, for example, one or more of the following: text, still images, and videos.

[0022] The first element 510 contains the main information of the presentation. The first element 510 may also be referred to as the main screen or main element.

[0023] The first element 610 contains supplementary information for the presentation. For example, the first element 610 contains supplementary information for the first element 510. The first element 610 may also be referred to as a subscreen or subelement.

[0024] The second element 520 is a three-dimensional object (a three-dimensional image). The form of the second element 520 may or may not change. The second element 520 is sometimes referred to as a solid.

[0025] The third element 530 is an image of the spatial environment. Specifically, the third element 530 is an image of the spatial environment in which the first elements 510, 610 and the second element 520 are arranged. In other words, the third element 530 is the base image of the spatial environment. The third element 530 contains one or more image elements. In the example in Figure 3, the third element 530 shows an image of a room and includes a table image 531 and a chair image 532. The third element 530 is also a type of background image for the first elements 510, 610 and the second element 520. Furthermore, the third element 530 is, for example, a still image spread across a virtual space VS. The third element 530 is also, for example, a graphics image. However, the third element 530 may be an image of real space captured by the HMD 400. In this case, the HMD 400 is a video see-through type. The third element 530 is sometimes referred to as a field. The HMD400 may also be an optical see-through type. In this case, the HMD400 does not display the third element 530 in the virtual space VS, but instead displays the world of the virtual space VS (for example, the first elements 510, 610 and the second element 520) superimposed on the world of the real space.

[0026] As described above with reference to Figure 3, according to this embodiment, scene SC includes first elements 510, 610, second element 520, and third element 530. Therefore, the audience can receive a presentation that makes full use of element EL in various forms, as if they were actually present in the virtual space VS via the HMD400 (as shown by audience P2 with the dashed line). As a result, the audience's sense of immersion in the presentation can be further enhanced. For example, the audience can not only watch, but also "experience" the content of the presentation via presentation video IM in the virtual space VS.

[0027] Furthermore, according to this embodiment, the virtual space VS (scene SC) is shared among multiple audiences by multiple HMDs 400. For example, the scene SC is shared in a manner similar to that of multiple audiences P2, as shown by the dashed line. In this case, for example, an HMD 400 belonging to group G1 may display the avatar 620 of an audience member wearing an HMD 400 belonging to group G2 in the virtual space VS. In this case, the HMD 400 can provide a more immersive presentation. Similarly, an HMD 400 belonging to group G2 may display the avatar 620 of an audience member wearing an HMD 400 belonging to group G1 in the virtual space VS.

[0028] Scene SC only needs to include at least one element EL from among the first element 510, the first element 610, the second element 520, and the third element 530. For example, Scene SC does not need to include the first element 610.

[0029] Next, with reference to Figures 4 to 7, we will illustrate the scene SCs displayed by the HMD400. Figures 4 to 7 show examples of scene SCs. As shown in Figures 4 to 7, the HMD400 displays each scene SC in the virtual space VS. Each scene SC in Figures 4 to 7(a) contains multiple element ELs. Figure 7(b) contains an element EL.

[0030] As shown in Figure 4, the multiple element EL includes the first element 510, 610, the second element 520, and the third element 530. In this way, according to this embodiment, the presentation scene SC is constructed by combining two-dimensional objects, three-dimensional objects, and images of the spatial environment. As a result, the audience's immersion in the presentation can be further enhanced.

[0031] As shown in Figure 5, the multiple element EL includes a first element 610, a second element 520, a third element 530, and a fourth element 540. The fourth element 540 is the image of another virtual space VSa connected to the virtual space VS. The image of the virtual space VSa may be, for example, a graphics image or an image based on camera footage.

[0032] The fourth element 540 includes a frame 541. Beyond the frame 541 lies the virtual space VSa. In other words, the virtual space VSa contains a world different from the virtual space VS, and various objects 542 and 543 are displayed there. Furthermore, the HMD400 can control the display of objects 542 so that objects 542 existing in the virtual space VSa enter the virtual space VS.

[0033] Thus, according to this embodiment, including the fourth element 540 in scene SC can further enhance audience immersion in the presentation. The fourth element 540 may be described as a portal or window.

[0034] As shown in Figure 6, the multiple element EL includes a second element 520, a third element 530, and a fifth element 550. The fifth element 550 is a visual effect that is dynamically displayed across the entire display area of ​​the virtual space VS. The visual effect is a visual effect that creates a spatial environment in the virtual space VS (a visual effect for the spatial environment). In other words, the visual effect shows the atmosphere of the virtual space VS. The visual effect is not limited to, for example, a visual effect of falling particles such as snow, a visual effect of rain, or a visual effect of changing brightness.

[0035] Thus, according to this embodiment, including the fifth element 550 in scene SC can further enhance audience immersion in the presentation. The fifth element 550 may sometimes be referred to as an effect.

[0036] As shown in Figure 7(a), the second element 520a is displayed. The second element 520a shows the exterior of a three-dimensional object (e.g., a building). On the other hand, as shown in Figure 7(b), the second element 520b is a three-dimensional object (e.g., the interior of the building) that shows the interior of the second element 520a in Figure 7(a).

[0037] Thus, in this embodiment, the HMD400 moves the viewpoint from the outside to the inside of the three-dimensional object represented by the second element 520a, and displays the inside of the three-dimensional object by the second element 520b. In this way, the viewpoint moves from the outside to the inside of the second element 520a, and displays the inside of the second element 520a (second element 520b). As a result, the audience's immersion in the presentation can be further enhanced.

[0038] You can move the viewpoint within the same scene (SC), or you can switch scenes (SC) to move the viewpoint.

[0039] Next, the database server 200 will be described with reference to Figure 8. Figure 8 is a block diagram showing an example configuration of the database server 200. As shown in Figure 8, the database server 200 comprises a processing unit 210, a storage unit 220, and a communication unit 230.

[0040] The processing unit 210 performs various processes (various calculations). The processing unit 210 controls the storage unit 220 and the communication unit 230. The processing unit 210 includes one or more processors. The processors are CPUs (Central Processing Units), GPUs (Graphics Processing Units), FPGAs (Field Programmable Gate Arrays), DSPs (Digital Signal Processors), or ASICs (Application Specific Integrated Circuits). The processors may be operated by computer programs or by hardwired logic.

[0041] The storage unit 220 includes one or more storage devices and stores data and computer programs. The storage unit 220 includes a main memory device such as semiconductor memory and an auxiliary storage device such as semiconductor memory and a storage drive. The storage drive is, for example, a hard disk drive or a solid-state drive. The storage unit 220 is, for example, a non-temporary computer-readable storage medium.

[0042] The communication unit 230 is connected to a network NW. The communication unit 230 communicates with various devices connected to the network NW. The communication unit 230 is a communication device that communicates according to a predetermined communication protocol, and includes, for example, a network interface controller. The predetermined communication protocol is, for example, a protocol compliant with Ethernet®, the Internet Protocol Suite, and a protocol compliant with a short-range wireless communication standard.

[0043] The communication unit 230, under the control of the processing unit 210, receives presentation data PD from the presentation data creation device 100 (Figure 1). In other words, the presentation data creation device 100 creates the presentation data PD. Specifically, the presentation data creation device 100 includes one or more computers. For example, the presentation data creation device 100 includes a processing unit, a storage unit, and a communication unit. The hardware configurations of the processing unit, storage unit, and communication unit are the same as those of the processing unit 210, storage unit 220, and communication unit 230, respectively. The presentation data creation device 100 is equipped with an integrated development environment (IDE) and creates the presentation data PD based on the operation of one or more operators. The integrated development environment is, for example, "Unity".

[0044] The storage unit 220 stores one or more presentation data PDs under the control of the processing unit 210. Each of the presentation data PDs includes a first presentation data FD and a second presentation data SD.

[0045] The first presentation data FD contains multiple scene information 10. Each of the multiple scene information 10 defines multiple scene SCs. The scene information 10 typically includes image information and control information for displaying the scene SCs on the HMD400.

[0046] The second presentation data SD contains information for managing the presentation. For example, the second presentation data SD contains multiple scene information 90. Each of the multiple scene information 90 contains information for managing multiple scene SCs. The scene information 90 typically includes image information and control information for displaying the management video of the scene SC on the management terminal 300.

[0047] The communication unit 230, under the control of the processing unit 210, transmits the first presentation data FD to the HMD400 in response to a request from the HMD400. The communication unit 230, also under the control of the processing unit 210, transmits the second presentation data SD to the management terminal 300 in response to a request from the management terminal 300.

[0048] Next, the management terminal 300 will be described with reference to Figure 9. The management terminal 300 is, for example, a personal computer (PC) or a mobile terminal. A personal computer is, for example, a tablet PC or a notebook PC. A mobile terminal is, for example, a smartphone. The management terminal 300 manages the progress of Scene SC, for example.

[0049] Figure 9 is a block diagram showing an example configuration of the management terminal 300. As shown in Figure 9, the management terminal 300 comprises a processing unit 310, a storage unit 320, and a communication unit 330. The processing unit 310 performs various processes (various calculations). The storage unit 320 stores data and computer programs. The communication unit 330 is connected to a network NW. The communication unit 330 communicates with various devices connected to the network NW. The hardware configurations of the processing unit 310, storage unit 320, and communication unit 330 are the same as those of the processing unit 210, storage unit 220, and communication unit 230 in Figure 8, respectively.

[0050] The management terminal 300 further comprises an input unit 340 and a display unit 350. The input unit 340 is an input device for inputting various information to the processing unit 310. For example, the input unit 340 includes a touch panel or a keyboard and pointing device. The display unit 350 displays various information. The display unit 350 includes, for example, a liquid crystal display or an organic electroluminescent display.

[0051] The operation will be explained further with reference to Figure 9. The processing unit 310 includes a display control unit 311, a switching instruction unit 312, a storage control unit 313, and a communication control unit 314. Specifically, the storage unit 320 stores the second computer program SPG. The processing unit 310 then functions as the display control unit 311, the switching instruction unit 312, the storage control unit 313, and the communication control unit 314 by executing the second computer program SPG. The second computer program SPG is an application program for managing presentations.

[0052] The communication control unit 314 instructs the communication unit 330 to download the second presentation data SD from the database server 200. The storage control unit 313 instructs the storage unit 320 to store the second presentation data SD. The management terminal 300 operates based on the second presentation data SD.

[0053] The display control unit 311 displays a management video for managing the presentation on the display unit 350 based on the second presentation data SD. The management video is, for example, the first management video or the second management video.

[0054] The first management video includes all or part of one or more elements (hereinafter referred to as "confirmation elements") that represent one or more elements EL included in the scene SC displayed on the HMD400. For example, the first management video includes one or more confirmation elements from among the confirmation elements representing the first element 510, the confirmation element representing the first element 610, and the confirmation element representing the second element 520. Specifically, the display control unit 311 displays the first management video including the confirmation elements on the display unit 350 based on the scene information 90. In this case, the scene information 90 includes image information and control information for the confirmation elements, corresponding to the scene SC. The control information includes information on the display position of the confirmation elements.

[0055] The second management video is the video of Scene SC displayed on the HMD400 (the video of Scene SC that the audience sees through the HMD400). In this case, the second management video includes the video of Scene SC displayed on each HMD400 through screen splitting. Specifically, the communication control unit 314 causes the communication unit 330 to acquire the video of Scene SC displayed on each HMD400 in real time. The display control unit 311 then displays the second management video, which includes the video of Scene SC displayed on each HMD400, on the display unit 350 in real time.

[0056] The display control unit 311 may receive the presenter's operation via the input unit 340 and, in response to the operation, switch between the first management video and the second management video and display them on the display unit 350.

[0057] The display control unit 311 may display basic information of the scene SC on the display unit 350 based on the scene information 90. The basic information is not particularly limited, but for example, it may include a list and functions of the element ELs that constitute the scene SC, and an overview of the scene SC.

[0058] The switching instruction unit 312 receives a scene SC switching instruction from the presenter via the input unit 340. The switching instruction unit 312 then controls the communication unit 330 to transmit the switching instruction information to each HMD 400. As a result, the communication unit 330 transmits switching instruction information to each HMD 400 instructing it to switch the scene SC. Each HMD 400 switches the scene SC in response to receiving the switching instruction information. The display control unit 311 also switches the currently displayed management video to the management video corresponding to the next scene SC in response to the switching instruction unit 312 receiving the scene SC switching instruction from the presenter.

[0059] Next, the HMD400 will be described with reference to Figure 10. Figure 10 is a block diagram showing an example configuration of the HMD400. As shown in Figure 10, the HMD400 comprises a processing unit 410, a storage unit 420, and a communication unit 430. The processing unit 410 performs various processes (various calculations). The storage unit 420 stores data and computer programs. The communication unit 430 is connected to a network NW. The communication unit 430 communicates with various devices connected to the network NW. The hardware configuration of the processing unit 410, storage unit 420, and communication unit 430 is the same as the hardware configuration of the processing unit 210, storage unit 220, and communication unit 230 in Figure 8.

[0060] The HMD400 further comprises an input unit 440, a display unit 450, an audio input unit 460, an audio output unit 470, and an imaging unit 380. The input unit 440 is an input device for inputting various information to the processing unit 410. The input unit 440 is, for example, various switches. The display unit 450 displays presentation video IM for each scene SC. The display unit 450 includes, for example, two displays corresponding to the left and right eyes in order to realize stereoscopic vision. Each display is, for example, an organic electroluminescent display or a liquid crystal display. The audio input unit 460 accepts audio input. The audio input unit 460 includes, for example, a microphone. The audio output unit 470 outputs audio. The audio output unit 470 includes, for example, left and right speakers. The imaging unit 480 captures images of subjects. The imaging unit 480 includes, for example, one or more cameras (for example, visible light cameras). The cameras include, for example, image sensors.

[0061] The HMD400 further comprises a sensor unit 490. The sensor unit 490 detects the state of the audience wearing the HMD400 and the surrounding environment. The sensor unit 490 includes an eye-tracking sensor 491, a face-tracking sensor 492, a spatial recognition sensor 493, a depth sensor 494, and an inertial measurement unit (IMU) 495. The eye-tracking sensor 491 detects the state of the audience's eyes. For example, the eye-tracking sensor 491 tracks eye movements, detects the direction of gaze, and detects blinking. The eye-tracking sensor 491 includes, for example, an infrared camera and an infrared light-emitting diode for each eye. The face-tracking sensor 492 detects the state of the audience's face. For example, the face-tracking sensor 492 tracks mouth and jaw movements, detects changes in facial expression, and detects speech. The face-tracking sensor 492 includes, for example, an infrared camera and an infrared light-emitting diode. The spatial recognition sensor 493 primarily performs wide-range three-dimensional measurements. For example, the spatial recognition sensor 493 detects objects in real space over a wide range and acquires wide-range three-dimensional information. The spatial recognition sensor 493 is, for example, a LiDAR (Light Detection and Ranging) sensor. The depth sensor 494 primarily performs short-range three-dimensional measurements. For example, the depth sensor 494 recognizes hand gestures, detects objects in real space at close range, and acquires short-range three-dimensional information. The depth sensor 494 includes, for example, an infrared projector and an infrared camera. The inertial measurement unit 495 detects the posture and movement of the audience. For example, the inertial measurement unit 495 detects the orientation and movement of the HMD 400 and performs head tracking. The inertial measurement unit 495 includes, for example, a gyroscope, an accelerometer, and a geomagnetic sensor.

[0062] The HMD400 is not limited to any particular type; for example, it can be a goggle type, glasses type, helmet type, or hybrid type. The hybrid type combines the features of both goggles and glasses.

[0063] The operation will now be explained with reference to Figure 10. The processing unit 410 includes a display control unit 411, a display switching unit 412, an audio control unit 413, a storage control unit 414, and a communication control unit 415. Specifically, the storage unit 420 stores the first computer program FPG. The processing unit 410 then functions as the display control unit 411, the display switching unit 412, the audio control unit 413, the storage control unit 414, and the communication control unit 415 by executing the first computer program FPG. The first computer program FPG is an application program for executing a presentation.

[0064] The communication control unit 415 instructs the communication unit 430 to download the first presentation data FD from the database server 200. The storage control unit 414 instructs the storage unit 420 to store the first presentation data FD.

[0065] The display control unit 411 displays the presentation video IM on the display unit 450 based on the first presentation data FD. Specifically, the display control unit 411 displays scene SC on the display unit 450 based on scene information 10.

[0066] Scene information 10 includes element information ELX for each element EL included in scene SC, which controls the display of the element EL. Therefore, according to this embodiment, the display in the virtual space VS can be controlled for each element EL with different representation formats by each element information ELX. As a result, a wide variety of effects can be realized in the virtual space VS in each scene SC, or in consecutive scenes SC. As a result, the audience's sense of immersion in the presentation can be further enhanced.

[0067] For example, by displaying product-representing element ELs in a virtual space VS at actual size or exaggerated size, the audience can recognize the details of the product. For example, advanced or complex products and processes can be presented through animation and 3D display of element ELs to deepen the audience's understanding. For example, by incorporating entertainment elements into element ELs with different forms of representation, the sense of immersion can be further enhanced.

[0068] Figure 11(a) shows an example of the configuration of the first presentation data FD. As shown in Figure 11(a), the first presentation data FD includes multiple scene information 10. Each of the scene information 10 includes one or more element information ELX. Specifically, the scene information 10 includes element information ELX for each element EL that constitutes the scene SC. The element information ELX includes image information and control information for the element EL. The control information includes information on the display position of the element EL.

[0069] Furthermore, it is possible to attach audio information AU to each element information ELX. In this case, the audio control unit 413 outputs sound by setting a virtual sound source at the display position of element EL indicated by the element information ELX to which the audio information AU is attached. In other words, the audio control unit 413 adjusts the volume and phase of the sound output from the audio output unit 470 (left and right speakers) based on the audio information AU, thereby recreating the sound field as if the sound were emanating from a virtual sound source set at the display position of element EL. As a result, by stimulating not only the audience's visual sense but also their auditory sense, the audience's immersion in the presentation can be further enhanced.

[0070] Furthermore, for example, different audio information AU may be attached to element information ELX, which represents each element EL with a different representation format. In this case, the audio control unit 413 can output audio to the audio output unit 470 based on different audio information AU for each element EL with a different representation format. In other words, the audio control unit 413 can reproduce a sound field as if different sounds were being generated from virtual sound sources set at the respective display positions of different element ELs, based on each audio information AU. As a result, auditory stimulation is further enhanced, and the audience's immersion in the presentation can be further improved.

[0071] Furthermore, two scene information 10 defining two adjacent scenes SC on the time axis TX may include element information ELX that indicates the same element EL. In other words, the same element EL may be registered as a control target in two scene information 10 defining two adjacent scenes SC on the time axis TX. In this case, the display control unit 411 changes the display form of the same element EL in the virtual space VS so that it is spatially continuous from the previous scene SC in the later scene SC on the time axis TX. As a result, according to this embodiment, the visual sense is further stimulated, and the audience's immersion in the presentation can be further improved. For example, the display control unit 411 causes the display unit 450 to display the element EL in the first display mode in the second display mode. For example, the first display mode is a mode in which the element EL is displayed in a static state, and the second display mode is a mode in which the element EL is displayed with animation effects. Animation effects include, for example, decomposition, enlargement, reduction, rotation, deformation, change in transparency, or change in color. For example, the first display mode shows the external appearance of element EL, and the second display mode shows the internal structure of element EL (Figure 7).

[0072] Furthermore, in the element information ELX, it is possible to set either a first viewpoint format or a second viewpoint format for each element EL. In other words, for each element information ELX, it is possible to set either a first viewpoint format or a second viewpoint format that indicates the viewpoint format of the element EL. For example, it is possible to set either a first viewpoint format or a second viewpoint format for each element EL with a different representation format.

[0073] The first viewpoint format refers to a format in which the element EL is displayed at the same viewpoint on multiple HMD400s when the virtual space VS is shared by multiple HMD400s (e.g., subjective viewpoint). The second viewpoint format refers to a format in which the element EL is displayed at different viewpoints depending on the viewpoint position of each of the multiple HMD400s in the virtual space VS (e.g., overhead viewpoint). According to this embodiment, the first viewpoint format or the second viewpoint format can be selected depending on the content to be presented to the audience by the element EL. As a result, the audience's sense of immersion in the presentation can be further improved.

[0074] Figure 11(b) shows an example of scene information 10. As shown in Figure 11(b), scene information 10 includes multiple element information ELX. The multiple element information ELX includes first element information 11, first element information 21, second element information 12, and third element information 13. First element information 11 defines first element 510 (Figure 4). First element information 21 defines first element 610 (Figure 4). Second element information 12 defines second element 520 (Figure 4). Third element information 13 defines third element 530 (Figure 4). Some or all of the first element information 11, first element information 21, second element information 12, and third element information 13 may be accompanied by audio information AU.

[0075] Based on the scene information 10, the display control unit 411 causes the display unit 450 to display scene SC, which includes the first elements 510 and 610, the second element 520, and the third element 530.

[0076] Scene information 10 may include one of the first element information 11, first element information 21, second element information 12, and third element information 13, or it may include any combination of two or more. Also, in scene information 10, the number of first element information 11s, first element information 21s, second element information 12s, and third element information 13s may be one or two or more.

[0077] Figure 11(c) shows another example of scene information 10. As shown in Figure 11(c), scene information 10 includes multiple element information ELX. The multiple element information ELX includes first element information 11, first element information 21, second element information 12, third element information 13, fourth element information 14, and fifth element information 15. The fourth element information 14 defines the fourth element 540 (Figure 5). The fifth element information 15 defines the fifth element 550 (Figure 6). Audio information AU may be attached to some or all of the first element information 11, first element information 21, second element information 12, third element information 13, fourth element information 14, and fifth element information 15.

[0078] Based on the scene information 10, the display control unit 411 causes the display unit 450 to display scene SC, which includes the first elements 510, 610, the second element 520, the third element 530, the fourth element 540, and the fifth element 550.

[0079] Scene information 10 may include one of the first element information 11, first element information 21, second element information 12, third element information 13, fourth element information 14, and fifth element information 15, or it may include any combination of two or more. Also, in scene information 10, the number of first element information 11, first element information 21, second element information 12, third element information 13, fourth element information 14, and fifth element information 15 may be one or two or more.

[0080] Next, the transition of scene information 10 will be explained with reference to Figure 12. Figure 12 is a diagram showing an example of the transition of scene information 10. As shown in Figure 12, the first presentation data FD contains N scene information 10-1 to 10-N, each defining N consecutive scenes SC on the time axis TX. "N" represents an integer.

[0081] Scene information 10-1 includes first element information 11, first element information 21, second element information 12, and third element information 13, which are associated with audio information AU. Based on the scene information 10-1, the display control unit 411 of the HMD400 displays the first element 510 (e.g., main screen), the first element 610 (e.g., subscreen), the second element 520 (e.g., solid), and the third element 530 (e.g., field) in the virtual space VS (e.g., Figure 4).

[0082] Scene information 10-2 includes first element information 21, second element information 12 with attached audio information AU, third element information 13, and fourth element information 14 with attached audio information AU. Based on the scene information 10-2, the display control unit 411 of the HMD400 displays the first element 610, second element 520, third element 530, and fourth element 540 (e.g., a portal) in the virtual space VS (e.g., Figure 5).

[0083] Scene information 10-3 includes two different second element information 12, third element information 13, and fifth element information 15 with attached audio information AU. One of the second element information 12 is attached with audio information AU. The display control unit 411 of the HMD400 displays two different second elements 520, third element 530, and fifth element 550 (e.g., effects) in the virtual space VS based on the scene information 10-3 (e.g., Figure 6).

[0084] Then, the display control unit 411 of the HMD400 displays each scene SC on the display unit 450 based on the scene information 10-4 to 10-(N-1).

[0085] Scene information 10-N includes first element information 11 with attached audio information AU, two different first element information 21, second element information 12, and third element information 13. Based on the scene information 10-N, the display control unit 411 of the HMD400 displays the first element 510, two different first elements 610, second element 520, and third element 530 in the virtual space VS.

[0086] Furthermore, the audio control unit 413 outputs audio corresponding to the audio information AU to the audio output unit 470 in each scene SC.

[0087] Next, the presentation method will be described with reference to Figures 13 and 14. Figure 13 is a flowchart of the presentation method according to this embodiment. The presentation method includes a first presentation method performed by the HMD400 and a second presentation method performed by the management terminal 300. The first presentation method is performed by each HMD400 in Figure 1.

[0088] As shown in Figure 13, the first presentation method includes steps S1 to S5. The first computer program FPG stored in the memory unit 420 of the HMD400 causes the processing unit 410 to execute steps S1 to S5. In other words, the computer program product realizes steps S1 to S5 when the first computer program FPG is executed by the processing unit 410. The first computer program FPG corresponds to an example of a “presentation program” in this disclosure. The processing unit 410 corresponds to an example of a “computer” in this disclosure.

[0089] On the other hand, the second presentation method includes steps S11 to S14. The second computer program SPG stored in the storage unit 320 of the management terminal 300 causes the processing unit 310 to execute steps S11 to S14. In other words, the computer program product realizes steps S11 to S14 when the second computer program SPG is executed by the processing unit 310.

[0090] First, in step S11 of the management terminal 300, the communication control unit 314 instructs the communication unit 330 to download the second presentation data SD from the database server 200. The storage unit 320 stores the second presentation data SD.

[0091] Meanwhile, in step S1 of the HMD400, the communication control unit 415 causes the communication unit 430 to download the first presentation data FD from the database server 200. The storage unit 420 stores the first presentation data FD. Step S1 corresponds to an example of the "download step" of this disclosure.

[0092] Next, in step S12 of the management terminal 300, the display control unit 311 causes the display unit 350 to display a code. The code is used to set the reference position and reference direction in the virtual space VS when displaying the scene SC. The code is, for example, a two-dimensional code. The two-dimensional code is, for example, a QR code (registered trademark).

[0093] On the other hand, in step S2 of the HMD400, the display control unit 411 causes the imaging unit 480 and the depth sensor 494 to read the code displayed on the display unit 350 of the management terminal 300. Alternatively, the imaging unit 480 and the spatial recognition sensor 493 may be made to read the code, or only the imaging unit 480 may be made to read the code. Step S2 corresponds to an example of the "reading step" in this disclosure.

[0094] Next, in step S3 of the HMD400, the display control unit 311 sets the reference position and reference direction in the virtual space VS when displaying the scene SC, based on the code. Step S3 corresponds to an example of a “setting step” in this disclosure.

[0095] In particular, in step S3, the display control unit 411 recognizes the orientation (orientation) of the code in real space relative to the HMD 400 by analyzing the code pattern read by the imaging unit 480. The display control unit 411 also obtains the position of the code in real space relative to the HMD 400 from the depth sensor 494 or the spatial recognition sensor 493. Then, based on the position and orientation (orientation) of the code relative to the HMD 400, the display control unit 411 sets the reference position and reference direction in the virtual space VS when displaying the scene SC. The reference position and reference direction are fixed in the virtual space VS. For example, the display control unit 411 associates the position of the code in real space with the virtual space VS and sets the position of the code in the virtual space VS as the reference position. Alternatively, for example, the display control unit 411 sets a specific direction relative to the code as the reference direction. According to this embodiment, each HMD 400 can set a common reference position and reference direction in the virtual space VS by reading the code displayed on the management terminal 300 at its respective position. As a result, multiple HMD400s can share the virtual space VS, further enhancing immersion.

[0096] Next, in step S4 of the HMD400, the communication control unit 415, in response to the reading of the code in step S2, causes the communication unit 430 of the HMD400 to establish communication with the communication unit 430 of the management terminal 300. Therefore, according to this embodiment, the reading of the code for setting the reference position and reference direction in the virtual space VS can be used as a trigger to start establishing communication. Thus, it is not necessary to generate a dedicated trigger to start establishing communication, and the process can be simplified.

[0097] Meanwhile, in step S13 of the management terminal 300, the communication control unit 314 causes the communication unit 330 of the management terminal 300 to establish communication with the communication unit 430 of the HMD 400. Step S13 corresponds to an example of a "communication step" in this disclosure.

[0098] Next, in step S14 of the management terminal 300, the management terminal 300 executes the presentation management process. Once the presentation management process is completed, the second presentation method is terminated.

[0099] Meanwhile, in step S5 of the HMD400, the HMD400 performs presentation processing under the management of the management terminal 300. Step S5 is performed simultaneously by multiple HMD400s as shown in Figure 1. When the presentation processing is completed, the first presentation method ends.

[0100] Figure 14 is a flowchart showing the presentation processing in step S5 and the presentation management processing in step S14 of Figure 13. As shown in Figure 14, the presentation processing performed by the HMD400 includes steps S51 to S58. The presentation management processing performed by the management terminal 300 includes steps S141 to S146.

[0101] First, in step S141 of the management terminal 300, the display control unit 311 instructs the communication unit 330 to send start instruction information to the HMD400, instructing it to start the presentation.

[0102] Next, in step S142 of the management terminal 300, the display control unit 311 displays the management video on the display unit 350 based on the second presentation data SD.

[0103] Meanwhile, in step S51 of the HMD400, the display control unit 411 determines whether or not it has received presentation start instruction information from the management terminal 300. If the determination in step S51 is negative (NO), processing is delayed until the start instruction information is received. On the other hand, if the determination in step S51 is positive (YES), processing proceeds to step S52.

[0104] Next, in step S52 of the HMD400, the display control unit 411 acquires scene information 10 from the first presentation data FD.

[0105] Next, in step S53 of the HMD400, the display control unit 411 displays the scene SC in the virtual space VS using the display unit 450 based on the scene information 10. Step S53 corresponds to an example of the "display step" of this disclosure.

[0106] In particular, in step S53, the display control unit 411 places the element EL in the virtual space VS based on the reference position and reference direction (step S3) set (fixed) in the virtual space VS, and the display position information of the element EL included in the element information ELX. Also, for example, the viewpoint in the virtual space VS displayed on the HMD400 is determined for each HMD400. Specifically, the viewpoint of each HMD400 is determined based on the position and orientation (posture) of the code relative to each HMD400, and the reference position and reference direction. For example, if the orientation (posture) of the code relative to a certain HMD400 is a specific orientation (specific posture), that is, if a certain HMD400 is located in a specific direction relative to the code, the viewpoint of that HMD400 is placed in the reference direction relative to the reference position.

[0107] Next, in step S54 of the HMD400, the audio control unit 413 outputs audio via the audio output unit 470 based on the audio information AU attached to the scene information 10. Step S54 corresponds to an example of the “audio step” of this disclosure.

[0108] Next, in step S55 of the HMD400, the display control unit 411 of the HMD400 displays the avatar 620 of the audience wearing the HMD400 (Figure 1) belonging to group G2 in the virtual space VS using the display unit 450. For example, the avatar 620 is generated based on captured images obtained by capturing the audience in advance with a camera. Furthermore, the avatar 620 reflects the gaze and facial expressions of the audience acquired by the eye tracking sensor 491 and the face tracking sensor 492. Information about the avatar 620 is transmitted from the HMD400 belonging to group G2, for example, via a specific server.

[0109] Meanwhile, in step S143 of the management terminal 300, the switching instruction unit 312 determines whether or not it has received a scene SC switching instruction from the presenter via the input unit 340. If the determination in step S143 is negative (NO), the process proceeds to step S142. On the other hand, if the determination in step S143 is positive (YES), the process proceeds to step S144.

[0110] Next, in step S144 of the management terminal 300, the switching instruction unit 312 instructs the communication unit 330 to send switching instruction information to the HMD 400, instructing it to switch scene SC.

[0111] Meanwhile, in step S56 of the HMD400, the display switching unit 412 determines whether or not it has received switching instruction information from the management terminal 300. If the determination in step S56 is negative (NO), the process proceeds to step S58. On the other hand, if the determination in step S56 is positive (YES), the process proceeds to step S57.

[0112] Next, in step S57 of the HMD400, the display switching unit 412 switches the scene SC according to the switching instruction information. Specifically, the display switching unit 412 obtains the next scene information 10 from the first presentation data FD. Then, the process proceeds to step S53. Step S57 corresponds to an example of the "switching step" in this disclosure.

[0113] Meanwhile, in step S145 of the management terminal 300, the display control unit 311 determines whether or not it has received a presentation end instruction from the presenter via the input unit 340. If the determination in step S145 is negative (NO), the process proceeds to step S142. On the other hand, if the determination in step S145 is positive (YES), the process proceeds to step S146.

[0114] Next, in step S146 of the management terminal 300, the communication control unit 314 instructs the communication unit 330 to send termination instruction information to the HMD400, instructing it to end the presentation. The presentation management process then ends.

[0115] Meanwhile, in step S58 of the HMD400, the display control unit 411 determines whether or not it has received a presentation end instruction from the management terminal 300. If the determination in step S58 is negative (NO), the process proceeds to step S53. On the other hand, if the determination in step S58 is positive (YES), the presentation process ends.

[0116] As described above with reference to Figures 13 and 14, according to this embodiment, the HMD 400 is instructed to download the first presentation data FD from the database server 200 (step S1). Then, the display control unit 411 displays the scene SC in the virtual space VS using the head-mounted display 400 (display unit 450) based on the first presentation data FD (step S53). In this way, according to this embodiment, by displaying the scene SC based on the downloaded first presentation data FD, the impact of communication failures such as communication errors can be reduced, and the presentation can be carried out smoothly. In addition, since complex communication protocols are not required, communication control can be simplified.

[0117] Furthermore, according to this embodiment, the display switching unit 412 switches the scene SC in response to switching instruction information from the management terminal 300, which operates based on the second presentation data SD (step S57). In particular, the management terminal 300 transmits switching instruction information to multiple HMDs 400 simultaneously (step S144). Therefore, multiple HMDs 400 effectively synchronize to display each scene SC. As a result, according to this embodiment, a presentation in the virtual space VS can be provided to multiple audiences simultaneously. Thus, since multiple audiences can simultaneously experience and share the presentation in the virtual space VS, the synergistic effect can further enhance the audience's immersion in the presentation.

[0118] Furthermore, the management terminal 300 simultaneously transmits start instruction information to multiple HMDs 400 (step S141). The management terminal 300 also simultaneously transmits end instruction information to multiple HMDs 400 (step S146). Thus, from the start to the end of the presentation, multiple HMDs 400 can effectively synchronize and display each scene SC. Note that the start instruction information and end instruction information can be considered a type of switching instruction information. This is because displaying a scene SC from a hidden state (start) and hiding a scene SC (end) can be broadly considered as switching the scene SC.

[0119] Preferred embodiments and modifications of the present disclosure have been described in detail above with reference to the attached drawings, but the technical scope of the present disclosure is not limited to such examples. It is clear to any person with ordinary skill in the art of the present disclosure that various modifications or alterations can be conceived within the scope of the technical idea set forth in the claims, and these too are understood to fall within the technical scope of the present disclosure.

[0120] For example, in the above embodiment, the first presentation data FD was downloaded to the HMD400 before the presentation, but the HMD400 may acquire each scene information 10 of the first presentation data FD from the database server 200 in accordance with the progress of the presentation. In this case, the database server 200 also functions as a "presentation device" in this disclosure. Alternatively, for example, a management terminal 300 may download the first presentation data FD from the database server 200. The management terminal 300 may then transmit each scene information 10 of the first presentation data FD to the HMD400 in accordance with the progress of the presentation. In this case, the management terminal 300 also functions as a "presentation device" in this disclosure.

[0121] Furthermore, the devices or systems described herein may be implemented as a single device, or they may be implemented by a group of devices (e.g., cloud servers) that are partially or entirely connected by a network.

[0122] Furthermore, the series of processes performed by the apparatus described herein may be implemented using software, hardware, or a combination of software and hardware.

[0123] Furthermore, the processes described using flowcharts in this specification do not necessarily have to be executed in the order shown. Some processing steps may be executed in parallel. Additional processing steps may be adopted, and some processing steps may be omitted. For example, in Figure 14, step S55 may be omitted. Also, for example, in Figure 13, steps S12 and steps S2 and S3 may be omitted. In this case, for example, the reference position and reference direction are predetermined, and the viewpoint of the virtual space VS is set to be the same for multiple HMDs 400. Also, for example, in steps S13 and S4, the trigger for establishing communication is generated in response to the operation of the input unit 440.

[0124] Furthermore, the effects described herein are merely descriptive or illustrative and not limiting. In other words, the technology relating to this disclosure may produce other effects that will be apparent to those skilled in the art from the description herein, in addition to or instead of the effects described herein.

[0125] Furthermore, the following configurations also fall within the technical scope of this disclosure.

[0126] (Item 1) A display step in which the aforementioned scenes of a presentation video, which consists of multiple scenes arranged in chronological order, are displayed in a virtual space using a head-mounted display based on first presentation data, The computer is instructed to perform a switching step to switch between the aforementioned scenes. At least one of the aforementioned multiple scenes includes multiple elements with different forms of expression, The first presentation data includes multiple scene information that defines each of the multiple scenes, The scene information includes, for each element, element information that controls the display of the element, in a presentation program.

[0127] (Item 2) The aforementioned plurality of elements include a first element which is a two-dimensional object, a second element which is a three-dimensional object, and a third element which is an image of a spatial environment, as described in item 1, for presentation.

[0128] (Item 3) The aforementioned plurality of elements include a second element which is a three-dimensional object, The presentation program according to claim 1, wherein the display step involves moving the viewpoint from outside to inside the second element to display the inside of the second element.

[0129] (Item 4) A reading step of reading a code for setting the reference position and reference direction in the virtual space when displaying the aforementioned scene, A presentation program according to any one of items 1 to 3, which causes the computer to perform a setting step of setting the reference position and reference direction in the virtual space based on the code.

[0130] (Item 5) A presentation program according to item 4, which, in response to reading the code, causes the computer to perform a communication step that establishes communication between the head-mounted display and a management terminal that manages the progress of the scene.

[0131] (Item 6) The computer is instructed to perform a download step that causes the head-mounted display to download the first presentation data from the server. The presentation program described in any of items 1 to 5 switches the scene in the aforementioned switching step in response to switching instruction information from a management terminal that operates based on second presentation data for managing the presentation.

[0132] (Item 7) In the aforementioned scene information, a first viewpoint format or a second viewpoint format can be set for each element with a different representation format. The first viewpoint format refers to a format in which the elements are displayed at the same viewpoint on multiple head-mounted displays, including the head-mounted display, when the virtual space is shared by multiple head-mounted displays, The second viewpoint format is a presentation program according to any one of items 1 to 6, which indicates a format in which the elements are displayed from different viewpoints depending on the viewpoint position of each of the multiple head-mounted displays in the virtual space.

[0133] (Item 8) A presentation program according to any one of items 1 to 7, wherein, if two scene pieces defining two adjacent scenes on the time axis include element piece information indicating the same element, the display step displays the same element in the virtual space with a changed display form so as to be spatially continuous in a later scene on the time axis.

[0134] (Item 9) A presentation program according to any one of items 1 to 8, wherein the plurality of elements with different representation formats include a fourth element which is a video of another virtual space connected to the virtual space.

[0135] (Item 10) A presentation program according to any one of items 1 to 9, wherein the plurality of elements with different forms of expression include a fifth element which is a performance video that is dynamically displayed across the entire display area of ​​the virtual space.

[0136] (Item 11) For each of the aforementioned element information, it is possible to attach audio information to the element information. A presentation program according to any one of items 1 to 10, which causes the computer to execute an audio step that outputs sound by setting a virtual sound source at the display position of the element indicated by the element information to which the aforementioned audio information is attached.

[0137] (Item 12) A display step in which the aforementioned scenes of a presentation video, which consists of multiple scenes arranged in chronological order, are displayed in a virtual space using a head-mounted display based on first presentation data, Includes a switching step for switching the aforementioned scene, At least one of the aforementioned multiple scenes includes multiple elements with different forms of expression, The first presentation data includes multiple scene information that defines each of the multiple scenes, A presentation method in which the scene information includes information that controls the display of each element.

[0138] (Item 13) A display control unit that displays the scenes of a presentation video, which consists of multiple scenes arranged in chronological order, in a virtual space using the display unit of a head-mounted display, based on first presentation data, The system comprises a switching control unit for switching between the aforementioned scenes, At least one of the aforementioned multiple scenes includes multiple elements with different forms of expression, The first presentation data includes multiple scene information that defines each of the multiple scenes, The scene information includes information for each element that controls the display of the element, in a presentation device.

[0139] (Item 14) A storage unit for storing first presentation data and second presentation data, The system includes a communication unit that transmits the first presentation data to a head-mounted display and the second presentation data to a management terminal, The head-mounted display displays the scenes of a presentation video, which consists of multiple scenes arranged in chronological order, in a virtual space based on the first presentation data. At least one of the aforementioned multiple scenes includes multiple elements with different forms of expression, The first presentation data includes multiple scene information that defines each of the multiple scenes, The scene information includes, for each element, information that controls the display of the element. The second presentation data contains information for managing the presentation. The management terminal is a presentation system that operates based on the second presentation data and transmits switching instruction information to the head-mounted display, instructing it to switch scenes. [Explanation of symbols]

[0140] 100 Presentation data creation device, 200 Database server (server), 220 Storage unit, 230 Communication unit, 300 Management terminal, 400 Head-mounted display (presentation device), 411 Display control unit, 412 Display switching unit, 413 Audio control unit

Claims

1. A display step in which the aforementioned scenes of a presentation video, which consists of multiple scenes arranged in chronological order, are displayed in a virtual space using a head-mounted display based on first presentation data, The computer is instructed to perform a switching step to switch between the aforementioned scenes. At least one of the aforementioned multiple scenes includes multiple elements with different forms of expression, The first presentation data includes multiple scene information that defines each of the multiple scenes, The scene information includes, for each element, element information that controls the display of the element, in a presentation program.

2. The presentation program according to claim 1, wherein the plurality of elements include a first element which is a two-dimensional object, a second element which is a three-dimensional object, and a third element which is an image of a spatial environment.

3. The aforementioned plurality of elements include a second element which is a three-dimensional object, The presentation program according to claim 1, wherein the display step involves moving the viewpoint from outside to inside the second element to display the inside of the second element.

4. A reading step of reading a code for setting the reference position and reference direction in the virtual space when displaying the aforementioned scene, A presentation program according to claim 1 or 2, which causes the computer to perform a setting step of setting the reference position and reference direction in the virtual space based on the code.

5. The presentation program according to claim 4, wherein, in response to reading the code, the computer performs a communication step that causes the head-mounted display to establish communication with a management terminal that manages the progress of the scene.

6. The computer is instructed to perform a download step that causes the head-mounted display to download the first presentation data from the server. The presentation program according to claim 1 or 2, wherein in the switching step, the scene is switched in response to switching instruction information from a management terminal that operates based on second presentation data for managing the presentation.

7. In the aforementioned scene information, a first viewpoint format or a second viewpoint format can be set for each element with a different representation format. The first viewpoint format refers to a format in which the elements are displayed at the same viewpoint on multiple head-mounted displays, including the head-mounted display, when the virtual space is shared by multiple head-mounted displays, The presentation program according to claim 1 or 2, wherein the second viewpoint format indicates a format in which the elements are displayed from different viewpoints depending on the viewpoint position of each of the plurality of head-mounted displays in the virtual space.

8. The presentation program according to claim 1 or 2, wherein, if two scene pieces defining two adjacent scenes on the time axis include element piece information indicating the same element, the display step displays the same element in the virtual space with a changed display form so as to be spatially continuous in a later scene on the time axis.

9. The presentation program according to claim 1 or 2, wherein the plurality of elements with different representation formats include a fourth element which is an image of another virtual space connected to the virtual space.

10. The presentation program according to claim 1 or claim 2, wherein the plurality of elements with different representation formats include a fifth element which is a performance video that is dynamically displayed across the entire display area of ​​the virtual space.

11. For each of the aforementioned element information, it is possible to attach audio information to the element information. A presentation program according to claim 1 or 2, which causes the computer to execute an audio step that outputs sound by setting a virtual sound source at the display position of the element indicated by the element information to which the audio information is attached.

12. A display step in which the aforementioned scenes of a presentation video, which consists of multiple scenes arranged in chronological order, are displayed in a virtual space using a head-mounted display based on first presentation data, Includes a switching step for switching the aforementioned scene, At least one of the aforementioned multiple scenes includes multiple elements with different forms of expression, The first presentation data includes multiple scene information that defines each of the multiple scenes, A presentation method in which the scene information includes information that controls the display of each element.

13. A display control unit that displays the scenes of a presentation video, which consists of multiple scenes arranged in chronological order, in a virtual space using the display unit of a head-mounted display, based on first presentation data, The system comprises a switching control unit for switching between the aforementioned scenes, At least one of the aforementioned multiple scenes includes multiple elements with different forms of expression, The first presentation data includes multiple scene information that defines each of the multiple scenes, The scene information includes information for each element that controls the display of the element, in a presentation device.

14. A storage unit for storing first presentation data and second presentation data, The system includes a communication unit that transmits the first presentation data to a head-mounted display and the second presentation data to a management terminal, The head-mounted display displays the scenes of a presentation video, which consists of multiple scenes arranged in chronological order, in a virtual space based on the first presentation data. At least one of the aforementioned multiple scenes includes multiple elements with different forms of expression, The first presentation data includes multiple scene information that defines each of the multiple scenes, The scene information includes, for each element, information that controls the display of the element. The second presentation data mentioned above includes information for managing the presentation, The management terminal is a presentation system that operates based on the second presentation data and transmits switching instruction information to the head-mounted display, instructing it to switch scenes.

Images