Programs, methods, and information processing devices.

Abstract

To visually represent a musical performance. [Solution] The program causes the computer to perform a process that updates the user's viewpoint in the virtual space based on performance information and outputs an image of the virtual space based on the updated user viewpoint.

Description

Technical Field

[0001] The present disclosure relates to a program, a method, and an information processing apparatus.

Background Art

[0002] For example, there is known a computer graphics moving image generation system that moves an image object such as a dancer in synchronization with the performance of music such as a MIDI (Musical Instrument Digital Interface) music, and generates a moving image that not only follows the musical idea of the music but also changes integrally with the progression of the musical sound (see Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the computer graphics moving image generation system described in Patent Document 1, since it simply dynamically shows a character representing a dancer during the performance, there is room for improvement in terms of visually expressing the performance.

[0005] In view of the above circumstances, an embodiment of the present disclosure aims to provide a program, a method, and an information processing apparatus capable of visually expressing a performance.

Means for Solving the Problems

[0006] A program according to an embodiment of the present disclosure causes a computer to execute processing of updating a user's viewpoint in a virtual space based on performance information and outputting an image of the virtual space based on the updated user's viewpoint.

Effects of the Invention

[0007] According to one embodiment of the present disclosure, a program, method, and information processing device are provided that can visually represent a performance. [Brief explanation of the drawing]

[0008] [Figure 1] This is a schematic diagram showing a system according to one embodiment of the present disclosure. [Figure 2] This is a block diagram showing the configuration of an information processing device according to one embodiment of the present disclosure. [Figure 3] This figure shows an example of an image displayed on the screen of an information processing device according to one embodiment of the present disclosure. [Figure 4] This figure shows an example of an image displayed on the screen of an information processing device according to one embodiment of the present disclosure. [Figure 5] This figure shows an example of an image displayed on the screen of an information processing device according to one embodiment of the present disclosure. [Figure 6] This is an explanatory diagram regarding the user's perspective in a virtual space. [Figure 7] This is an explanatory diagram regarding the user's perspective in a virtual space. [Figure 8] This flowchart shows a process performed by an information processing device in one embodiment of the present disclosure. [Figure 9] This is a schematic diagram showing a system according to another embodiment of the present disclosure. [Figure 10] This is a schematic diagram showing a system according to another embodiment of the present disclosure. [Figure 11] This figure shows an example of an image displayed on the screen of an information processing device in another embodiment of the present disclosure. [Modes for carrying out the invention]

[0009] The following description relates to a program, method, and information processing apparatus according to one embodiment of the present disclosure. Common or corresponding elements are denoted by the same or similar reference numerals, and redundant descriptions are omitted or simplified as appropriate. In each figure, the configuration may be enlarged, reduced, or omitted as appropriate for the sake of explanation. In order to improve the visibility of the drawings, elements in the figures may be shown with lines other than solid lines (such as dashed lines or dotted lines) as necessary.

[0010] As shown in Figure 1, System 1 includes an information processing device 10 and an electronic musical instrument 20. The information processing device 10 is an example of a computer. The information processing device 10 can be, for example, a smartphone, a tablet device, or a PC (Personal Computer). In the example in Figure 1, a tablet device, which is an example of an information processing device 10, is placed on the music stand of the electronic musical instrument 20.

[0011] The electronic instrument 20 is, for example, an electronic keyboard. The electronic instrument 20 may also be other forms of electronic keyboard instruments such as electronic pianos, synthesizers, and electronic organs. The electronic instrument 20 may also be other forms of electronic instruments such as electronic percussion instruments, electronic wind instruments, and electronic string instruments.

[0012] As shown in Figure 2, the information processing device 10 includes a processor 100, a flash ROM (Read Only Memory) 110, a UI (User Interface) 120, a communication interface 130, and a display unit 140.

[0013] The processor 100 is, for example, a single processor or a multi-processor, and includes at least one processor. In a configuration including multiple processors, the processor 100 may be packaged as a single device, or it may consist of multiple physically separated devices within the information processing device 10. The processor 100 may be called, for example, a control unit, a CPU (Central Processing Unit), an MPU (Micro Processor Unit), or an MCU (Micro Controller Unit).

[0014] The processor 100 includes a DSP (Digital Signal Processor), a RAM (Random Access Memory), etc. The processor 100 controls the information processing device 10 by reading various programs and data stored in the flash ROM 110 and using the RAM as a work area.

[0015] The flash ROM 110 is a non-volatile semiconductor memory such as a flash memory, an EPROM (Erasable Programmable ROM), or an EEPROM (Electrically Erasable Programmable ROM). By the processor 100 executing an application 110A which is an example of a program, various processes (for example, various image processes) according to an embodiment of the present disclosure are executed.

[0016] The UI 120 is, for example, a mechanical switch, a touch panel, a mouse, a keyboard, etc. provided in the information processing device 10. The UI 120 may include user interfaces such as gesture recognition and voice recognition. The UI 120 may include various HMIs (Human Machine Interfaces).

[0017] The communication interface 130 is, for example, an interface that connects the information processing device 10 to various external devices communicably. External devices communicable with the information processing device 10 are, for example, an electronic musical instrument 20, a smartphone, a tablet terminal, a PC, a web server, a cloud, etc.

[0018] Some of the various processes according to an embodiment of the present disclosure may be executed by an external device connected to the information processing device 10 via the communication interface 130, for example, instead of the processor 100.

[0019] The flash ROM 110 stores the library 110B. The library 110B contains, for example, objects (e.g., CG (Computer Graphic) data) that are placed in a three-dimensional virtual space VS (see Figure 6, etc.). Various formats that can be rendered as CG can be used for the object format. The processor 100 retrieves objects (e.g., objects such as trees and hills, and backgrounds such as the sky) from the library 110B and places them in the virtual space VS.

[0020] The display unit 140 includes a display composed of, for example, an LCD (Liquid Crystal Display) panel, an organic EL (Electro Luminescence) panel, or an LED (Light Emitting Diode) panel. The display may also be a touch panel display.

[0021] Figures 3 to 5 show examples of images displayed on the display unit 140 in response to user U's performance operations on the electronic instrument 20. Figures 6 to 7 conceptually illustrate the relationship between the virtual space VS and the images displayed on the display unit 140. For example, the virtual space VS depicts a landscape with a undulating grassland with trees under the sky.

[0022] In the following explanation, the two mutually orthogonal horizontal directions in the virtual space VS world coordinate system are referred to as the X and Y directions. The vertical direction that is orthogonal to both the X and Y directions is referred to as the Z direction. That is, the X, Y, and Z directions are mutually orthogonal. For convenience, the axes extending in the X, Y, and Z directions may be referred to as the X-axis, Y-axis, and Z-axis, respectively. The X direction may also be called the front-back direction. The Y direction may also be called the left-right direction. The Z direction may also be called the up-down direction. Note that the directional names are used for convenience to explain the relative positional relationships of the constituent elements and do not indicate absolute directions. For example, the X direction may be defined as the left-right direction, and the Y direction may be defined as the front-back direction.

[0023] In Figure 6, the symbol P1 indicates the origin of the camera coordinate system. The origin P1 of the camera coordinate system initially coincides with the origin of the global coordinate system of the virtual space VS, for example, when application 110A is started. Similarly, the orientation D1 of the camera coordinate system initially coincides with the orientation of the global coordinate system. The origin P1 and orientation D1 of the camera coordinate system represent the viewpoint of user U in the virtual space VS. More specifically, the origin P1 of the camera coordinate system represents the viewpoint position of user U in the virtual space VS. The orientation D1 of the camera coordinate system represents the viewpoint direction of user U in the virtual space VS. For convenience, below, the origin P1 will be referred to as "viewpoint position P1," and the orientation D1 will be referred to as "viewpoint direction D1." "Viewpoint direction" may also be rephrased as "line of sight direction."

[0024] The processor 100 updates the user U's viewpoint (i.e., viewpoint position P1 and viewpoint direction D1) in the virtual space VS based on the performance information. For example, the processor 100 calculates the average tempo over a predetermined period (e.g., a time corresponding to the most recent n beats). The faster the calculated performance tempo, the faster the processor 100 moves the viewpoint position P1 forward (positive X direction) in the virtual space VS. The slower the calculated performance tempo is compared to the reference tempo, the faster the processor 100 moves the viewpoint position P1 backward (negative X direction) in the virtual space VS. For example, if there is no performance (tempo is zero), the processor 100 stops the movement of the viewpoint position P1 in the virtual space VS. If it is already stopped, the processor 100 does not move the viewpoint position P1. The performance tempo can be obtained, for example, using the method described in Japanese Patent Publication No. 5672280.

[0025] The processor 100 converts the global coordinates to the camera coordinate system. The processor 100 then performs a projection transformation on the converted camera coordinate system. By performing a viewport transformation, the processor 100 displays the two-dimensional image projected onto the projection surface within a viewport, which is a designated area on a window represented in the screen coordinate system. The processor 100 performs these coordinate transformations sequentially. In Figures 6 and 7, the symbol S1 conceptually represents the screen area of ​​the display unit 140 defined in the screen coordinate system.

[0026] Based on the performance information, the user U's viewpoint in the virtual space VS (i.e., viewpoint position P1 and viewpoint direction D1) is updated, and the area within the virtual space VS displayed on the screen area S1 is also updated. For example, if the viewpoint position P1 moves forward (positive X direction) in accordance with the tempo of the performance, the screen area S1 also moves forward within the virtual space VS in accordance with the viewpoint position P1. As the viewpoint position P1 moves, the image displayed on the display unit 140 gradually changes, for example, from image 30a to image 30b shown in Figure 3. As can be seen by comparing image 30a and image 30b, as the viewpoint position P1 moves forward, the trees move closer to the foreground of the field of view, and distant scenery that was previously invisible becomes visible. In this embodiment, the direction of movement of the user U's viewpoint position is fixed to forward (positive X direction). Fixing the direction of movement in one direction has the advantage of reducing the processing load on the processor 100 to render images, because the difference between multiple images becomes smaller compared to when the direction of movement is not fixed in one direction. Therefore, changes in the video corresponding to the performance information are reflected with a short response (in other words, the rendering process is executed with low latency in response to the input of performance information). For example, if objects placed in the virtual space VS are moved in a complex manner according to the performance information (for example, if each leaf of a tree is shaken according to the performance information), the processing load of the processor 100 increases, and the delay of the rendering process in response to the input of performance information becomes large. Therefore, in this embodiment, instead of moving objects placed in the virtual space VS in a complex manner according to the performance information, the video displayed on the screen is ultimately changed by updating the user U's viewpoint of the virtual space VS based on the performance information. That is, because the video is changed by changing the viewpoint, the performance content can be represented visually even if, for example, the objects in the virtual space VS are stationary objects (i.e., even when the rendering load is reduced).

[0027] The processor 100 analyzes performance information based on the user U's performance on the electronic musical instrument 20, for example, to obtain performance features. The processor 100 calculates an SD (Semantic Differential) value based on the features obtained from the performance information. For example, the method described in Japanese Patent Publication No. 2024-121044 is used to calculate the SD value based on the performance features.

[0028] For example, the processor 100 calculates standard deviation (SD) values ​​corresponding to antonym pairs such as "bright and dark (or cheerful and calm)" based on features obtained from performance information. These SD values ​​are calculated, for example, based on the average of the tonality (a numerical value representing the tonality) and tempo over a predetermined period (for example, a time corresponding to the most recent n beats). These SD values ​​may also be calculated using only the tonality over the predetermined period.

[0029] Processor 100 determines, for example, that the performance is bright (or cheerful) if this SD value is greater than or equal to the first threshold. Processor 100 determines, for example, that the performance is moderate (for example, neither bright nor dark) if this SD value is less than the first threshold but greater than or equal to the second threshold which is less than the first threshold. Processor 100 determines, for example, that the performance is dark (or calming) if this SD value is less than the second threshold.

[0030] Any reference to elements using designations such as “First,” “Second,” etc., as used in this disclosure, does not generally limit the quantity or order of those elements. These designations are used for convenience to distinguish between two or more elements. Therefore, references to the First and Second elements do not imply, for example, that only two elements are adopted, or that the First element must precede the Second element.

[0031] The processor 100 changes the scene in the virtual space VS according to these judgment results. Figure 4 illustrates the case when the judgment result is "bright (or cheerful)". In this case, the processor 100 changes the time of day in the virtual space VS from night (see image 40a) to morning (see image 40b). If the judgment result is "dark (or calm)", the processor 100 changes the time of day in the virtual space VS from morning (see image 40b) to night (see image 40a). If the judgment result is "neutral", the processor 100 changes the time of day in the virtual space VS to daytime. The time of day in the virtual space VS may be changed instantaneously or gradually. If the same judgment result (e.g., bright) continues, the time of day in the virtual space VS (e.g., morning) does not change.

[0032] In addition to the scene changes in the virtual space VS, the processor 100 may also update the user U's viewpoint (i.e., viewpoint position P1 and viewpoint direction D1). For example, if the judgment result is "bright (or fun)", the processor 100 raises the viewpoint position P1, as shown in Figure 6. For example, if the judgment result is "dark (or calm)", the processor 100 lowers the viewpoint position P1, as shown in Figure 6. For example, if the judgment result is "dark", the processor 100 may continue to lower the viewpoint position P1, moving from the grassland into the ground. For example, if the judgment result is "neutral", the processor 100 does not raise or lower the viewpoint position P1.

[0033] Thus, in this embodiment, the altitude of the viewpoint position P1 changes in conjunction with the key of the music being played. For example, if User U plays a cheerful piece of music, User U can experience what it's like to look out over a landscape from above. For example, if User U plays a dark piece of music, User U can experience what it's like to descend underground and wander through an underground world.

[0034] In another example, if the judgment result is "bright (or fun)", the processor 100 raises the viewpoint position P1 and changes the viewpoint direction D1 to an angle looking diagonally downward and forward (in other words, tilts downward), as shown in the upper diagram of Figure 7. In this case, the image displayed on the display unit 140 gradually changes, for example, from image 50a to image 50b shown in Figure 5. Specifically, as shown in Figure 5, the viewpoint position P1 rises and the viewpoint direction D1 tilts downward as the time of day in the virtual space VS changes from night to morning.

[0035] For example, if the judgment result is "dark (or calm)," the processor 100 lowers the viewpoint position P1 and changes the viewpoint direction D1 to an angle looking diagonally upward and forward (in other words, tilts upward), as shown in the lower diagram of Figure 7. For example, if the judgment result is "moderate," the processor 100 neither raises nor lowers the viewpoint position P1, nor changes the viewpoint direction D1.

[0036] Thus, in this embodiment, the tilt angle of the viewpoint position P1 changes in conjunction with the key of the song being played. For example, when User U plays a bright song (when the processor 100 determines the key to be major), User U can have an experience of looking down on the scenery (a pleasant experience as if floating in the air on an aircraft). For example, when User U plays a dark song (when the processor 100 determines the key to be minor), User U can have an experience of looking up at the scenery.

[0037] In the example shown in Figure 7, the processor 100 changes both the altitude of the viewpoint position P1 and the tilt angle of the viewpoint direction D1 based on the determination result. However, in another embodiment, the processor 100 may change only the tilt angle of the viewpoint direction D1 based on the determination result.

[0038] Thus, the processor 100 updates at least one of the user U's viewpoint position P1 and viewpoint direction D1 in the virtual space VS based on the performance information. In addition, the processor 100 obtains the tempo of the performance and updates the user U's viewpoint in the virtual space VS based on the obtained tempo. The processor 100 also obtains the key of the song being performed and changes the time period (an example of a scene) in the virtual space VS or updates the user U's viewpoint in the virtual space VS based on the obtained key. For example, if the processor 100 determines that the key of the song is bright (or cheerful) (to put it another way, determines that the key is major), it raises the user U's viewpoint. If the processor 100 determines that the key of the song is dark (or calm) (to put it another way, determines that the key is minor), it lowers the user U's viewpoint. The processor 100 outputs images of the virtual space VS (for example, images 30a, 30b, etc. shown in Figure 3) based on the user U's viewpoint updated based on the performance information.

[0039] The features referenced to change the scene in the virtual space VS or the user U's viewpoint are not limited to tempo or tonality. The processor 100 may change the scene in the virtual space VS or the user U's viewpoint in accordance with SD values ​​based on other features (e.g., pitch, velocity, chord type, number of chords, etc.).

[0040] According to this embodiment, user U can enjoy the images in the virtual space VS and have an experience that cannot be obtained by simply playing the electronic instrument 20. Because the images in the virtual space VS change in accordance with the performance, user U can visually enjoy the musical expression. For example, by changing the tempo of the performance, the user can control the speed and direction of movement of the viewpoint position P1 (such as the way the scenery displayed in the field of view flows in the examples in Figures 3 to 5). For example, by changing the key, the user can control the viewpoint position P1 and viewpoint direction D1, and also control the scene in the virtual space VS.

[0041] Because the content of the performance is reflected in the video in the virtual space VS, user U can confirm their own performance as visualized information. For example, user U can feel the enjoyment of performing through the video in the virtual space VS, which can lead to increased motivation for practicing. Others can enjoy user U's performance not only aurally but also visually. Even people with hearing impairments can enjoy performing through the video in the virtual space VS.

[0042] Using Figure 8, a process executed by the information processing device 10 (processor 100) in one embodiment of this disclosure will be described. For example, when application 110A is started, the execution of the process shown in Figure 8 begins. For example, when application 110A is terminated, the process shown in Figure 8 ends.

[0043] The steps in the flowchart shown in this embodiment may be rearranged, provided they are consistent with each other. For example, while this disclosure presents various steps in an exemplary order, it is not limited to this order. Furthermore, the steps in the flowchart shown in this embodiment may be executed in parallel or concurrently, provided they are consistent with each other.

[0044] As shown in Figure 8, the information processing device 10 performs various initial settings (step S101). For example, the information processing device 10 sets the origin of the 3D global coordinate system to the virtual space VS. The information processing device 10 calls each default object (such as a tree object) from the library 110B. The information processing device 10 performs a modeling transformation on each of the called objects, which are represented in the local coordinate system, and places them in the global coordinate system. That is, the information processing device 10 renders each default object in the virtual space VS. The information processing device 10 further places the origin of the camera coordinate system at a predetermined position in the virtual space VS.

[0045] The information processing device 10 performs a projection transformation on the camera coordinates using internal parameters. To display the two-dimensional image projected onto the projection surface within a viewport, which is a designated area on a window represented in the screen coordinate system, the information processing device 10 performs a viewport transformation. The information processing device 10 performs these coordinate transformations sequentially. As a result, the image of the virtual space VS is displayed on the display unit 140.

[0046] The information processing device 10 waits for input of performance information (step S102). The input performance information is, for example, information corresponding to the performance of the electronic instrument 20 by user U (such as MIDI signals). The input performance information is not limited to the performance by user U, but may also be pre-prepared music data.

[0047] When performance information is input from the electronic instrument 20 (step S102: YES), the information processing device 10 analyzes the input performance information and obtains feature quantities of the performance (step S103). The obtained feature quantities include, for example, pitch, velocity, key, tempo, chord type, and number of chords. For example, immediately after the start of a performance, there is insufficient performance information necessary for analyzing the performance. In this case, the information processing device 10 starts analyzing the performance information after the necessary amount of performance information has been collected.

[0048] The information processing device 10 calculates SD values ​​based on the features acquired in step S103 (step S104). The information processing device 10 calculates SD values ​​corresponding to antonym pairs such as "bright and dark (or happy and calm)".

[0049] The information processing device 10 performs a threshold determination on the SD value calculated in step S104 (step S105). For example, as described above, if the SD value is greater than or equal to the first threshold, the information processing device 10 determines that the performance is bright (or cheerful). If the SD value is less than the first threshold but greater than or equal to the second threshold which is smaller than the first threshold, the information processing device 10 determines that the performance is moderate. If the SD value is less than the second threshold, the information processing device 10 determines that the performance is dark (or subdued).

[0050] The information processing device 10 updates the user U's viewpoint in the virtual space VS (step S106). For example, the information processing device 10 updates the movement speed and direction of the viewpoint position P1 based on the tempo acquired in step S103, and updates the altitude and viewpoint direction D1 (tilt angle) of the viewpoint position P1 based on the result of the threshold determination in step S105. Depending on the acquired tempo and the result of the threshold determination, the user U's viewpoint may not be updated.

[0051] The information processing device 10 further changes the scene of the virtual space VS based on the result of the threshold determination in step S105 (step S107). Depending on the result of the threshold determination, the scene of the virtual space VS may not change. The information processing device 10 repeats the processes in steps S102 to S107, for example, until application 110A is terminated.

[0052] As a result of the execution of steps S106 to S107, the image of the virtual space VS displayed on the display unit 140 changes, as illustrated in Figures 3 to 5. User U can, for example, visually enjoy the musical performance through the image of the virtual space VS displayed on the display unit 140.

[0053] The above is a description of exemplary embodiments of the present disclosure. Embodiments of the present disclosure are not limited to those described above, and various modifications are possible within the scope of the technical idea of ​​the present disclosure. For example, embodiments of the present application include combinations of embodiments explicitly shown in the specification or obvious embodiments as appropriate.

[0054] In the above embodiment, the video of the virtual space VS is displayed on the screen of a portable device such as a tablet terminal placed on a music stand. However, in another embodiment, as shown in Figure 9, the video of the virtual space VS may be displayed on a stationary large display connected to a PC or the like. In the example in Figure 9, the video of the virtual space VS is displayed on a triple display. Compared to the above embodiment, user U can experience an even greater sense of presence and immersion.

[0055] As shown in Figure 10, the information processing device 10 may be an XR (Cross Reality) device such as an HMD (Head Mounted Display). The information processing device 10 is, for example, an image device that allows user U to observe images of a virtual world or images that fuse the real world and the virtual world. User U can play the electronic musical instrument 20 while viewing images of the virtual space VS displayed on the XR device or images that fuse objects from the real world and the virtual world. To make it easier for user U to play the electronic musical instrument 20, the information processing device 10 displays the electronic musical instrument 20 in a pass-through manner, for example. Even when using an XR device, user U can experience an even greater sense of presence and immersion compared to the above embodiment.

[0056] The information processing device 10 is not limited to an HMD; it may also be other forms of wearable devices such as smart glasses. These types of wearable devices are sometimes called VR (Virtual Reality) goggles, AR (Augmented Reality) goggles, MR (Mixed Reality) goggles, glasses displays, or glasses devices.

[0057] In the above embodiment, objects pre-registered in library 110B are placed in the virtual space VS. However, in another embodiment, for example, objects generated using a generation AI (Artificial Intelligence) program may be placed in the virtual space VS.

[0058] For example, performance information based on a performance on an electronic instrument 20 is input to the information processing device 10. The information processing device 10 analyzes the input performance information to obtain features and calculates the standard deviation (SD) value corresponding to the antonym pair "bright and dark (or fun and calm)". For example, if this SD value is a value indicating brightness (for example, a value close to 0), the information processing device 10 generates text information such as "bright". The information processing device 10 provides the generated text information (i.e., words that represent the impression given by the performance) to a trained model. The trained model may be held by the information processing device 10, or it may be provided via cloud computing.

[0059] A trained model is a publicly known or well-known generative AI program that generates objects from text information, and is implemented, for example, by a neural network. When text information representing the impression given by a performance is input to the trained model, it generates an object that matches the performance (for example, an object of a landscape or object suitable for a bright impression) and returns it to the information processing device 10. The information processing device 10 places the object obtained from the generative AI program into the virtual space VS. As a result, for example, a landscape with a bright impression (see, for example, reference numeral 60 in Figure 11) is displayed on the display unit 140. For example, if the text information generated from the SD value is "moderate", a landscape with a moderate impression (see, for example, reference numeral 62 in Figure 11) is displayed on the display unit 140. For example, if the text information generated from the SD value is "dark", a landscape with a dark impression (see, for example, reference numeral 64 in Figure 11) is displayed on the display unit 140.

[0060] In other words, in the above embodiment, the virtual space VS changes to a time period (an example of a scene) that matches the impression given by the performance, but in another embodiment, the virtual space VS switches to a landscape (an example of a scene) that matches the impression given by the performance. [Explanation of Symbols]

[0061] 1: System, 10: Information Processing Unit, 20: Electronic Musical Instrument, 100: Processor, 110A: Application, 110B: Library, 140: Display Unit

Claims

1. The user's viewpoint in the virtual space is updated based on performance information. Output an image of the virtual space based on the updated viewpoint. To have the computer perform the process. program.

2. Based on the performance information, update at least one of the user's viewpoint position and viewpoint direction in the virtual space. The program according to claim 1.

3. Get the tempo of the performance, The user's viewpoint in the virtual space is updated based on the acquired tempo. The program according to claim 1.

4. The key of the piece to be played is obtained, The user's viewpoint in the virtual space is updated based on the acquired tone. The program according to claim 1.

5. When the aforementioned tone is determined to be bright, the user's viewpoint is raised. The program according to claim 4.

6. The key of the piece to be played is obtained, The scene in the virtual space is changed based on the acquired tone. The program according to claim 1.

7. The user's viewpoint in the virtual space is updated based on performance information. Output an image of the virtual space based on the updated viewpoint. To have the computer perform the process. method.

8. The user's viewpoint in the virtual space is updated based on performance information. Output an image of the virtual space based on the updated viewpoint. Equipped with a control unit, Information processing device.

Images