Performance information generation method, performance information generation device, and program

Abstract

To provide a performance information generation method with which a person who is not good at performance can be assisted by a person who is good at performance, with which more advanced performance is achieved, and which can realize new performance which appears to a viewer as if the person is performing alone.SOLUTION: A performance information generation method acquires first performance information related to performance by a first performer and second performance information related to performance by a second performer, generates third performance information obtained by integrating the acquired first performance information and second performance information, and outputs the generated third performance information.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] An embodiment of the present invention relates to a performance information generating method, a performance information generating device, and a program. [Background technology]

[0002] Patent document 1 discloses an information processing device that places a first avatar associated with a first user and a second avatar associated with a second user in a virtual space, causes the second avatar to perform a first performance in response to the movements of the second user, requests the second user to perform a second performance by the second avatar, and causes the second avatar to perform the second performance in response to the movements of the second user.

[0003] Patent document 2 discloses a method for providing heart rate information by defining a number of heart rate zones as percentage ranges of maximum heart rate, assigning colors to each of the heart rate zones, receiving heart rate information from a user, and providing a graphical display of the heart rate information, in which the color of a subregion of the graphical display corresponds to the color assigned to one of the heart rate zones. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2019-192173 [Patent Document 2] Patent Publication No. 2010-267267 Summary of the Invention [Problem to be solved by the invention]

[0005] One aspect of the present disclosure aims to provide a performance information generating method that enables skilled performers to assist those who are not good at playing, achieves more advanced performances, and realizes new performances that appear to the audience as if the performer is playing alone. [Means for solving the problem]

[0006] A performance information generating method according to one embodiment of the present invention acquires first performance information relating to a performance of a first performer and second performance information relating to a performance of a second performer, generates third performance information by integrating the acquired first performance information and second performance information, and outputs the generated third performance information. Effect of the Invention

[0007] According to one embodiment of the present invention, a skilled performer can assist a less skilled performer, enabling a more advanced performance to be achieved, and a new performance can be realized that gives the viewer the impression that the performer is performing alone. [Brief description of the drawings]

[0008] [Figure 1] 1 is a block diagram showing a configuration of a performance information generating device 1. FIG. [Diagram 2] FIG. 2 is a block diagram showing the configuration of PC1A. [Diagram 3] 1 is a flowchart showing the operation of a performance information generating method. [Figure 4] FIG. 13 is a configuration diagram of a performance information generating system according to a fourth modified example. [Diagram 5] FIG. 13 is a configuration diagram of a performance information generating system according to a sixth modified example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] 1 is a configuration diagram of a performance information generating system according to this embodiment. The performance information generating system includes a PC (personal computer) 1A installed at a first location 10, a PC 1B installed at a second location 20, and a PC 1C installed at a third location 30.

[0010] A first performer 3 at a first location 10 connects an instrument 5 and a motion sensor 50 to a PC 1A. A second performer 7 at a second location 20 connects an instrument 9 and a motion sensor 51 to a PC 1B. The motion sensors 50 and 51 are sensors for detecting the performance motion of the performers, and may be optical, inertial, or image-based sensors, for example. The motion sensor 50 detects the motion of the left hand of the first performer 3. The motion sensor 51 detects the motion of the right hand of the second performer 7.

[0011] In this embodiment, as an example, the instruments 5 and 9 are electric guitars. The first performer 3 is an expert in playing electric guitars, and the second performer 7 is a beginner. The first performer 3 plays the instrument 5 with both hands. That is, the first performer 3 presses the frets with his left hand and picks with his right hand. The second performer 7 picks with his right hand, but does not press the frets with his left hand.

[0012] In this embodiment, "playing" is not limited to playing an instrument, but also includes singing using a microphone.

[0013] Fig. 2 is a block diagram showing the configuration of PC1A. PC1A is a general-purpose information processing device. PC1A, PC1B, and PC1C have the same main configuration. Fig. 2 shows the configuration of PC1A as a representative.

[0014] The PC 1A includes a display 31, a user I / F 32, a flash memory 33, a processor , a RAM 35, a communication I / F , a speaker (SP) 37, and an audio I / F .

[0015] The display 31 is, for example, an LED, an LCD, or an OLED, and displays various information. The user I / F 32 is a touch panel that is laminated on the LCD or OLED of the display 31. Alternatively, the user I / F 32 may be a keyboard, a mouse, or the like. When the user I / F 32 is a touch panel, the user I / F 32, together with the display 31, constitutes a GUI (Graphical User Interface).

[0016] The communication I / F 36 includes a network interface and is connected to a network such as the Internet via a router (not shown). The communication I / F 36 also includes a general-purpose communication interface. The communication I / F 36 of PC 1A is connected to a motion sensor 50 and receives a first sensor signal from the motion sensor 50. The communication I / F 36 of PC 1B is connected to a motion sensor 51 and receives a second sensor signal from the motion sensor 51.

[0017] The audio I / F 38 has an analog audio terminal. The audio I / F 38 is connected to an acoustic device such as a musical instrument or a microphone via an audio cable and receives an analog sound signal. In this embodiment, the audio I / F 38 of PC1A is connected to the musical instrument 5 and receives an analog sound signal related to a performance sound from the musical instrument 5. The audio I / F 38 of PC1B is connected to the musical instrument 9 and receives an analog sound signal related to a performance sound from the musical instrument 9. The audio I / F 38 converts the received analog sound signal into a digital sound signal. The audio I / F 38 also converts the digital sound signal into an analog sound signal. The SP37 plays a sound based on the analog sound signal.

[0018] The processor 34 generates performance information based on the first sensor signal received from the motion sensor 50 and the digital sound signal received from the audio I / F 39. In this embodiment, the processor 34 of PC1A generates first performance information including first motion data based on the first sensor signal and information on the performance sound based on the received digital sound signal. The processor 34 of PC1A transmits the first performance information to PC1C via the network. The processor 34 of PC1B generates second performance information including second motion data based on the second sensor signal. The processor 34 of PC1B transmits the second performance information to PC1C via the network.

[0019] 3 is a flowchart showing the operation of the performance information generating method. In this embodiment, the processor 34 of the PC 1C executes the performance information generating method shown in FIG.

[0020] Processor 34 acquires first performance information related to the performance of the first performer and second performance information related to the performance of the second performer (S11). That is, processor 34 of PC1C receives the first performance information from PC1A via the network, and receives the second performance information from PC1B via the network.

[0021] The processor 34 generates third performance information by integrating the acquired first performance information and second performance information (S12). The first performance information includes first motion data based on the first sensor signal and information on a performance sound. In this embodiment, the second performance information includes second motion data based on the second sensor signal. The first motion data corresponds to the movement of the left hand of the first performer 3. The second motion data corresponds to the movement of the right hand of the second performer 7. As third motion data for controlling certain 3D model data, the processor 34 controls the left hand model with the first motion data and controls the right hand model with the second motion data. That is, the processor 34 generates third motion data by integrating the first motion data and the second motion data. The processor 34 also generates information on a performance sound included in the third performance information by using information on a performance sound included in the first performance information as information on a performance sound synchronized with the performance action of the 3D model data.

[0022] In this way, the processor 34 generates third performance information including third motion data for controlling the 3D model data and information on performance sounds synchronized with the performance motion of the 3D model data.

[0023] The processor 34 outputs the generated third performance information (S13). For example, the processor 34 records the third performance information in the flash memory 33. Alternatively, the processor 34 uses the generated third performance information to render the 3D model data and play back the performance sound.

[0024] First, the processor 34 controls the designated 3D model data using the third motion data included in the third performance information. The 3D model data includes model data of a certain performer (character) and model data of a certain musical instrument. The model data is downloaded from another device such as a server (not shown) (for example, a computer used by a creator) and stored in the flash memory 33. The performer's model data has, for example, a plurality of polygon data and bone data for constituting the performer's face, torso, arms, fingers, legs, etc. The plurality of bone data has a link structure connected by a plurality of joint data. The position information of each bone data of the model data is defined by the motion data. The musical instrument model data has, for example, a plurality of polygon data and bone data for constituting the body, neck, strings, etc. of a guitar. The plurality of bone data constituting the musical instrument model data also has a link structure connected by a plurality of joint data. However, since a guitar does not have a range of motion other than the strings, a link structure is not essential. In the case of a musical instrument having a range of motion, it is preferable that the model data includes a link structure.

[0025] The processor 34 controls the position information of the model data of the performer based on the third motion data included in the third performance information. The processor 34 renders the 3D model data of the performer and controls the position information of the 3D model data based on the third motion data included in the third performance information. The processor 34 displays an image related to the rendered 3D model data on the display 31.

[0026] 1, images of 3D model data 90 of a certain performer and 3D model data 91 of a musical instrument are displayed on the display 31. The right hand of the 3D model data 90 moves in response to the picking movement of the right hand of the first performer 3, and the left hand of the 3D model data 90 moves in response to the fret pressing movement of the left hand of the second performer 7.

[0027] Furthermore, the processor 34 reproduces the performance sound information included in the third performance information to generate a sound signal related to the performance sound. The processor 34 outputs the generated sound signal to the SP 37 to reproduce the performance sound.

[0028] A user of the PC 1C watches a performance in which an actor represented by 3D model data 90 plays an instrument represented by 3D model data 91, the performance being displayed on a display device 31.

[0029] In this way, the performance information generating method of this embodiment can realize a new performance that looks as if the audience is performing alone, even when a performance is performed by multiple people (first performer 3 and second performer 7). This allows the first performer 3 and second performer 7 who provide the performance to achieve a more advanced performance, providing a new customer experience that was not possible in the past. Furthermore, the performance information generating method of this embodiment can realize a new performance that looks as if the audience is performing alone, even when the first performer 3 who is good at performing assists the second performer 7 who is not good at performing. This allows the second performer 7 who is not good at performing to provide a more advanced performance to the audience with the help of the first performer 3 who is good at performing, providing a new customer experience.

[0030] (Variation 1) In the above embodiment, an example has been shown in which the first performance information includes information on the performance sound received from the musical instrument 5 as information on the performance. However, the second performance information may include information on the performance sound received from the musical instrument 9 as information on the performance. In this case, the processor 34 of the PC 1C generates the third performance information using the information on the performance sound included in the second performance information.

[0031] (Variation 2) In the second modification, both the first performance information and the second performance information include information related to the performed sound.

[0032] In this case, the first performance information includes a sound signal of the performance sound related to the instrument 5 played by the first performer 3. The second performance information includes information related to the picking with the right hand of the second performer 7 as information related to the performance sound.

[0033] The processor 34 of the PC 1B detects information related to the picking of the second performer 7. The processor 34 extracts, for example, a feature of a sound signal received from the musical instrument 9, compares it with a feature related to picking detected in advance, and extracts information indicating the timing of picking. The feature is, for example, a spectral envelope, power, an amplitude spectrum, a fundamental frequency, a formant frequency, a mel spectrum envelope, a mel spectrum, or a cepstrum. The processor 34 detects the feature from the sound signal acquired at the time of picking, and stores it in the flash memory 33 or the like. The processor 34 refers to the flash memory 33 with the feature of the acquired sound signal, and extracts information indicating the timing of picking.

[0034] Also, the processor 34 may prepare a trained model in which the relationship between the sound signal and the information indicating the timing of picking is trained by a DNN (Deep Neural Network) or the like, and obtain information indicating the timing of picking by inputting the sound signal to the trained model. The processor 34 may obtain, as a training stage, a data set of, for example, the sound signal and the information indicating the timing of picking from a server or the like. Alternatively, the processor 34 may obtain, as a training stage, second motion data based on a second sensor signal from the motion sensor 51, extract information indicating the timing of picking (for example, the timing at which the right hand overlaps with the string), and obtain a sound signal received at that timing. The processor 34 trains a relationship between the sound signal and the information indicating the timing of picking in a predetermined model based on the acquired sound signal and the information indicating the timing of picking. The processor 34 may obtain, as an execution stage, the sound signal received from the musical instrument 9 to the trained model, and obtain information indicating the timing of picking.

[0035] Alternatively, the processor 34 may obtain information indicating the timing of picking based on a sensor signal of the motion sensor 51. For example, when the motion sensor 51 is an image-type sensor, the processor 34 obtains information indicating the timing of picking based on an image of the picking movement of the second actor's right hand.

[0036] Then, processor 34 generates second performance information including the second motion data and information indicating the picking timing.

[0037] The processor 34 of the PC 1C reproduces the information on the performance sounds included in the first performance information based on the information indicating the picking timing included in the second performance information. As a result, the performance sounds related to the instrument 5 played by the first performer 3 are reproduced in accordance with the picking movement of the right hand of the second performer 7.

[0038] (Variation 3) In the above embodiment, the information on the performance is a performance sound (sound signal). The information on the performance in the third modification includes musical instrument operation information. The processor 34 of the PC 1C generates a performance sound based on the musical instrument operation information.

[0039] For example, in the case of a guitar, the operation information of a musical instrument is information indicating which fret is being pressed, the timing at which the fret is pressed, the timing at which the fret is released, information indicating which string is being picked, the picking timing, the picking speed, whether or not a mute operation is performed, etc. Furthermore, in the case of a keyboard instrument such as a synthesizer, the operation information of a musical instrument is, for example, the pitch (note number), tone color, and time parameters such as attack, decay, sustain, and release.

[0040] Operation information of these musical instruments is acquired, for example, based on sensor signals from motion sensors 50 and 51. When motion sensors 50 and 51 are image-type sensors, processor 34 detects operation information of the musical instruments from images of the hands of first performer 3 and second performer 7.

[0041] The processor 34 can also obtain operational information of the instrument by a sensor mounted on the instrument. For example, in the case of a guitar, the sensor mounted on the instrument is a fret sensor attached to each fret. The processor 34 obtains the sensor signal for each fret to obtain operational information of the instrument. Alternatively, in the case of an electronic instrument in which the strings to be picked are replaced with sensors, the processor 34 obtains the sensor signal to obtain operational information of the instrument.

[0042] The processor 34 may also extract features of the sound signal received from the musical instrument, compare them with features corresponding to previously detected operation information, and obtain operation information of the musical instrument. The processor 34 may also obtain operation information of the musical instrument by preparing a trained model in which the relationship between the sound signal and the operation information of the musical instrument is trained by DNN or the like, and inputting the sound signal into the trained model. The processor 34 may obtain a data set of the sound signal and the operation information of the musical instrument from a server or the like as a training stage. Alternatively, the processor 34 may obtain the operation information of the musical instrument by acquiring a sensor signal for each fret as a training stage, and obtain the sound signal received at that timing. The processor 34 trains the relationship between the sound signal and the operation information of the musical instrument in a predetermined model based on the acquired sound signal and the operation information of the musical instrument. The processor 34 may input the sound signal received from the musical instrument 9 into the trained model as an execution stage, and obtain the operation information of the musical instrument.

[0043] Such operation information of the musical instrument corresponds to parameters for synthesizing the sound of the sound source. Processor 34 of PC1A acquires operation information of the right hand of first performer 3, and processor 34 of PC1B acquires operation information of the left hand of second performer 7. Processor 34 of PC1C generates operation information that integrates operation information of the right hand of first performer 3 and operation information of the left hand of second performer 7. This generates performance operation information that integrates the picking action of the right hand of first performer 3 and the fret pressing action of the left hand of second performer 7. Processor 34 synthesizes the sound of the sound source (a guitar sound source in this embodiment) based on the integrated operation information.

[0044] As a result, a performance sound based on a performance operation that combines the picking action of the first performer 3 with the right hand and the fret pressing action of the second performer 7 with the left hand is reproduced.

[0045] (Variation 4) Fig. 4 is a configuration diagram of a performance information generating system according to Modification 4. Configurations common to Fig. 1 are given the same reference numerals and description thereof will be omitted.

[0046] In the performance information generating system according to the fourth modification, a microphone 75 is connected to a PC 1C installed at a third point 30. At the third point 30, a third performer 2 performs a singing performance.

[0047] In the performance information generating system, the third performance information includes information on the first performance related to at least either the first performance information or the second performance information. In this example, the third performance information includes a performance sound related to the first performance of the first performer 3. The third performer 2 listens to the performance sound of the first performer 3. The third performer 2 also views the 3D model data 90 displayed on the display 31 of the PC 1C. The third performer 2 sings while viewing the performance of the 3D model data 90 and listening to the performance sound of the first performer 3.

[0048] The processor 34 of the PC 1C receives an audio signal related to the singing sound of the third performer 2 picked up by the microphone 75. The processor 34 transmits the received audio signal to other devices (PC 1A and PC 1B) via the network.

[0049] In this way, the first performer 3 at the first location 10, the second performer 7 at the second location 20, and the third performer 2 at the third location 30 perform a remote ensemble. The third performer 2 at the third location 30 can have a new customer experience in that he or she perceives that he or she is performing an ensemble with one other performer. Also, the second performer 7 at the second location 20 can have a new customer experience in that he or she perceives that he or she is performing an ensemble with the third performer 2 just by pressing the frets with his or her left hand.

[0050] (Variation 5) In the above embodiment, an example has been shown in which the first motion data (motion data of the left hand) is acquired by the motion sensor 50, and the second motion data (motion data of the right hand) is acquired by the motion sensor 51. However, the first motion data or the second motion data is not limited to being acquired by a motion sensor.

[0051] For example, the processor 34 may estimate the performance motion based on a trained model in which the relationship between the sound signal of the performance sound and the performance motion is trained by DNN. The processor 34 may also acquire the performance motion by a sensor mounted on the musical instrument. For example, the processor 34 may acquire the performance motion related to the fingering of the left hand by a fret sensor attached to each fret. The processor 34 acquires, for example, a data set of the sound signal and the performance motion (motion data of the right hand or left hand in the case of a guitar) from a server or the like as a training stage. Alternatively, the processor 34 may acquire the performance motion detected by using a motion sensor as a training stage, and acquire the sound signal received at that timing. The processor 34 trains the relationship between the sound signal and the performance motion in a predetermined model based on the acquired sound signal and performance motion. The processor 34 inputs the sound signal received from the musical instrument 9 to the trained model as an execution stage, and acquires the performance motion.

[0052] (Variation 6) In the performance information generating system according to the sixth modification, at least one of the first performance information and the second performance information includes information that is created and recorded in advance.

[0053] Fig. 5 is a configuration diagram of a performance information generating system according to Modification 6. Configurations common to Fig. 1 are given the same reference numerals and descriptions thereof will be omitted. The performance information generating system according to Modification 6 has a server 100 instead of PC 1B. The server 100 records the second performance information generated by PC 1B in Fig. 1. The server 100 distributes the recorded second performance information.

[0054] The processor 34 of the PC 1C obtains the first performance information from the PC 1A and obtains the second performance information from the server 100.

[0055] (Variation 7) The performance information generation system of variant example 7 records a non-fungible token (hereinafter referred to as NFT) corresponding to at least either the first performance information or the second performance information in a digital ledger on the Internet.

[0056] For example, the first performer 3 shown in FIG. 1 records an NFT corresponding to his / her first performance information in a digital ledger. This allows the first performer 3 to prove his / her first performance information with the NFT. The first performer 3 can sell his / her first performance information proved with the NFT, for example, and receive payment in return. In addition, the first performer 3 may record the picking movement of the right hand and the fret pressing action of the left hand as separate performance information in association with an NFT, even if the first performance information is of the same first performer 3. In this case, the first performer 3 can sell the picking movement of the right hand and the fret pressing action of the left hand at different prices, for example.

[0057] (More examples) In the above embodiment, a performance operation is shown as an example of performance information. However, the performance information may also include, for example, a conductor's conducting action, a dancer's dancing action, etc. For example, a first dancer may have a motion sensor attached to his or her upper body, and a second dancer may have a motion sensor attached to his or her lower body. The processor 34 may generate motion data that integrates the upper body action of the first dancer and the lower body action of the second dancer.

[0058] In the above embodiment, the first performance information is motion data showing the picking movement of the first performer 3's right hand on the guitar, and the second performance information is motion data showing the fret pressing movement of the second performer 7's left hand. However, for example, the first performance information may be motion data showing both the picking movement of the first performer 3's right hand and the fret pressing movement of the left hand. The second performer 7 may perform a movement not directly related to the performance (e.g., dancing), and the second performance information may be motion data showing the dancing movement of the second performer 7. In this way, the performance information generating method of this embodiment can control the 3D model data with motion data that integrates performance and movement other than performance (e.g., dancing), and can realize a new performance that was not possible in the past.

[0059] The second performer 7 may play something other than the guitar (for example, an electronic organ), and the second performance information may be motion data indicating the foot movements of the second performer 7 playing the electronic organ. As a result, the performance information generating method of this embodiment makes it possible to control the 3D model data using motion data that incorporates the foot movements of the electronic organ while playing the guitar, thereby realizing a new performance that was not possible in the past.

[0060] Alternatively, the second performer 7 may simply press a button on a keyboard or mouse, and the second performance information may be motion data indicating the button pressing action of the second performer 7 in time with the rhythm. The processor 34 of the PC 1C plays back the performance sound information included in the first performance information based on the information indicating the button pressing action included in the second performance information. As a result, the performance sound related to the instrument 5 played by the first performer 3 is played back in time with the button pressing action of the second performer 7. Therefore, the second performer 7 can provide a sophisticated performance by simply pressing a button in time with the rhythm.

[0061] The musical instruments are not limited to guitars, but may also include string instruments, wind instruments, percussion instruments, and the like.

[0062] The description of the present embodiment should be considered as illustrative in all respects and not restrictive. The scope of the present invention is indicated by the claims, not by the above-described embodiments. Furthermore, the scope of the present invention includes the scope equivalent to the claims. [Explanation of symbols]

[0063] 1: Performance information generating device 2: Third speaker 3: First speaker 5: Musical Instruments 7: 2nd performer 9: Musical Instruments 10: First location 20: Second location 30: Third location 31:Display unit 32: User interface 33: Flash memory 34: Processor 35: RAM 36: Communication I / F 38: Audio I / F 39: Audio I / F 50: Motion sensor 51: Motion sensor 75: Mike 90: 3D model data 91: 3D model data 100: Server

Claims

1. The system acquires first performance information relating to the performance of the first person and second performance information relating to the performance of the second person, which has been created and recorded in advance. A third performance information is generated by integrating the acquired first performance information and the second performance information. Output the generated third performance information. Method for generating performance information.

2. The second performance information includes information relating to the physical movements of the second person, The aforementioned output includes the output of the performance sound or the operation of the model data. The method for generating performance information according to claim 1.

3. The information relating to the bodily movement includes information indicating a dance movement or a button-pressing movement. The method for generating performance information according to claim 2.

4. The output includes the operation of model data representing a single performer and the output of sound corresponding to said operation, A method for generating performance information according to any one of claims 1 to 3.

5. At least one of the first performance information or the second performance information includes information about the performance, The third performance information includes information relating to the performance, A method for generating performance information according to any one of claims 1 to 3.

6. The aforementioned performance information includes information on the operation of the instrument, The system generates a sound based on the operation information of the aforementioned instrument. The method for generating performance information according to claim 5.

7. The aforementioned performance information includes information about the sound of the performance, Based on the information of the aforementioned performance sound, the performance sound is output. The method for generating performance information according to claim 5.

8. At least one of the first performance information or the second performance information includes information relating to the first performance, The third performance information includes information about the first performance, Based on the information regarding the first performance described above, the performance sound is output. Obtain information regarding the second performance, The second information relating to the performance is transmitted to another device via the network. A method for generating performance information according to any one of claims 1 to 3.

9. The first performance information and the second performance information each include first motion data and second motion data, The third performance information includes third motion data generated based on the first motion data and the second motion data. A method for generating performance information according to any one of claims 1 to 3.

10. The second performance information is obtained from the server as information recorded on the server. A method for generating performance information according to any one of claims 1 to 3.

11. Record a non-fungible token corresponding to at least one of the first performance information or the second performance information. A method for generating performance information according to any one of claims 1 to 3.

12. Obtain information regarding the sound played by the first or second person, Based on a trained model that has been trained to understand the relationship between the information regarding the performance sound and the first performance information or the second performance information, the first performance information or the second performance information is acquired. A method for generating performance information according to any one of claims 1 to 3.

13. The system acquires first performance information relating to the performance of the first person and second performance information relating to the performance of the second person, which has been created and recorded in advance. A third performance information is generated by integrating the acquired first performance information and the second performance information. Output the generated third performance information. A performance information generation device equipped with a processor.

14. The second performance information includes information relating to the physical movements of the second person, The aforementioned output includes the output of the performance sound or the operation of the model data. The performance information generating device according to claim 13.

15. The output includes the operation of model data representing a single performer and the sound output corresponding to said operation, The performance information generating device according to claim 13 or claim 14.

16. At least one of the first performance information or the second performance information includes information about the performance, The third performance information includes information relating to the performance, The performance information generating device according to claim 13 or claim 14.

17. The aforementioned performance information includes information on the operation of the instrument, The processor generates a sound based on the operation information of the instrument. The performance information generating device according to claim 16.

18. The aforementioned performance information includes information about the sound of the performance, The processor outputs the performance sound based on the performance sound information. The performance information generating device according to claim 16.

19. At least one of the first performance information or the second performance information includes information relating to the first performance, The third performance information includes information about the first performance, The processor outputs a performance sound based on the first performance information, Obtain information regarding the second performance, The second information relating to the performance is transmitted to another device via the network. The performance information generating device according to claim 13 or claim 14.

20. The first performance information and the second performance information each include first motion data and second motion data, The third performance information includes third motion data generated based on the first motion data and the second motion data. The performance information generating device according to claim 13 or claim 14.

21. The processor obtains the second performance information from the server as information recorded on the server. The performance information generating device according to claim 13 or claim 14.

22. The processor records a non-fungible token corresponding to at least one of the first performance information or the second performance information. The performance information generating device according to claim 13 or claim 14.

23. The system acquires first performance information relating to the performance of the first person and second performance information relating to the performance of the second person, which has been created and recorded in advance. A third performance information is generated by integrating the acquired first performance information and the second performance information. Output the generated third performance information. A program that instructs an information processing device to perform a task.

Images