Information processing method, information processing system and recording medium

Abstract

A learning model is obtained that learns the relationship between first performance data representing a performance performed by a performer and first evaluation data representing an evaluation given by an audience that receives the performance, second performance data is obtained, the second performance data is processed using the learning model, an evaluation of the second performance data is inferred, and second evaluation data representing the inference result is output.

Description

Technical Field

[0001] The present invention relates to a method, an information processing system and a program for inferring audience evaluation of performance data. Background Art

[0002] Conventionally, there are used performance evaluation devices that evaluate performance operations performed by a user. For example, Patent Document 1 discloses a technique for evaluating performance operations by selecting a portion of the entire performed music piece as an object.

[0003] Patent Document 1: Japanese Patent No. 3678135 Summary of the invention

[0004] Patent Document 1 discloses a technique for evaluating the accuracy of a performance performed by a user, but not a technique for inferring how the performance is evaluated by the audience (whether the audience likes it or not). In order for the user to appropriately improve his or her performance, it is required to infer the evaluation of the performance in advance.

[0005] An object of the present invention is to provide a method, an information processing system, and a program for appropriately inferring evaluation of performance data.

[0006] In order to achieve the above-mentioned purpose, one embodiment of the present invention involves a method implemented by a computer, obtaining a learning model that learns the relationship between first performance data representing a performance performed by a performer and first evaluation data representing an evaluation performed by an audience who receives the performance, obtaining second performance data, using the learning model to process the second performance data, inferring the evaluation of the second performance data, and outputting second evaluation data representing the inference result.

[0007] Effects of the Invention

[0008] According to the present invention, it is possible to appropriately infer evaluation of performance data. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 It is a diagram showing the overall configuration of an information processing system according to an embodiment of the present invention.

[0010] Figure 2 This is a block diagram showing the hardware configuration of the information processing device according to the embodiment of the present invention.

[0011] Figure 3 This is a block diagram showing the hardware configuration of a learning server according to an embodiment of the present invention.

[0012] Figure 4 It is a block diagram showing the functional structure of the information processing system according to the embodiment of the present invention.

[0013] Figure 5 Detailed description of the invention The following is a sequence diagram showing the machine learning process according to the embodiment of the present invention.

[0014] Figure 6 It is a sequence diagram showing the inference presentation process according to the embodiment of the present invention. DETAILED DESCRIPTION

[0015] Hereinafter, the embodiments of the present invention will be described in detail with reference to the accompanying drawings. Each embodiment described below is merely an example of a structure that can implement the present invention. Each of the following embodiments can be appropriately modified or changed according to the structure of the device to which the present invention is applied and various conditions. In addition, not all combinations of elements included in the following embodiments are necessary for implementing the present invention, and a part of the elements can be appropriately omitted. Therefore, the scope of the present invention is not limited by the structures described in the following embodiments. In addition, as long as there is no contradiction with each other, a structure obtained by combining multiple structures described in the embodiments can also be adopted.

[0016] Figure 1 1 is a diagram showing the overall structure of an information processing system S involved in an embodiment of the present invention. Figure 1 As shown, the information processing system S of this embodiment includes an information processing device 100 and a learning server 200. The information processing device 100 and the learning server 200 can communicate with each other via a network NW. A delivery server DS described later may also be connected to the network NW.

[0017] The information processing device 100 is an information terminal used by a user, such as a personal device such as a tablet terminal, a smartphone, a personal computer (PC), etc. The information processing device 100 may be connected to an electronic musical instrument EM described later in a wireless or wired manner.

[0018] The learning server 200 is a cloud server connected to the network NW, which can train the learning model M described later and supply the trained learning model M to other devices such as the information processing device 100. The server 300 is not limited to a cloud server, but can also be a server in a local network. In addition, the functions of the server 300 of this embodiment can also be realized by the collaborative action of the cloud server and the server in the local network.

[0019] In the information processing system S of the present embodiment, by inputting the performance data A of the inference object into a learning model M that performs machine learning on the relationship between the performance data A representing the performance performed by the performer and the evaluation data B representing the evaluation of the performance, it is possible to infer the evaluation of the input performance data A.

[0020] Figure 2is a block diagram showing the hardware structure of the information processing device 100. Figure 2 As shown, the information processing device 100 includes a CPU (Central Processing Unit) 101 , a RAM (Random Access Memory) 102 , a storage 103 , an input / output unit 104 , a sound receiving unit 105 , an imaging unit 106 , a transceiver unit 107 , and a bus 108 .

[0021] The CPU 101 is a processing circuit that executes various operations of the information processing device 100. The RAM 102 is a volatile storage medium that stores setting values ​​used by the CPU 101 and functions as a working memory for developing various programs. The storage 103 is a nonvolatile storage medium that stores various programs and data used by the CPU 101.

[0022] The input / output unit 104 is an element (user interface) that receives user operations on the information processing device 100 and displays various information, and is constituted by, for example, a touch panel.

[0023] The sound pickup unit 105 is an element, such as a microphone, that converts the collected sound into an electrical signal and supplies it to the CPU 101. The sound pickup unit 105 may be built in the information processing device 100 or connected to the information processing device 100 via an interface (not shown).

[0024] The imaging unit 106 is a component, such as a digital camera, that converts an imaged image into an electrical signal and supplies the signal to the CPU 101. The imaging unit 106 may be built in the information processing device 100 or connected to the information processing device 100 via an interface (not shown).

[0025] The transceiver 107 is an element for transmitting and receiving data with other devices such as the learning server 200. The transceiver 107 is connected to the electronic musical instrument EM used by the user when playing music and can transmit and receive data. The transceiver 107 can include multiple models (for example, a Bluetooth (registered trademark) model and a Wi-Fi (registered trademark) model for short-range wireless communication).

[0026] The bus 108 is a signal transmission path that connects the hardware elements of the above-mentioned information processing device 100 to each other.

[0027] Figure 3 2 is a block diagram showing the hardware structure of the learning server 200. Figure 3 As shown, the learning server 200 includes a CPU 201 , a RAM 202 , a storage 203 , an input unit 204 , an output unit 205 , a transceiver unit 206 , and a bus 207 .

[0028] The CPU 201 is a processing circuit that executes various operations of the learning server 200. The RAM 202 is a volatile storage medium that stores setting values ​​used by the CPU 201 and functions as a working memory for developing various programs. The storage 203 is a nonvolatile storage medium that stores various programs and data used by the CPU 201.

[0029] The input unit 204 is an element for receiving operations on the learning server 200 , and receives input signals from, for example, a keyboard and a mouse connected to the learning server 200 .

[0030] The output unit 205 is an element that displays various information, and outputs a video signal to, for example, a liquid crystal display connected to the learning server 200 .

[0031] The transceiver 206 is an element that transmits and receives data with other devices such as the information processing device 100 , and is, for example, a network interface card (NIC).

[0032] The bus 207 is a signal transmission path that connects the hardware elements of the above-mentioned learning server 200 to each other.

[0033] The CPU 101, 201 of each of the above-mentioned devices 100, 200 reads the program stored in the storage 103, 203 to the RAM 102, 202 and executes it, thereby realizing the following functional blocks (control units 150, 250, etc.) and various processes involved in this embodiment. Each CPU is not limited to a conventional CPU, and can be a DSP, an inference processor, or any combination of two or more of the above. In addition, the various processes involved in this embodiment can also be realized by executing a program by one or more processors such as a CPU, a DSP, an inference processor, a GPU, etc.

[0034] Figure 4 It is a block diagram showing the functional structure of the information processing system S according to the embodiment of the present invention.

[0035] The learning server 200 includes a control unit 250 and a storage unit 260. The control unit 250 is a functional block that comprehensively controls the operation of the learning server 200. The storage unit 260 is composed of a RAM 202 and a storage 203, and stores various data (particularly, performance data A and evaluation data B) used by the control unit 250. The control unit 250 includes a server authentication unit 251, a data acquisition unit 252, a data pre-processing unit 253, a learning processing unit 254, and a model issuing unit 255 as sub-functional blocks.

[0036] The server authentication unit 251 is a functional block that cooperates with the information processing device 100 (authentication unit 151) to authenticate the user. The server authentication unit 251 determines whether the authentication data supplied from the information processing device 100 is consistent with the authentication data stored in the storage unit 260, and sends the authentication result (permission or rejection) to the information processing device 100.

[0037] The data acquisition unit 252 is a functional block that receives transmission data from an external transmission server DS via a network NW and obtains performance data A and evaluation data B. The transmission server DS is, for example, a server that transmits animations such as live animations that include images and sounds as transmission data. The transmission data includes image data (e.g., animation data), sound data (e.g., audio data), and operation data (e.g., MIDI data) representing the performance of the performer. In addition, the transmission data includes subjective data for the performance. Subjective data is an evaluation value given by the audience to the performance of the performer, and is associated with the animation in a sequential manner. For example, the evaluation value of the evaluation data can be given to the moment of the corresponding animation, or the serial number (frame number) of the animation can be given. In addition, the animation and subjective data can also be constituted as a whole. In addition, it is preferred that operation data such as MIDI data representing the performance operation performed by the performer during the performance is included in the transmission data. The operation data can also include the pedal operation of the electronic piano and the effect operation of the electric guitar.

[0038] The data acquisition unit 252 acquires the performance data A by dividing the image data and the sound data included in the received transmission data into a plurality of performance segments in time series, and stores the obtained performance data A in the storage unit 260. The data acquisition unit 252 may divide the image data and the sound data into performance segments for each phrase indicated by a pause in the performance, may divide the image data and the sound data into performance segments based on a motif of the performance, or may divide the image data and the sound data into performance segments based on a chord pattern.

[0039] Furthermore, the performance data A may include the operation data divided in time series instead of the sound data divided in time series or in addition thereto. That is, the performance data A includes either or both the sound data representing the sound generated by the performance and the operation data generated based on the performance of the electronic musical instrument EM.

[0040] In addition, the data acquisition unit 252 acquires evaluation data B including evaluation segments indicating evaluations for each divided performance segment based on the subjective data and evaluation time included in the received transmission data, and stores the acquired evaluation data B in the storage unit 260. The evaluation data B is data indicating the transition of the temporal evaluation of the performance data A that is structured temporally. When the evaluation segment included in the evaluation data B includes the corresponding performance segment, a serial number corresponding to the performance segment and the evaluation segment may be added, and the evaluation segment may be embedded in the corresponding performance segment. The data acquisition unit 252 stores the acquired performance data A and evaluation data B in the storage unit 260.

[0041] The data preprocessing unit 253 is a functional block that performs data preprocessing such as scaling on the performance data A and the evaluation data B stored in the storage unit 260 so that the data are in a format suitable for training the learning model M (machine learning).

[0042] The learning processing unit 254 is a functional block that uses the performance data A after data pre-processing as input data and the evaluation data B after data pre-processing as teacher data to train the learning model M. For the learning model M of this embodiment, any machine learning model can be used. Preferably, a recursive neural network (RNN) and its derivatives (long short-term memory (LSTM), gated recursive unit (GRU), etc.) suitable for time series data are used in the learning model M. The learning model M can also be constructed according to an algorithm based on the attention mechanism.

[0043] The model issuing unit 255 is a functional block that supplies the learning model M trained by the learning processing unit 254 to the information processing device 100 .

[0044] The information processing device 100 includes a control unit 150 and a storage unit 160. The control unit 150 is a functional block that comprehensively controls the operation of the information processing device 100. The storage unit 160 is composed of a RAM 102 and a storage 103, and stores various data used by the control unit 150. The control unit 150 includes an authentication unit 151, a performance acquisition unit 152, an animation acquisition unit 153, a data pre-processing unit 154, an inference processing unit 155, and an evaluation prompting unit 156 as sub-functional blocks.

[0045] The authentication unit 151 is a functional block that cooperates with the learning server 200 (server authentication unit 251) to authenticate the user. The authentication unit 151 sends authentication data such as a user identifier and a password input by the user using the input / output unit 104 to the learning server 200, and permits or denies the user's access based on the authentication result received from the learning server 200. The authentication unit 151 can supply the user identifier of the authenticated user (whose access is permitted) to other functional blocks.

[0046] The performance acquisition unit 152 is a functional block that acquires either or both of the sound data and operation data representing the user's performance. The sound data and the operation data are data (sound characteristic data) representing the characteristics of multiple sounds contained in the music involved in the performance (for example, the sound generation time and pitch), and are a kind of high-dimensional time series data that expresses the performance performed by the user. The performance acquisition unit 152 can also acquire the sound data based on the electrical signal generated by the sound receiving unit 105 receiving the sound obtained by the user's performance. In addition, the performance acquisition unit 152 can also acquire the operation data generated based on the performance of the electronic musical instrument EM performed by the user from the electronic musical instrument EM via the transceiver unit 107. The electronic musical instrument EM can be, for example, an electronic keyboard instrument such as an electronic piano, an electronic string instrument such as an electric guitar, or an electronic wind instrument such as an electric wind synthesizer. The performance acquisition unit 152 supplies the acquired sound characteristic data to the data pre-processing unit 154. In addition, the performance acquisition unit 152 can also assign the user identifier supplied from the authentication unit 151 to the sound characteristic data and send it to the learning server 200.

[0047] The animation acquisition unit 153 is a functional block that acquires image data representing the user's performance. Image data is motion data representing the characteristics of the motion of the performing user (performer), and is a type of high-dimensional time series data that represents the performance performed by the user. The animation acquisition unit 153 can acquire motion data based on the electrical signal generated by the shooting unit 106 shooting the performing user. The motion data is, for example, data of the skeleton (skeleton) of the user acquired in a time series. The animation acquisition unit 153 supplies the acquired image data to the data pre-processing unit 154. In addition, the animation acquisition unit 153 can also assign the user identifier supplied from the authentication unit 151 to the image data and send it to the learning server 200.

[0048] The data preprocessing unit 154 is a functional block that performs data preprocessing such as scaling the performance data A including the sound characteristic data supplied from the performance acquisition unit 152 and the image data supplied from the animation acquisition unit 153 so as to make the performance data A suitable for the inference performed by the learning model M.

[0049] The inference processing unit 155 is a functional block that infers evaluation data B indicating evaluation of the performance data A by inputting the pre-processed performance data A as input data to the learning model M trained by the aforementioned learning processing unit 254. In addition, the evaluation data B includes the evaluation segment indicating the evaluation of each of the plurality of performance segments included in the performance data A as described above.

[0050] The evaluation prompting unit 156 is a functional block that prompts the user of the evaluation data B inferred by the inference processing unit 155. The evaluation prompting unit 156, for example, sequentially displays the evaluation of each of the plurality of performance segments included in the performance data A on the input / output unit 104. In addition, the evaluation prompting unit 156 may present the evaluation data B to the user auditorily or tactilely instead of visually presenting the evaluation data B or in addition thereto. In addition, the evaluation prompting unit 156 may also display the above-mentioned evaluation on a display unit of another device, such as the electronic musical instrument EM.

[0051] Figure 5 1 is a timing diagram showing the machine learning process of the information processing system S according to the embodiment of the present invention. The machine learning process of the present embodiment is executed in the learning server 200. In addition, the machine learning process of the present embodiment may be executed periodically or in response to a request from the information processing device 100 based on a user instruction.

[0052] In step S510, the data acquisition unit 252 acquires the performance data A and the evaluation data B based on the delivery data received from the delivery server DS, and stores them in the storage unit 260. The delivery data may be acquired in advance by the data acquisition unit 252 and stored in the storage unit 260, or may be acquired by the data acquisition unit 252 in this step.

[0053] In step S520 , the data pre-processing unit 253 reads out the data set including the performance data A and the evaluation data B stored in the storage unit 260 and performs data pre-processing.

[0054] In step S530, the learning processing unit 254 trains the learning model M based on the data set pre-processed in step S520, using the performance data A as the distribution data and the evaluation data B as the teacher data, and stores the trained learning model M in the storage unit 260. For example, in the case where the learning model M is a neural network system, the learning processing unit 254 may also perform machine learning of the learning model M using an error back propagation method or the like.

[0055] In step S540 , the model issuing unit 255 supplies the learning model M trained in step S530 to the information processing device 100 via the network NW. The control unit 150 of the information processing device 100 stores the received learning model M in the storage unit 160 .

[0056] Figure 6 1 is a sequence diagram showing the inference presentation process of the information processing system S according to the embodiment of the present invention. In this embodiment, the information processing device 100 infers the evaluation of each performance passage and visually presents the inferred evaluation to the user.

[0057] In step S610 , the performance acquisition unit 152 acquires one or both of the sound data and the operation data (sound characteristic data) from the electronic musical instrument EM or the like as described above, and supplies the acquired data to the data pre-processing unit 154 .

[0058] In step S620 , the moving image acquisition unit 153 acquires the image data as described above and supplies the image data to the data pre-processing unit 154 .

[0059] In step S630 , the data preprocessing unit 154 performs data preprocessing on the performance data A including the sound characteristic data supplied from the performance acquisition unit 152 in step S610 and the image data supplied from the animation acquisition unit 153 in step S620 , and supplies the preprocessed performance data A to the inference processing unit 155 .

[0060] In step S640, the inference processing unit 155 inputs the performance data A supplied from the data pre-processing unit 154 as input data to the trained learning model M stored in the storage unit 160. The learning model M processes the input performance data A and infers the audience's evaluation of each performance segment included in the performance data A. The inference value representing the evaluation can be a discrete value or a continuous value. The inferred evaluation (evaluation data B) for each performance segment is supplied from the inference processing unit 155 to the evaluation prompt unit 156.

[0061] In step S650, the evaluation presentation unit 156 presents the evaluation data B inferred by the inference processing unit 155 in step S640 to the user. Various methods are conceivable for presenting the evaluation data B to the user.

[0062] For example, an application is envisioned in which the reaction of a virtual audience (e.g., an avatar in a VR (Virtual Reality) space) to a user's performance is simulated and displayed. In the above application, the evaluation prompt unit 156 displays the reaction of the virtual audience on the input / output unit 104 based on the evaluation data B in synchronization with the reproduction of the performance data A. The evaluation prompt unit 156 displays a reaction indicating a high atmosphere, such as standing up or cheering, when the inferred evaluation is higher than the threshold, and displays a reaction indicating a low atmosphere, such as sitting still, being quiet, or booing, when the inferred evaluation is lower than the threshold.

[0063] In the above application, the evaluation presenting unit 156 displays the waveform representing the performance data A and the transition of the evaluation data B corresponding to the performance data A as a graph on the input / output unit 104 .

[0064] Furthermore, the above-described inference display processing of steps S610 to S650 may be executed in real time in parallel with the input of the performance data A to the information processing device 100 , or may be executed post-process on the performance data A stored in the information processing device 100 .

[0065] As described above, in the information processing system S of this embodiment, the evaluation corresponding to each of the plurality of performance segments included in the performance data A is appropriately inferred by the trained learning model M. The information processing device 100 presents the inferred evaluation of each performance segment to the user. As a result, the user can predict how the performance he or she performs will be evaluated by the audience.

[0066] <Modification>

[0067] The above embodiments can be modified in various ways. Specific modified forms are exemplified below. Two or more forms arbitrarily selected from the above embodiments and the following examples can be appropriately combined within the scope that they do not contradict each other.

[0068] In the above-mentioned embodiment, the performance data A is divided into a plurality of performance segments in time series and used for the learning process and the inference process. However, the performance data A may not be divided but may correspond to one music piece.

[0069] Regarding the above-mentioned embodiment, various methods can be used to divide the performance data A. For example, the plurality of performance sections may be a plurality of performance intervals divided into a piece of music at predetermined time intervals, or may be a plurality of phrases determined based on the performance data A.

[0070] The evaluation data B in the above-described embodiment is subjective data indicating the evaluation value given by the viewer to the performance of the performer indicated by the transmission data. However, other information may be used as the evaluation data B.

[0071] For example, the number of submissions submitted by viewers in association with the performance of the performer may be used as evaluation data B. The submission data is, for example, text information associated with an animation segment included in the animation, included in the transmission data, and the number of submissions is counted for each performance segment.

[0072] In addition, for example, reaction data representing the behavior of the audience of the performance may be used as evaluation data B. The reaction data is information representing characteristics related to the actions of the audience of the performance. The data acquisition unit 252 can analyze the image (audience image) during the period in which the audience is displayed in the music performance animation included in the transmission data to obtain the reaction data. The reaction data may be, for example, data that sequentially obtains the skeleton (skeleton) of each audience member, data that represents the size of the overall movement of the audience, data that represents the facial expression of each audience member, or data that represents the body temperature of the audience obtained by an infrared camera or the like.

[0073] In the above-mentioned embodiment, the evaluation presentation unit 156 visually presents the evaluation data B to the user. Instead of presenting the evaluation data B or in addition thereto, the control unit 150 may present candidates for the image effects for the animation represented by the performance data A to improve the inferred evaluation. The image effects for the animation are, for example, information indicating the switching timing of the camera angles when the animation is shot by multiple cameras, and the start / end timing of fade-out.

[0074] In the above-described embodiment, the information processing device 100 uses the learning model M supplied from the learning server 200 to infer the evaluation. However, each process involved in the inference of the evaluation may be performed by any device constituting the information processing system S. For example, the learning server 200 may pre-process the performance data A supplied from the information processing device 100, and input the pre-processed performance data A as input data into the learning model M stored in the storage unit 260, thereby inferring the evaluation for the performance data A. According to the structure of this variation, the learning server 200 can perform inference processing based on the learning model M using the performance data A as input data. As a result, the processing load of the information processing device 100 is reduced.

[0075] Furthermore, the electronic musical instrument 100 of the above-described embodiment may have the function of the control device 200 , and the control device 200 may have the function of the electronic musical instrument 100 .

[0076] In addition, the same effect as the present invention can be achieved by reading out a storage medium storing each control program represented by the software for implementing the present invention to each device. In this case, the program code itself read out from the storage medium realizes the new function of the present invention, and the non-temporary computer-readable recording medium storing the program code constitutes the present invention. In addition, the program code can also be provided through a transmission medium, etc. In this case, the program code itself constitutes the present invention. In addition, as the storage medium in the above case, in addition to ROM, a floppy disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a DVD-ROM, a DVD-R, a magnetic tape, a non-volatile memory card, etc. can also be used. "Non-temporary computer-readable recording medium" also includes a medium that stores the program for a certain period of time, such as a volatile memory (e.g., DRAM (Dynamic Random Access Memory)) inside a computer system of a server or client in the case where the program is sent via a network such as the Internet or a communication line such as a telephone line.

[0077] The present invention has been described in detail above based on the preferred embodiments of the present invention, but the present invention is not limited to these specific embodiments, and various modes within the scope of the present invention are also included in the present invention. Parts of the above-mentioned embodiments may be appropriately combined.

[0078] Description of the label

[0079] 100 information processing device, 150 control unit, 160 storage unit, 200 learning server, 250 control unit, 260 storage unit A performance data, B evaluation data, DS transmission server, EM electronic musical instrument, M learning model, S information processing system

Claims

1. An information processing method, which is implemented by a computer, acquiring a learning model that learns a relationship between first performance data representing a performance performed by a performer and first evaluation data representing an evaluation performed by an audience that received the performance, Get the second performance data, The second performance data is processed using the learning model to infer second evaluation data representing evaluations by audience members who received the performance represented by the second performance data. outputting the second evaluation data, The first performance data is divided into a series of performance segments. The first evaluation data includes a plurality of evaluation segments associated with any one of the series of performance segments. The first performance data includes sound data representing a performed sound and video data representing a video of a performer performing the performance.

2. The information processing method according to claim 1, characterized in that: The division of the first performance data into the series of performance sections is performed for each phrase.

3. The information processing method according to claim 1, characterized in that: The first performance data is divided into the series of performance sections based on the theme.

4. The information processing method according to claim 1, characterized in that: The first performance data is divided into the series of performance sections based on a chord pattern.

5. The information processing method according to any one of claims 1 to 4, wherein: The first performance data further includes operation data indicating a performance operation of a performer during the performance.

6. The information processing method according to any one of claims 1 to 4, wherein: The image data is motion data representing features of motions of the player in the performance.

7. The information processing method according to any one of claims 1 to 4, wherein: The first evaluation data includes at least any one of subjective data indicating an evaluation given by an audience to the performance, response data indicating an audience response to the performance, and contribution data related to the amount of contributions to the performance.

8. The information processing method according to any one of claims 1 to 4, wherein: Candidates for the video effect of the animation represented by the second performance data are presented so as to improve the evaluation represented by the second evaluation data.

9. The information processing method according to any one of claims 1 to 4, characterized in that: The first performance data and the first evaluation data are received from the outside as transmission data.

10. The information processing method according to any one of claims 1 to 4, characterized in that: The audience is a virtual audience.

11. An information processing system comprising a control unit, The control unit acquires a learning model that learns a relationship between first performance data representing a performance performed by a performer and first evaluation data representing an evaluation performed by an audience that received the performance. Get the second performance data, The second performance data is processed using the learning model to infer second evaluation data representing evaluations by audience members who received the performance represented by the second performance data. outputting the second evaluation data, The first performance data is divided into a series of performance segments. The first evaluation data includes a plurality of evaluation segments associated with any one of the series of performance segments. The first performance data includes sound data representing a performed sound and video data representing a video of a performer performing the performance.

12. The information processing system according to claim 11, characterized in that: The division of the first performance data into the series of performance sections is performed for each phrase.

13. The information processing system according to claim 11, characterized in that: The first performance data is divided into the series of performance sections based on the theme.

14. The information processing system according to claim 11, characterized in that: The first performance data is divided into the series of performance sections based on a chord pattern.

15. The information processing system according to any one of claims 11 to 14, wherein: The first performance data further includes operation data indicating a performance operation of a performer during the performance.

16. The information processing system according to any one of claims 11 to 14, wherein: The image data is motion data representing features of motions of the player in the performance.

17. The information processing system according to any one of claims 11 to 14, wherein: The first evaluation data includes at least any one of subjective data indicating an evaluation given by an audience to the performance, response data indicating an audience response to the performance, and contribution data related to the amount of contributions to the performance.

18. The information processing system according to any one of claims 11 to 14, wherein: Candidates for the video effect of the animation represented by the second performance data are presented so as to improve the evaluation represented by the second evaluation data.

19. The information processing system according to any one of claims 11 to 14, characterized in that: The first performance data and the first evaluation data are received from the outside as transmission data.

20. The information processing system according to any one of claims 11 to 14, characterized in that: The audience is a virtual audience.

21. A recording medium storing an information processing program for causing a computer to execute the following processing: acquiring a learning model that learns a relationship between first performance data representing a performance performed by a performer and first evaluation data representing an evaluation performed by an audience that received the performance, Get the second performance data, The second performance data is processed using the learning model to infer second evaluation data representing evaluations by audience members who received the performance represented by the second performance data. outputting the second evaluation data, The first performance data is divided into a series of performance segments. The first evaluation data includes a plurality of evaluation segments associated with any one of the series of performance segments. The first performance data includes sound data representing a performed sound and video data representing a video of a performer performing the performance.

Images