Control device, system, control method, and program

Abstract

To provide a control device, system, control method, and program for achieving effective acoustic effects.SOLUTION: In a system 90, a control device 300 comprises: a storage unit that stores information of performance data processing device 201, 202 to 209 for processing performance data 11, 12 received via network 80 using an instrument 500 to generate processed performance data 21, 22 to 29; and a matching unit that matches performance data communication devices with the performance data processing devices using the information about the performance data processing device stored in the storage unit, so as to cause the performance data processing devices to process the performance data transmitted by a transmission unit of the performance data communication devices 101, 102, the transmission unit for transmitting the performance data.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a control device, a system, a control method, and a program. [Background technology]

[0002] Patent document 1 describes "an ensemble device that is placed at a first location to realize an ensemble performance between a first performer at a first location and a second performer at a second location different from the first location via a network." [Prior art document] [Patent documents] [Patent Document 1] JP 2016-200712 A Summary of the Invention [Means for solving the problem]

[0003] According to one embodiment of the present invention, there is provided a control device. The control device may include a memory unit that stores information about a performance data processing device that processes performance data of a musical instrument performance received via a network to generate processed performance data. The control device may also include a correspondence unit that associates the performance data communication device with the performance data processing device using the information about the performance data processing device stored in the memory unit, so as to have the performance data processing device process the performance data transmitted by the transmission unit of a performance data communication device that transmits the performance data.

[0004] In the control device, the storage unit may store information on a plurality of performance data processing devices that perform different processing of performance data, and the association unit may associate a performance data processing device designated by the performance data communication device with the performance data communication device among the plurality of performance data processing devices.

[0005] Any of the control devices may include a destination designation receiving unit that receives designation of a destination to which the processed performance data is to be transmitted, and the associating unit may control the processed performance data to be transmitted via the network to the destination designated by the destination designation receiving unit.

[0006] Any of the control devices may include a communication status acquisition unit that acquires a communication status of a communication path from the performance data processing device to the destination. The control device may include a processing adjustment unit that adjusts processing of the performance data by the performance data processing device based on the communication status. The processing adjustment unit may control the content of the processing so that the data volume of the processed performance data is less than the data volume corresponding to the communication delay indicated by the communication status.

[0007] In any of the above control devices, the associating unit may associate a first performance data communication device with a first performance data processing device, and may associate a second performance data communication device with a second performance data processing device. The control device may include a receiving unit that receives from the first performance data processing device first processed performance data obtained by processing the performance data received from the first performance data communication device, and receives from the second performance data processing device second processed performance data obtained by processing the performance data received from the second performance data communication device. The control device may include an ensemble data generating unit that generates ensemble data by combining the first processed performance data and the second processed performance data received by the receiving unit.

[0008] In any of the control devices, the associating unit may associate the plurality of performance data communication devices with the performance data processing device, and the associating unit may cause the performance data processing device to process the performance data received from each of the plurality of performance data communication devices and synthesize the plurality of processed performance data to generate ensemble data.

[0009] According to one embodiment of the present invention, there is provided a system including any one of the control devices and a performance data generating device. The performance data generating device may have a pseudo-generation unit that generates pseudo-performance data of a specified user playing a specified piece of music. The system may also include an ensemble data generating unit that generates ensemble data by combining processed performance data obtained by the performance data processing device associated with the performance data communication device by the association unit, which processes performance data received from the performance data communication device, and the pseudo-performance data pseudo-generated by the pseudo-generation unit.

[0010] In the system, the performance data generation device may include a learning model storage unit that stores performance data generated by a user playing a musical piece on an instrument and a learning model that takes score data as input and outputs performance data, the learning model being generated by machine learning using training data including score data for the musical piece. In the system, the performance data generation device may include a score data receiving unit that receives score data. In the system, the pseudo-performance generation unit may input the score data received by the score data receiving unit to the learning model corresponding to the specified user, and obtain the performance data output from the learning model as pseudo-performance data.

[0011] In any of the above systems, the learning model storage unit may store ensemble data generated by a first user and a second user playing together, and a learning model generated by machine learning using learning data including performance data of the first user in the ensemble, the learning model taking performance data of the first user as input and performance data of the second user as output. The pseudo-performance generation unit may input the performance data of the first user to the learning model and obtain the performance data output from the learning model as pseudo-performance data.

[0012] According to one embodiment of the present invention, there is provided a control method executed by a computer. The control method may include a storing step of storing, in a storage unit, information about a performance data processing device that processes performance data of a musical instrument performance received via a network to generate processed performance data. The control method may also include a correlating step of correlating a performance data communication device and a performance data processing device using the information about the performance data processing device stored in the storage unit, so as to have the performance data processing device process the performance data transmitted by the transmission unit of a performance data communication device that transmits the performance data.

[0013] According to one embodiment of the present invention, there is provided a program for causing a computer to execute the control method.

[0014] The above summary of the invention does not list all of the necessary features of the present invention, and subcombinations of these features may also constitute inventions. [Brief explanation of the drawings]

[0015] [Figure 1] An example of a system 90 is shown schematically. [Figure 2] An example of the flow of processing in the system 90 is shown in outline. [Figure 3] An example of the flow of processing in the system 90 is shown in outline. [Figure 4] An example of the flow of processing in the system 90 is shown in outline. [Figure 5] An example of the flow of processing in the system 90 is shown in outline. [Figure 6] An example of the flow of processing in the system 90 is shown in outline. [Figure 7] An example of the flow of processing in the system 90 is shown in outline. [Figure 8] An example of the flow of processing in the system 90 is shown in outline. [Figure 9] An example of a system 90 is shown schematically. [Figure 10] 2 shows an example of a functional configuration of the control device 300. [Figure 11] 1 shows an example of the functional configuration of a performance data generating device 400. [Figure 12] An example of the hardware configuration of a computer 1200 that functions as the performance data communication device 100, the performance data processing device 200, the control device 300, the performance data generation device 400, or the ensemble data generation unit 92 is shown in the figure. DETAILED DESCRIPTION OF THE INVENTION

[0016] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the scope of the invention according to the claims. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0017] In an ensemble using an existing remote ensemble service, for example, if a user wanted to use an effector or other device to process the sound of an electronic guitar or other instrument before participating in the ensemble, the sound processing had to be performed in the user's local environment and then the processed performance data had to be transmitted over the Internet. As a result, in order to perform the desired sound processing, the user had to purchase and own relatively expensive sound processing equipment such as an effector or amplifier simulator.

[0018] The control device according to this embodiment has a configuration that contributes to solving these problems. For example, the control device controls the connection between a sound processing device and a user terminal on a network, and causes the sound processing device to process performance data received from the user terminal via the network. This allows a user (performer) to achieve the desired sound processing without having to purchase and own multiple sound processing devices. Furthermore, users other than performers, such as recording engineers, PA (Public Address) engineers, and listeners, can also link the data of their favorite performers to their own usage environment and process performances, broadening the scope of usage patterns.

[0019] 1 shows an example of a system 90. The system 90 includes a performance data communication device 100, a performance data processing device 200, and a control device 300. The performance data communication device 100, the performance data processing device 200, and the control device 300 are interconnected via a network 80 so as to be able to communicate with each other.

[0020] The performance data communication device 100 has a function of transmitting performance data 10, which is generated when a user 50 plays a musical instrument 500, via a network 80. The type of musical instrument 500 is not particularly limited. The musical instrument 500 may be an electronic instrument such as an electronic guitar, electronic bass, electronic drum, electronic violin, or electronic piano. The musical instrument 500 may also be an acoustic instrument such as a guitar, bass, drum, violin, or piano. In this case, a pickup device may be attached to the acoustic instrument, and the performance data communication device 100 may generate the performance data 10 by acquiring an electrical signal from the pickup device. The performance data communication device 100 may also generate the performance data 10 by collecting the sound of the musical instrument 500 with a microphone and acquiring the electrical signal.

[0021] The performance data communication device 100 may have a performance input unit 130 and a conversion unit 150. The performance data communication device 100 may receive an analog signal of a performance using a musical instrument 500 at the performance input unit 130, and the conversion unit 150 may convert the analog signal into a digital signal, thereby generating performance data 10. The performance data communication device 100 has a transmission unit 110. The transmission unit 110 transmits the performance data 10. The performance input unit 130 may receive an analog signal from the musical instrument 500. The performance input unit 130 may receive an analog signal from a pickup device attached to the musical instrument 500. The performance input unit 130 may receive an analog signal from a microphone.

[0022] The performance input unit 130 may accept singing by the user 50. In this case, the performance data communication device 100 may generate vocal data by accepting an analog signal of the singing by the user 50 at the performance input unit 130 and converting the analog signal into a digital signal at the conversion unit 150. The transmission unit 110 may transmit the vocal data.

[0023] The performance input unit 130 may accept audio input by field recording. The subject of field recording is not particularly limited. For example, the performance input unit 130 may accept as field sounds sounds of natural environments such as the sea or mountains, or sounds of a busy urban area. The performance input unit 130 may accept as field sounds sounds played outdoors using a musical instrument 500. The performance input unit 130 may accept as field sounds sounds sung outdoors by a user 50. In these cases, the performance data communication device 100 may generate field sound data by receiving an analog signal of the field sound at the performance input unit 130 and converting the analog signal into a digital signal at the conversion unit 150. The transmission unit 110 may transmit the field sound data.

[0024] The performance data processing device 200 processes the performance data 10 received via the network 80 to generate processed performance data 20. The processing of the performance data 10 by the performance data processing device 200 may be processing that changes the acoustic characteristics of the performance. For example, the processing of the performance data 10 by the performance data processing device 200 may include distortion processing, oscillation processing, reverberation processing, dynamics adjustment processing, filter processing, amplifier simulation processing, harmony processing, volume processing, synthesizer processing, panning processing, tempo processing, etc. For example, the processing of the performance data 10 by the performance data processing device 200 may include distortion, tremolo, vibrato, delay, reverb, compressor, filter, wah, preamp, simulator, octaver, pitch shifter, harmonizer, chorus, equalizer, booster, guitar synthesizer, etc.

[0025] The processing of the performance data 10 by the performance data processing device 200 may be processing that takes advantage of the communication characteristics of the network 80. For example, the processing of the performance data 10 by the performance data processing device 200 may be processing that realizes a feedback performance technique that takes advantage of the communication characteristics of the network 80. For example, the processing of the performance data 10 by the performance data processing device 200 may be processing that realizes a feedback performance technique that utilizes network echo. For example, the control device 300 controls the performance data processing device 200 to return the performance data 10 to the performance data communication device 100 via multiple different communication paths. As a result, the timing at which the performance data 10 reaches the performance data communication device 100 differs for each communication path, so that when the performance data 10 is output from the performance data communication device 100, it sounds as if it has been subjected to feedback processing.

[0026] In this case, the control device 300 may control the performance data processing device 200 to return the processed performance data 20, instead of the performance data 10, to the performance data communication device 100 via a plurality of different communication paths. For example, by controlling the control device 300 to return the processed performance data 20, which is the performance data 10 that has been subjected to distortion processing, to the performance data communication device 100 via a plurality of different communication paths, the processed performance data 20 will sound as if it has been subjected to both distortion processing and feedback processing when output from the performance data communication device 100.

[0027] For example, the processing of the performance data 10 by the performance data processing device 200 is a processing that realizes a feedback performance technique by having the performance data processing device 200 transmit multiple pieces of performance data 10 to the network 80 at different transmission timings. For example, the control device 300 may control the performance data processing device 200 so that after the performance data processing device 200 transmits the performance data 10 to the network 80, the control device 300 transmits the same performance data 10 to the network 80 each time a predetermined delay interval has elapsed. As a result, each of the multiple pieces of performance data 10 arrives at the performance data communication device 100 with a timing difference, and when the performance data is output from the performance data communication device 100, it sounds as if it has been subjected to feedback processing.

[0028] In this case, the processing of the performance data 10 by the performance data processing device 200 may be processing that realizes a feedback performance style by transmitting a plurality of processed performance data 20 to the network 80 at different transmission timings instead of a plurality of performance data 10. For example, after the performance data processing device 200 transmits the processed performance data 20 to the network 80, the control device 300 may control the performance data processing device 200 to transmit the same processed performance data 20 to the network 80 every time a predetermined delay interval elapses.

[0029] For example, the processing of the performance data 10 by the performance data processing device 200 is a processing that realizes a feedback performance style by performing a loop process of inputting the processed performance data 20 back into the performance data processing device 200. For example, the control device 300 controls the performance data processing device 200 to execute a loop process in which the processed performance data 20, generated by processing the performance data 10, is transmitted to the network 80, and the performance data processing device 200 inputs the processed performance data 20 itself and transmits it to the network 80 together with the processed performance data 20 generated by processing the next performance data 10. As a result, the processed performance data 20 arrives at the performance data communication device 100 together with the previous processed performance data 20, so that when it is output from the performance data communication device 100, it sounds as if it has been subjected to feedback processing.

[0030] The control device 300 may control the performance data processing device 200 to further process the processed performance data 20 input thereto. For example, the control device 300 controls the performance data processing device 200 to transmit the processed performance data 20, which the performance data processing device 200 has generated by processing the performance data 10, over the network 80, and to process the processed performance data 20 together with the next performance data 10 and transmit the processed performance data 20 obtained by processing the processed performance data 20 together with the next performance data 10 over the network 80. The control device 300 may control the performance data processing device 200 to start a loop process in response to an instruction from the performance data communication device 100, or to control the performance data processing device 200 to end the loop process in response to an instruction from the performance data communication device 100.

[0031] The performance data processing device 200 may include a receiving unit 220 and a performance data processing unit 230. The receiving unit 220 may receive performance data 10 via a network 80. The performance data processing unit 230 may process the performance data 10 received by the receiving unit 220 to generate processed performance data 20. The performance data processing device 200 may include a transmitting unit 210. The transmitting unit 210 may transmit the processed performance data 20 generated by the performance data processing unit 230.

[0032] The control device 300 may control various devices that make up the system 90. The control device 300 may control communication connections between the various devices that make up the system 90.

[0033] The control device 300 stores information about the performance data processing device 200 (sometimes referred to as processing device information). The processing device information may include processing device identification information that can identify the performance data processing device 200. The processing device information may include network information that can identify the performance data processing device 200 on the network 80. The network information may include, for example, an IP address assigned to the performance data processing device 200. The processing device information may include processing content information that indicates the processing content performed by the performance data processing device 200. The control device 300 may store processing device information for multiple performance data processing devices 200. For example, the control device 300 may store a processing device list that includes processing device information for multiple performance data processing devices 200 that can be used by the performance data communication device 100.

[0034] The control device 300 may store information about the performance data communication device 100 (sometimes referred to as communication device information). The communication device information may include communication device identification information that can identify the performance data communication device 100. The communication device information may include network information that can identify the performance data processing device 200 on the network 80. The network information includes, for example, an IP address assigned to the performance data communication device 100. The communication device information may also include user identification information that can identify the user of the performance data communication device 100.

[0035] The control device 300 associates the performance data communication device 100 with the performance data processing device 200 using the processing device information stored in the control device 300, in order to have the performance data processing device 200 process the performance data 10. For example, the control device 300 may associate the performance data processing device 200 specified by the performance data communication device 100 with the performance data communication device 100. For example, the control device 300 transmits a processing device list to the performance data communication device 100, and associates the performance data processing device 200 selected by the performance data communication device 100 with the performance data communication device 100.

[0036] The control device 300 associates the performance data communication device 100 with the performance data processing device 200 by, for example, establishing a communication connection between the performance data communication device 100 and the performance data processing device 200. The control device 300 may establish a communication connection between the performance data communication device 100 and the performance data processing device 200 by transmitting an instruction to establish a communication connection to at least one of the performance data communication device 100 and the performance data processing device 200. The control device 300 associates the performance data communication device 100 with the performance data processing device 200 by managing the correspondence between the performance data communication device 100 and the performance data processing device 200. In this case, the control device 300 may relay communication between the performance data communication device 100 and the performance data processing device 200.

[0037] The transmitting section 110 of the performance data communication device 100, which has established a communication connection with the performance data processing device 200 by the control device 300, may transmit the performance data 10 to the performance data processing device 200 via the network 80. The receiving section 220 of the performance data processing device 200 receives the processed performance data 20. The performance data processing section 230 may process the performance data 10 received by the receiving section 220 to generate processed performance data 20. The transmitting section 210 may transmit the processed performance data 20 to the performance data communication device 100 via the network 80.

[0038] When the control device 300 relays communication between the performance data communication device 100 and the performance data processing device 200, the transmitting section 110 of the performance data communication device 100 may transmit the performance data 10 to the control device 300 via the network 80. The control device 300 may transmit the received performance data 10 to the performance data processing device 200 via the network 80. The receiving section 220 of the performance data processing device 200 may receive the performance data 10. The performance data processing section 230 may process the performance data 10 received by the receiving section 220 to generate processed performance data 20. The transmitting section 210 may transmit the processed performance data 20 to the control device 300 via the network 80. The control device 300 may transmit the processed performance data 20 to the performance data communication device 100 via the network 80.

[0039] The performance data communication device 100 may include a receiving unit 120, a converting unit 160, and a performance output unit 140. The receiving unit 120 may receive the processed performance data 20 transmitted by the transmitting unit 210. The converting unit 160 converts the processed performance data 20 into an analog signal. The converting unit 150 and the converting unit 160 may be integrated. The performance output unit 140 outputs the analog signal converted by the converting unit 160 as sound by a performance output device 600. The performance output device 600 is, for example, a speaker, headphones, or earphones. This allows the user 50 to listen to the processed performance data 20 that has been acoustically processed from their own performance, without having to prepare an acoustic processing device such as an effector in the environment where the musical instrument 500 is played.

[0040] The network 80 may include a mobile communication network. The mobile communication network may be compliant with 5G (5th Generation). The mobile communication network may be compliant with LTE (Long Term Evolution). The mobile communication network may be compliant with 6G (6th Generation) or later communication systems. The network 80 may include the Internet. The network 80 may include the cloud. The Internet may include a LAN (Local Area Network).

[0041] The performance data communication device 100 may be connected to the network 80 wirelessly. The performance data communication device 100 may be connected to the network 80 via a wireless base station. The performance data communication device 100 may be connected to the network 80 via a Wi-Fi (registered trademark) access point. The performance data communication device 100 may be connected to the network 80 by wire.

[0042] The performance data processing device 200 may be wirelessly connected to the network 80. The performance data processing device 200 may be connected to the network 80 via a wireless base station. The performance data processing device 200 may be connected to the network 80 via a Wi-Fi access point. The performance data processing device 200 may be wired to the network 80. The performance data processing device 200 may be disposed in, for example, a multi-access edge computing (MEC). If the mobile communication network of the network 80 has a hierarchical management system consisting of a management platform, a distribution platform at a lower level, and a wireless base station at a lower level, the performance data processing device 200 may be disposed in the distribution platform or the management platform. The management platform may be called a Core Brain, and the distribution platform may be called a Regional Brain. The distribution platform may have multiple levels. For example, if a two-tiered distributed infrastructure is placed under a management infrastructure, the management infrastructure may be called the Core Brain, the distributed infrastructure below that may be called the Regional Brain, and the distributed infrastructure below that may be called the Sub-Regional Brain.

[0043] The control device 300 may be wirelessly connected to the network 80. The control device 300 may be connected to the network 80 via a wireless base station. The control device 300 may be connected to the network 80 via a Wi-Fi access point. The control device 300 may be wired to the network 80. The control device 300 may be disposed in, for example, an MEC. When the mobile communication network of the network 80 has a hierarchical management system consisting of a management platform, a distributed platform in a lower layer, and a wireless base station in a lower layer, the control device 300 may be disposed in the distributed platform or the management platform.

[0044] As described above, the control device 300 may control the performance data processing device 200 to process the performance data 10 transmitted by the performance data communication device 100 and transmit the processed performance data 20 to the performance data communication device 100. The control device 300 may also control the performance data processing device 200 to process the performance data 10 transmitted by the performance data communication device 100 and transmit the processed performance data 20 to another device. For example, the control device 300 may receive a designation of a destination to which the processed performance data 20 is to be transmitted, and may control the processed performance data 20 to be transmitted to the designated destination via the network 80.

[0045] For example, the control device 300 establishes a communication connection between the performance data communication device 100 and the performance data processing device 200, and also establishes a communication connection between the performance data processing device 200 and a designated destination. The performance data communication device 100 transmits performance data 10 to the performance data processing device 200 with which the communication connection has been established, and the performance data processing device 200 processes the received performance data 10 to generate processed performance data 20, which it then transmits to the destination.

[0046] For example, the control device 300 receives performance data 10 from the performance data communication device 100, transmits it to the performance data processing device 200, and receives processed performance data 20 from the performance data processing device 200. Then, the control device 300 transmits the processed performance data 20 to a specified destination.

[0047] The control device 300 may adjust the processing of the performance data 10 by the performance data processing device 200 based on the communication status of the communication path from the performance data processing device 200 to the destination of the processed performance data 20. When the control device 300 relays communication between the performance data processing device 200 and the destination, the communication status between the performance data processing device 200 and the destination may be the communication status of the communication path from the performance data processing device 200 to the control device 300 and the communication status of the communication path from the control device 300 to the destination.

[0048] For example, when the performance data processing device 200 processes the performance data 10 transmitted by the performance data communication device 100 and transmits the processed performance data 20 to the performance data communication device 100, if the total time required for all processes from the sound produced by the musical instrument 500 to the sound output by the performance output device 600 exceeds a certain value, the performance may feel unnatural and the user 50 may feel stressed. Let us assume that the user 50 feels stressed if the total time exceeds 10 milliseconds.

[0049] As an example of this case, it is known from the communication status that the time required for transmitting the processed performance data 20 from the performance data processing device 200 to the performance data communication device 100 is 4 milliseconds. By calculating backward based on this, it is possible to reduce the total time to 10 milliseconds or less by limiting the time required for the processing of the performance data 10 by the performance data processing device 200 to 3 milliseconds or less. In this case, the control device 300 may adjust the processing of the performance data 10 by the performance data processing device 200 so that the time required for the processing is 3 milliseconds or less. For example, the control device 300 may reduce the time required for the processing by limiting the number of types of processing applied simultaneously. This reduces the discrepancy between the actual performance timing and the sound output timing due to communication delays, improving the user's musical experience.

[0050] The control device 300 may control the content of the processing so that the data volume of the processed performance data 20 is less than the data volume corresponding to the communication delay indicated by the communication status. For example, when the type of processing involves interpolation of sampling intervals, the control device 300 may control the content of the processing by adjusting the amount of the interpolation. For example, the control device 300 may reduce the data volume of the processed performance data 20 by reducing the amount of the interpolation. For example, when the type of processing involves changing the number of channels of the processed performance data 20, the control device 300 may control the content of the processing by adjusting the number of channels. For example, the control device 300 may reduce the data volume of the processed performance data 20 by reducing the number of channels. This reduces the discrepancy between the actual performance timing and the sound output timing due to communication delay, improving the user's musical experience.

[0051] 2 shows an example of the processing flow in the system 90. Here, the processing flow will be described in which the control device 300 associates a performance data processing device 200 specified by a performance data communication device 100 with the performance data communication device 100, among a plurality of performance data processing devices 200 that perform different processing of the performance data 10.

[0052] The control device 300 stores processing device information for the plurality of performance data processing devices 200. The control device 300 establishes a communication connection between the performance data communication device 100 and a performance data processing device 200 designated by the performance data communication device 100 among the plurality of performance data processing devices 200.

[0053] The performance data communication device 100, for example, specifies the performance data processing device 200 selected by the user 50 of the performance data communication device 100. As a specific example, when the user 50 plays the guitar, the user 50 selects, from among the multiple performance data processing devices 200, a performance data processing device 200 that can execute a desired effect.

[0054] The performance data communication device 100 transmits performance data 10 to the performance data processing device 200 via the network 80. The performance data processing device 200 processes the received performance data 10 to generate processed performance data 20, and transmits the processed performance data 20 to the performance data communication device 100 via the network 80. The performance data communication device 100 outputs the received processed performance data 20 as sound using the performance output device 600. The performance data communication device 100 and the performance data processing device 200 continuously execute these processes while the user 50 is playing the musical instrument 500.

[0055] When another performance data processing device 200 is designated by the performance data communication device 100 , the control device 300 establishes a communication connection between the other performance data processing device 200 and the performance data communication device 100 .

[0056] By having the system 90 execute the above-described processing, the user 50 can select the desired processing content as appropriate and obtain processed performance data 20 in which the desired processing has been applied to the performance data 10 that the user played.

[0057] 3 shows an example of a processing flow in the system 90. The system 90 may generate ensemble data 30 by processing and synthesizing performance data 10 of multiple users 50. Here, the processing flow will be described when the system 90 processes and synthesizes performance data 11 of a performance by user 51 and performance data 12 of a performance by user 52.

[0058] The control device 300 associates the performance data processing device 201 specified by the performance data communication device 101 with the performance data communication device 101, and associates the performance data processing device 202 specified by the performance data communication device 102 with the performance data communication device 102. The control device 300 receives, from the performance data processing device 201, processed performance data 21 obtained by processing the performance data 11 received by the performance data processing device 201 from the performance data communication device 101. The control device 300 receives, from the performance data processing device 202, processed performance data 22 obtained by processing the performance data 12 received by the performance data processing device 202 from the performance data communication device 102. The control device 300 generates ensemble data 30 by combining the received processed performance data 21 and processed performance data 22. The control device 300 transmits the ensemble data 30 to the performance data communication device 100 and the performance data processing device 200. This allows each user to obtain ensemble data in which the performance of each user has been processed as desired.

[0059] The control device 300 may perform control so that the performance data 11 and the performance data 12 are synthesized without processing, to generate the ensemble data 30. For example, the control device 300 receives the performance data 11 from the performance data communication device 101, and receives the performance data 12 from the performance data communication device 102. The control device 300 synthesizes the performance data 11 and the performance data 12 to generate the ensemble data 30, and transmits the ensemble data 30 to the performance data communication device 101 and the performance data communication device 102. Note that the control device 300 may also generate the ensemble data 30 by synthesizing only one of the performance data 11 and the performance data 12 after processing them.

[0060] 4 shows an example of a processing flow in the system 90. The system 90 may generate multiple types of ensemble data by processing and combining performance data 10 from multiple users 50. Here, the processing flow will be described for a case in which the system 90 processes performance data 11 from a performance by user 51, processes performance data 12 from a performance by user 52 in a different way, and combines the data to generate two types of ensemble data.

[0061] The control device 300 associates the performance data processing device 201 specified by the performance data communication device 101 with the performance data communication device 101, and associates the performance data processing device 202 specified by the performance data communication device 101 with the performance data communication device 102. The control device 300 associates the performance data processing device 201 specified by the performance data communication device 102 with the performance data communication device 101, and associates the performance data processing device 209 specified by the performance data communication device 102 with the performance data communication device 102. The control device 300 receives processed performance data 21 from the performance data processing device 201, which is obtained by processing the performance data 11 received by the performance data communication device 101. The control device 300 receives processed performance data 22 from the performance data processing device 202, which is obtained by processing the performance data 12 received by the performance data communication device 102 from the performance data communication device 102. The control device 300 receives processed performance data 29 from the performance data processing device 209, which is obtained by processing the performance data 12 received by the performance data processing device 209 from the performance data communication device 102. The control device 300 generates ensemble data 31 by combining the received processed performance data 21 and processed performance data 22. The control device 300 generates ensemble data 32 by combining the received processed performance data 21 and processed performance data 29. The control device 300 transmits the ensemble data 31 to the performance data communication device 101, and transmits the ensemble data 32 to the performance data communication device 102. This allows each of the different users participating in the ensemble to obtain ensemble data in which sound processing has been applied to the performance of each user according to their own preferences.

[0062] Here, an example has been described in which the control device 300 associates the performance data processing device 202 with the performance data communication device 102 as specified by the performance data communication device 101 in order to generate the ensemble data 31, but this is not limiting. The specification of which performance data processing device 200 and which performance data communication device 100 are to be associated may be made by any of the performance data communication devices 100. For example, in order to generate the ensemble data 31, the control device 300 may associate the performance data processing device 202 with the performance data communication device 102 as specified by the performance data communication device 102 instead of the performance data communication device 101.

[0063] FIG. 5 shows an example of the processing flow in the system 90. The system 90 may be available to a user 59 who does not play a musical instrument 500. The user 59 may be, for example, a recording engineer, a PA engineer, or a listener. The user 59 may instruct the control device 300 from the performance data communication device 109 how to process the performance data 10 of a user 50 (performer) using a performance data processing device 200 and synthesize the generated processed performance data 20. Note that, although the description here takes the case where the user 59 uses the performance data communication device 109 as an example, the present invention is not limited to this. The user 59 may use any device that is capable of communication and audibly outputting performance data and ensemble data.

[0064] The control device 300 associates a plurality of performance data communication devices 100 with a plurality of performance data processing devices 200 based on an instruction from the user 59. In the example shown in Fig. 5, the control device 300 associates the performance data communication device 101 with the performance data processing device 201, and associates the performance data communication device 102 with the performance data processing device 202. The control device 300 receives processed performance data 21 from the performance data processing device 201, which is obtained by processing the performance data 11 received from the performance data communication device 101 by the performance data processing device 201. The control device 300 receives processed performance data 22 from the performance data processing device 202, which is obtained by processing the performance data 12 received from the performance data communication device 102 by the performance data processing device 202. The control device 300 synthesizes the received processed performance data 21 and processed performance data 22 to generate ensemble data 31. The control device 300 transmits the ensemble data 31 to the performance data communication device 101, the performance data communication device 102, and the performance data communication device 109. In this way, by generating ensemble data by associating the performance data communication device 100 with the performance data processing device 200 in accordance with instructions from recording engineers, PA engineers, and others other than the performers, the recording engineers, PA engineers, and others can utilize ensemble sound sources to create ensemble works that incorporate ingenious sound processing and synthesis techniques.

[0065] Fig. 6 shows an example of a processing flow in the system 90. The system 90 may generate multiple types of ensemble data 30 from multiple types of performance data 10. Fig. 6 illustrates a processing flow for generating multiple types of ensemble data 30 from performance data 11 of user 51 and performance data 12 of user 52 in accordance with instructions from users 51 and 52 who are performers and a user 59 who does not play the musical instrument 500.

[0066] 6, the control device 300 controls the performance data communication device 101 to transmit ensemble data 31, which is a combination of processed performance data 21 obtained by processing the performance data 11 by the performance data processing device 201 and processed performance data 22 obtained by processing the performance data 12 by the performance data processing device 202. The control device 300 controls the performance data communication device 102 to transmit ensemble data 32, which is a combination of processed performance data 21 obtained by processing the performance data 11 by the performance data processing device 201 and processed performance data 29 obtained by processing the performance data 12 by the performance data processing device 209. The control device 300 controls the performance data communication device 109 to transmit ensemble data 33, which is a combination of processed performance data 21 obtained by processing the performance data 11 by the performance data processing device 201 and processed performance data 22 obtained by processing the performance data 12 by the performance data processing device 202.

[0067] 6, the control device 300 generates the ensemble data 33 using a synthesis method different from that used to generate the ensemble data 31. The control device 300 may generate the ensemble data 31 by synthesizing the processed performance data 21 and the processed performance data 22 using a synthesis method instructed by the performance data communication device 101, and may generate the ensemble data 33 by synthesizing the processed performance data 21 and the processed performance data 22 using a synthesis method instructed by the performance data communication device 109.

[0068] For example, the control device 300 generates ensemble data 31 by synthesizing the processed performance data 21 and the processed performance data 22 using a synthesis method instructed by the user 51 from the performance data communication device 101, which emphasizes the performance of the user 51. The control device 300 generates ensemble data 33 by synthesizing the processed performance data 21 and the processed performance data 22 using a synthesis method instructed by the user 59 from the performance data communication device 109, which optimizes the overall harmony of the volume balance between the processed performance data 21 and the processed performance data 22. The synthesis method instructed by the user 59 from the performance data communication device 109 may be a synthesis method that adjusts the volume balance for each range of the played instruments in the processed performance data 21 and the processed performance data 22. The synthesis method may also be a synthesis method that synthesizes the processed performance data 21 and the processed performance data 22 in accordance with the specifications of a specific performance output device 600, so as to achieve effective sound effects when output by the performance output device 600.

[0069] In the above example, the user 59 processes the performance data 11 by specifying the same performance data processing device 201 as the performance data processing device 201 specified by the user 51, but the user 59 may process the performance data 11 by specifying a performance data processing device 200 different from the performance data processing device 201 specified by the user 51, depending on the type of processing the user 59 desires. The user 59 processes the performance data 12 by specifying the same performance data processing device 202 as the performance data processing device 202 specified by the user 52, but the user 59 may process the performance data 12 by specifying a performance data processing device 200 different from the performance data processing device 202 specified by the user 52, depending on the type of processing the user 59 desires.

[0070] In the above example, the control device 300 synthesizes the processed performance data 21 and the processed performance data 22 to generate the ensemble data 33 in response to an instruction from the performance data communication device 109 by the user 59. However, the method of synthesizing the ensemble data 33 is not limited to this. For example, the control device 300 associates the performance data processing device 201 instructed by the user 59 from the performance data communication device 109 with the performance data communication device 109, and controls the processed performance data 21 to be transmitted to the performance data communication device 109, and associates the performance data processing device 202 instructed by the user 59 from the performance data communication device 109 with the performance data communication device 109, and controls the processed performance data 22 to be transmitted to the performance data communication device 109.

[0071] The control device 300 associates the performance data processing device 201 with the performance data communication device 109 by, for example, establishing a communication connection between the performance data processing device 201 and the performance data communication device 109. The control device 300 may establish a communication connection between the performance data processing device 201 and the performance data communication device 109 by transmitting an instruction to establish a communication connection to at least one of the performance data processing device 201 and the performance data communication device 109. The control device 300 may associate the performance data processing device 201 with the performance data communication device 109 by managing the correspondence between the performance data processing device 201 and the performance data communication device 109. In this case, the control device 300 may relay communication between the performance data processing device 201 and the performance data communication device 109.

[0072] The performance data processing device 201, which has established a communication connection with the performance data communication device 109 by the control device 300, may transmit the processed performance data 21 to the performance data communication device 109 via the network 80. The performance data communication device 109 receives the processed performance data 21.

[0073] When the control device 300 relays communication between the performance data processing device 201 and the performance data communication device 109, the performance data processing device 201 may transmit the processed performance data 21 to the control device 300 via the network 80. The control device 300 may transmit the received processed performance data 21 to the performance data communication device 109 via the network 80. The performance data communication device 109 receives the processed performance data 21.

[0074] In the above explanation, the correspondence between the performance data processing device 201 and the performance data communication device 109 by the control device 300 has been explained, but the correspondence between the performance data processing device 202 and the performance data communication device 109 by the control device 300 is similar.

[0075] In this way, the performance data communication device 109 receives the processed performance data 21 and the processed performance data 22. The performance data communication device 109 may synthesize the processed performance data 21 and the processed performance data 22 received by the performance data communication device 109 to generate the ensemble data 33. The performance data communication device 109 may synthesize the processed performance data 21 and the processed performance data 22 received by the performance data communication device 109 by a mixer connected to the performance data communication device 109 to generate the ensemble data 33.

[0076] 7 is a schematic diagram illustrating an example of a processing flow in the system 90. In the system 90, the performance data processing device 200, rather than the control device 300, may generate the ensemble data 30. In the example illustrated in FIG. 7, the control device 300 associates a plurality of performance data communication devices (performance data communication device 101, performance data communication device 102, ..., performance data communication device 109) with the performance data processing device 200, and causes the performance data processing device 200 to process the performance data (performance data 11, performance data 12, ..., performance data 19) received from each of the plurality of performance data communication devices 100 and synthesize the plurality of processed performance data (processed performance data 21, processed performance data 22, ..., processed performance data 29) to generate the ensemble data 30. The control device 300 may perform control so that the ensemble data 30 is transmitted to a plurality of performance data communication devices (performance data communication device 101, performance data communication device 102, . . . , performance data communication device 109).

[0077] Fig. 8 shows an example of a processing flow in the system 90. In the example shown in Fig. 8, the control device 300 associates a plurality of performance data communication devices (performance data communication device 101, performance data communication device 102, ..., performance data communication device 109) with the performance data processing device 200, and causes the performance data processing device 200 to process the performance data (performance data 11, performance data 12, ..., performance data 19) received from each of the plurality of performance data communication devices 100 and synthesize the plurality of processed performance data (processed performance data 21, processed performance data 22, ..., processed performance data 29) to generate a plurality of different ensemble data (ensemble data 31, ensemble data 32, ..., ensemble data 39). The control device 300 may control the transmission of ensemble data to each of the multiple performance data communication devices (performance data communication device 101, performance data communication device 102, ..., performance data communication device 109) in accordance with the specifications of each of the multiple performance data communication devices (performance data communication device 101, performance data communication device 102, ..., performance data communication device 109).

[0078] 9 is a schematic diagram of an example of a system 90. In the example shown in FIG. 9, the system 90 includes a performance data generating device 400 and an ensemble data generating section 92.

[0079] The performance data generating device 400 has a pseudo-generation unit that generates pseudo-performance data of a specified user (performer) playing a specified piece of music. The ensemble data generating unit 92 generates ensemble data 30 by combining processed performance data 20, which is obtained by a performance data processing device 200 associated with the performance data communication device 100 by the control device 300 processing the performance data 10 received from the performance data communication device 100, with the pseudo-performance data 40 that is pseudo-generated by the performance data generating device 400.

[0080] In the example shown in Figure 9, the ensemble data generation unit 92 is arranged independently of the performance data processing device 200 and the control device 300, but the performance data processing device 200 may have the ensemble data generation unit 92, or the control device 300 may have the ensemble data generation unit 92.

[0081] For example, in accordance with an instruction from the performance data communication device 100, the control device 300 associates the performance data communication device 100 with the performance data processing device 200 and specifies a performer and a piece of music to the performance data generation device 400. The performance data communication device 100 transmits performance data 10 of a piece of music performed by the user 50 to the performance data processing device 200. The performance data processing device 200 processes the performance data 10 to generate processed performance data 20 and transmits the processed performance data 20 to the ensemble data generation unit 92. The pseudo-generation unit of the performance data generation device 400 generates pseudo-performance data of a specified performer playing the specified piece of music and transmits the pseudo-performance data 20 to the ensemble data generation unit 92. The ensemble data generation unit 92 combines the received processed performance data 20 and the pseudo-performance data 40 to generate ensemble data 30 and transmits the ensemble data 30 to the performance data communication device 100. This allows the user 50 to add desired processing to his or her own performance and have a simulated experience of playing in an ensemble with a famous performer, etc.

[0082] Alternatively, ensemble data may be generated by combining the performance data 10 and the pseudo performance data 40 without processing the performance data 10. For example, the control device 300 specifies performers and music pieces to the performance data generating device 400 in accordance with an instruction from the performance data communication device 100. The performance data communication device 100 transmits performance data 10 of the music piece performed by the user 50 to the ensemble data generating unit 92. The pseudo generation unit of the performance data generating device 400 generates pseudo performance data 40 of the specified performers playing the specified music piece, and transmits it to the ensemble data generating unit 92. The ensemble data generating unit 92 combines the received performance data 10 and the pseudo performance data 40 to generate ensemble data 30, and transmits it to the performance data communication device 100.

[0083] The system 90 may generate ensemble data combining performances by multiple users 50 and a pseudo-generated performance. For example, if one of the multiple users 50 who are members of an ensemble is absent, the performance data generating device 400 generates pseudo-performance data for the absent user 50, thereby realizing a pseudo-ensemble. In this case, for example, the control device 300, in accordance with an instruction from at least one of the multiple users 50, associates the performance data communication device 100 with the performance data processing device 200 for each of the multiple users 50 who are present, and specifies the absent user 50 and the music piece to be played in the ensemble to the performance data generating device 400. The multiple performance data processing devices 200 process the performance data 10 received from the multiple performance data communication devices 100 to generate processed performance data 20, which are then transmitted to the ensemble data generating unit 92. The pseudo-generation unit of the performance data generation device 400 generates pseudo-performance data 40 of a specified user 50 playing a specified piece of music, and transmits the pseudo-performance data 40 to the ensemble data generation unit 92. The ensemble data generation unit 92 synthesizes the multiple pieces of processed performance data 20 and the pseudo-performance data 40 to generate ensemble data 30, which is then transmitted to multiple performance data communication devices 100. This allows for the provision of an environment in which ensemble practice can be carried out, even if one of the ensemble members is absent from the ensemble, by realizing a pseudo-ensemble.

[0084] 10 shows an example of the functional configuration of the control device 300. The control device 300 includes a storage unit 310, a receiving unit 320, a transmitting unit 330, a matching unit 340, a destination designation receiving unit 350, an ensemble data generating unit 360, a communication status acquiring unit 370, and a processing adjusting unit 380. Note that the control device 300 does not necessarily have to include all of these units. In addition to these "units," the control device 300 may further include any "unit" for realizing the functions of the control device 300.

[0085] The storage unit 310 stores various types of information. The storage unit 310 stores processing device information. The storage unit 310 may store multiple pieces of processing device information. The storage unit 310 stores communication device information. The storage unit 310 may store multiple pieces of communication device information.

[0086] The receiving unit 320 and the transmitting unit 330 communicate with various devices via the network 80. For example, the receiving unit 320 and the transmitting unit 330 communicate with the performance data communication device 100 via the network 80. For example, the receiving unit 320 and the transmitting unit 330 communicate with the performance data processing device 200 via the network 80. For example, the receiving unit 320 and the transmitting unit 330 communicate with the performance data generation device 400 via the network 80. For example, the receiving unit 320 and the transmitting unit 330 communicate with the ensemble data generation unit 92 via the network 80.

[0087] The association unit 340 associates the performance data communication device 100 with the performance data processing device 200 using the information stored in the storage unit 310. The association unit 340 associates the performance data communication device 100 with the performance data processing device 200 using the communication device information and the processing device information so that the performance data 10 transmitted by the performance data communication device 100 is processed by the performance data processing device 200.

[0088] The associating section 340 may associate the performance data communication device 100 with the performance data processing device 200 in accordance with an external specification. For example, the associating section 340 may associate the performance data communication device 100 with the performance data processing device 200 in accordance with a specification from the performance data communication device 100. The associating section 340 may associate the performance data communication device 100 with the performance data processing device 200 specified by the performance data communication device 100, among the multiple performance data processing devices 200.

[0089] The correspondence unit 340 may establish a communication connection between the performance data communication device 100 and the performance data processing device 200. The correspondence unit 340 may manage the correspondence between the performance data communication device 100 and the performance data processing device 200. The correspondence unit 340 may control the receiving unit 320 and the transmitting unit 330 to relay communication between the performance data communication device 100 and the performance data processing device 200 that are in a corresponding relationship.

[0090] The destination designation receiving unit 350 receives designation of a destination to which the processed performance data 20 is to be transmitted. For example, from a performance data communication device 100, the destination designation receiving unit 350 receives designation of a destination to which the processed performance data 20, which is obtained by processing the performance data 10 transmitted by the performance data communication device 100, is to be transmitted. For example, from a performance data communication device 100, the destination designation receiving unit 350 receives designation of a destination to which the processed performance data 20, which is obtained by processing the performance data 10 transmitted by a performance data communication device 100 other than the performance data communication device 100, is to be transmitted.

[0091] The association unit 340 controls the transmission of the processed performance data 20 to a destination designated by the destination designation receiving unit 350. When a communication connection has been established between the performance data processing device 200 and the destination, the association unit 340 may cause the performance data processing device 200 to transmit the processed performance data 20 to the destination via the communication connection. When a communication connection has not been established between the performance data processing device 200 and the destination, the association unit 340 may establish a communication connection and cause the performance data processing device 200 to transmit the processed performance data 20 to the destination via the communication connection. When the association unit 340 manages the correspondence between the performance data processing device 200 and the destination, the association unit 340 may receive the processed performance data 20 from the performance data processing device 200 and transmit it to the destination. When the correspondence between the performance data processing device 200 and the transmission destination is not being managed, the correspondence unit 340 may newly manage the correspondence between the performance data processing device 200 and the transmission destination, and control the reception of processed performance data 20 from the performance data processing device 200 and the transmission of the processed performance data 20 to the transmission destination.

[0092] The association unit 340 may cause the performance data processing device 200 to generate ensemble data. For example, the association unit 340 may associate a plurality of performance data communication devices 100 with the performance data processing device 200, and cause the performance data processing device 200 to process a plurality of pieces of performance data 10 received from the plurality of performance data communication devices 100 to generate a plurality of pieces of processed performance data 20, and to generate ensemble data by synthesizing the plurality of processed performance data 20. The association unit 340 may also cause the performance data processing device 200 to generate ensemble data by synthesizing the plurality of pieces of performance data 10 received from the plurality of performance data communication devices 100. The association unit 340 may also cause the performance data processing device 200 to generate ensemble data by synthesizing a portion of the plurality of pieces of performance data 10 received from the plurality of performance data communication devices 100 with processed performance data 20 obtained by processing the remaining portion.

[0093] The ensemble data generating unit 360 generates the ensemble data 30. The ensemble data generating unit 360 may generate the ensemble data 30 by combining multiple pieces of processed performance data 20. For example, the receiving unit 320 receives multiple pieces of processed performance data 20 obtained by processing multiple pieces of performance data 10 transmitted by multiple performance data communication devices 100 using multiple performance data processing devices 200, and the ensemble data generating unit 360 generates the ensemble data 30 by combining the multiple pieces of processed performance data 20 received by the receiving unit 320. As a specific example, the correspondence assignment unit 340 associates the first performance data communication device 100 with the first performance data processing device 200, and associates the second performance data communication device 100 with the second performance data processing device 200, the receiving unit 320 receives from the first performance data processing device 200 first processed performance data 20 that is obtained by processing the performance data 10 that the first performance data processing device 200 received from the first performance data communication device 100, and the second performance data processing device 200 receives from the second performance data processing device 200 second processed performance data 20 that is obtained by processing the performance data 10 that the second performance data processing device 200 received from the second performance data communication device 100, and the ensemble data generation unit 360 generates ensemble data by combining the first processed performance data 20 and the second processed performance data 20 received by the receiving unit 320.

[0094] The ensemble data generating unit 360 may generate ensemble data by combining multiple pieces of performance data 10. For example, the receiving unit 320 receives multiple pieces of performance data 10 transmitted by multiple performance data communication devices 100, and the ensemble data generating unit 360 combines the pieces of performance data 10 received by the receiving unit 320 to generate ensemble data 30.

[0095] The ensemble data generating unit 360 may generate the ensemble data 30 by combining one or more pieces of performance data 10 and one or more pieces of processed performance data 20.

[0096] The ensemble data generating unit 360 may generate ensemble data 30 by combining one or more pieces of processed performance data 20 with pseudo performance data 40 generated by a pseudo generation unit of the performance data generating device 400. For example, the receiving unit 320 receives one or more pieces of processed performance data 20 obtained by processing one or more pieces of performance data 10 transmitted by one or more performance data communication devices 100 by one or more performance data processing devices 200, and the pseudo performance data 40, and the ensemble data generating unit 360 generates ensemble data 30 by combining the one or more pieces of processed performance data 20 and the pseudo performance data 40 received by the receiving unit 320. The ensemble data generating unit 360 may generate ensemble data 30 by combining one or more pieces of processed performance data 20 with the multiple pieces of pseudo performance data 40.

[0097] The ensemble data generating unit 360 may generate ensemble data 30 by combining one or more pieces of performance data 10 with pseudo performance data 40 generated by a pseudo generation unit of the performance data generating device 400. For example, the receiving unit 320 receives one or more pieces of performance data 10 and the pseudo performance data 40 transmitted by one or more performance data communication devices 100, and the ensemble data generating unit 360 generates ensemble data 30 by combining the one or more pieces of performance data 10 and the pseudo performance data 40 received by the receiving unit 320. The ensemble data generating unit 360 may generate ensemble data 30 by combining one or more pieces of performance data 10 and multiple pieces of pseudo performance data 40.

[0098] The ensemble data generating unit 360 may generate the ensemble data 30 by combining one or more pieces of performance data 10, one or more pieces of processed performance data 20, and multiple pieces of pseudo performance data 40. The ensemble data generating unit 360 may generate the ensemble data 30 by combining one or more pieces of performance data 10, one or more pieces of processed performance data 20, and multiple pieces of pseudo performance data 40.

[0099] The communication status acquisition unit 370 acquires the communication status of the communication path from the performance data processing device 200 to the destination of the processed performance data 20. Examples of the communication status include, but are not limited to, communication speed, communication delay, communication traffic, and communication capacity.

[0100] When a communication connection is established between the performance data processing device 200 and the destination, the communication status acquiring section 370 may acquire the communication status of the communication path between the performance data processing device 200 and the destination. The communication status acquiring section 370 may acquire the communication status of the communication path between the performance data processing device 200 and the destination from at least one of the performance data processing device 200 and the destination.

[0101] When the control device 300 relays communication between the performance data processing device 200 and the destination, the communication status acquisition unit 370 may acquire the communication status of the communication path between the performance data processing device 200 and the control device 300, and the communication status of the communication path between the control device 300 and the destination.

[0102] The processing adjustment unit 380 may adjust the processing of the performance data 10 by the performance data processing device 200 based on the communication status acquired by the communication status acquisition unit 370. For example, consider a case where the performance data 10 produced by the user 50 playing the musical instrument 500 is transmitted from the performance data communication device 100, processed by the performance data processing device 200, and synthesized by the performance data processing device 200, the control device 300, or the ensemble data generation unit 92 to generate ensemble data 30, which is received by the performance data communication device 100 and output as sound from the performance output device 600. In this case, the processing adjustment unit 380 causes the performance data processing device 200 to adjust the processing of the performance data 10 so that the total time (sometimes referred to as total processing time) required for all processes from when the user 50 sounds the musical instrument 500 to when the performance output device 600 outputs as sound the ensemble data 30 derived from the sounds does not exceed a predetermined threshold corresponding to the time at which the user 50 begins to feel stressed. For example, the processing adjustment unit 380 causes the performance data processing device 200 to adjust the processing of the performance data 10 so as to reduce the time required for the processing step of the performance data 10 by the performance data processing device 200 out of the total processing time. For example, the processing adjustment unit 380 limits the number of types of processing that the performance data processing device 200 can perform simultaneously.

[0103] The processing adjustment unit 380 may control the content of processing so that the data volume of the processed performance data 20 is less than the data volume corresponding to the communication delay indicated by the communication status. For example, if the processing of the performance data 10 by the performance data processing device 200 involves interpolation of the sampling interval of the performance data 10, the processing adjustment unit 380 causes the performance data processing device 200 to adjust the amount of interpolation. For example, the processing adjustment unit 380 causes the performance data processing device 200 to reduce the amount of interpolation. For example, if the processing of the performance data 10 by the performance data processing device 200 involves changing the number of channels of the performance data 10, the processing adjustment unit 380 causes the performance data processing device 200 to adjust the number of channels. For example, the processing adjustment unit 380 causes the performance data processing device 200 to reduce the number of channels.

[0104] 11 shows an example of the functional configuration of the performance data generation device 400. The performance data generation device 400 includes a learning model generation unit 410, a learning model storage unit 420, a musical score data receiving unit 430, and a pseudo generation unit 440. Note that the performance data generation device 400 does not necessarily have to include all of these units. In addition to these "units," the performance data generation device 400 may further include any "unit" for realizing the functions of the performance data generation device 400.

[0105] The learning model generation unit 410 generates a learning model. The learning model generation unit 410 may generate a learning model that receives musical score data as input and performance data as output. For example, the learning model generation unit 410 generates the learning model through machine learning using performance data generated by a user playing a piece of music on an instrument and training data including musical score data for the piece of music. The training data may include a large amount of performance data and musical score data generated by a user playing various pieces of music. The learning model can simulate performance data of a piece of music based on input musical score data played by the user. The learning model generation unit 410 may generate a learning model for each of multiple users.

[0106] The learning model generation unit 410 may perform machine learning using ensemble data generated by multiple users playing together and training data including performance data by some of the users in the ensemble to generate a learning model that takes performance data by the some of the users as input and outputs ensemble data by the multiple users. For example, the learning model generation unit 410 may generate a learning model that takes performance data by a first user, generated by machine learning using ensemble data generated by a first user and a second user playing together and training data including the performance data by the first user in the ensemble, as input and outputs ensemble data by the first user and the second user. The training data may include a large amount of ensemble data generated by the first user and the second user playing various pieces of music together and the performance data of the first user. According to the learning model, even when the second user is absent, performance data of the second user can be simulated to generate ensemble data with the first user.

[0107] The learning model generation unit 410 may perform machine learning using ensemble data generated by multiple users playing together and training data including performance data by some of the users in the ensemble, thereby generating a learning model that takes performance data by some of the users as input and performance data by the remaining users as output. For example, the learning model generation unit 410 may generate a learning model that takes performance data by a first user as input and performance data by a second user as output, the learning model being generated by machine learning using ensemble data generated by a first user and a second user playing together and training data including the performance data by the first user in the ensemble. The training data may include a large amount of ensemble data generated by the first user and the second user playing various pieces of music together and the performance data of the first user. Using this learning model, it is possible to generate simulated performance data 40 of the second user playing together with the first user, even when the second user is absent.

[0108] The learning model storage unit 420 stores learning models. The learning model storage unit 420 may store a learning model that takes score data as input and performance data as output. The learning model storage unit 420 may store multiple learning models, each corresponding to a plurality of users, that take score data as input and output performance data of the user playing the music piece of the score data. The learning model storage unit 420 may store a learning model that takes score data as input and output performance data of a famous performer playing the music piece of the score data. The learning model storage unit 420 may store a learning model that takes performance data by some of the plurality of users as input and output ensemble data of the plurality of users. The learning model storage unit 420 may store a learning model that takes performance data by some of the plurality of users as input and output performance data of the remaining users of the plurality of users. The learning model storage unit 420 may store a learning model that takes performance data by some of the plurality of users as input and output performance data of the remaining users of the plurality of users. The learning model storage unit 420 may store learning models generated by the learning model generation unit 410. The learning model storage unit 420 may store learning models generated by other devices.

[0109] The score data receiving unit 430 receives score data of a musical piece to be simulated from an external source. For example, the score data receiving unit 430 receives the score data from the performance data communication device 100. For example, the score data receiving unit 430 receives the score data from the control device 300. For example, the score data receiving unit 430 receives the score data from the performance data processing device 200.

[0110] The pseudo generation unit 440 generates pseudo performance data. The pseudo generation unit 440 may generate pseudo performance data of a specified user playing a specified musical piece. The pseudo generation unit 440 may accept a user and a musical piece designated by the performance data communication device 100. The pseudo generation unit 440 may accept a user and a musical piece designated by the control device 300. The pseudo generation unit 440 may accept a user and a musical piece designated by the performance data processing device 200. The pseudo generation unit 440 may use the score data of the specified musical piece among the score data received by the score data receiving unit 430. The pseudo generation unit 440 may input the score data of the specified musical piece to a learning model corresponding to the specified user, which is stored in the learning model storage unit 420, and obtain the performance data output from the learning model as the pseudo performance data 40.

[0111] The pseudo-generation unit 440 may generate pseudo-performance data for the remaining users from the performance data of some of the users. The pseudo-generation unit 440 may acquire the performance data of some of the users from their performance data communication devices 100, input the data to corresponding learning models stored in the learning model storage unit 420, and acquire the performance data output from the learning models as the pseudo-performance data 40 for the remaining users. For example, the pseudo-generation unit 440 may input the performance data of a first user out of a first user and a second user to learning models corresponding to the first user and the second user, and acquire the performance data output from the learning models as the pseudo-performance data 40 for the second user.

[0112] The pseudo generation unit 440 may generate pseudo ensemble data. For example, the pseudo generation unit 440 may generate pseudo ensemble data for multiple users from performance data of some of the multiple users. The pseudo generation unit 440 may acquire performance data of some of the multiple users from their performance data communication devices 100, input the data to corresponding learning models stored in the learning model storage unit 420, and acquire the ensemble data output from the learning models as pseudo ensemble data for the multiple users. For example, the pseudo generation unit 440 may input performance data of a first user out of a first user and a second user to learning models corresponding to the first user and the second user, and acquire the ensemble data output from the learning models as pseudo ensemble data for the first user and the second user.

[0113] 12 schematically illustrates an example of the hardware configuration of a computer 1200 that functions as the performance data communication device 100, the performance data processing device 200, the control device 300, the performance data generation device 400, or the ensemble data generation unit 92. A program installed on the computer 1200 can cause the computer 1200 to function as one or more "units" of the device according to the present embodiment, or to execute operations associated with the device according to the present embodiment or one or more "units," and / or to execute a process according to the present embodiment or steps of the process. Such a program may be executed by the CPU 1212 to cause the computer 1200 to perform specific operations associated with some or all of the blocks in the flowcharts and block diagrams described herein.

[0114] The computer 1200 according to this embodiment includes a CPU 1212, a GPU 1213, a RAM 1214, and a graphics controller 1216, which are interconnected by a host controller 1210. The computer 1200 also includes input / output units such as a communications interface 1222, a storage device 1224, a DVD drive, and an IC card drive, which are connected to the host controller 1210 via an input / output controller 1242. The DVD drive may be a DVD-ROM drive, a DVD-RAM drive, or the like. The storage device 1224 may be a hard disk drive, a solid-state drive, or the like. The computer 1200 also includes a ROM 1230 and legacy input / output units such as a keyboard, which are connected to the input / output controller 1242 via an input / output chip 1240.

[0115] The CPU 1212 operates according to programs stored in the ROM 1230 and the RAM 1214, thereby controlling each unit. The graphics controller 1216 acquires image data generated by the CPU 1212 into a frame buffer or the like provided in the RAM 1214 or into the graphics controller itself, and causes the image data to be displayed on the display device 1218.

[0116] The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 in the computer 1200. The DVD drive reads programs or data from a DVD-ROM or the like and provides them to the storage device 1224. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0117] The ROM 1230 stores therein a boot program or the like that is executed by the computer 1200 upon activation, and / or programs that depend on the hardware of the computer 1200. The input / output chip 1240 may also connect various input / output units to the input / output controller 1242 via a USB port, a parallel port, a serial port, a keyboard port, a mouse port, etc.

[0118] The programs are provided by a computer-readable storage medium such as a DVD-ROM or an IC card. The programs are read from the computer-readable storage medium, installed in the storage device 1224, RAM 1214, or ROM 1230, which are also examples of computer-readable storage media, and executed by the CPU 1212. Information processing described in these programs is read by the computer 1200, and causes cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be configured by implementing operations or processing of information in accordance with the use of the computer 1200.

[0119] For example, when communication is performed between the computer 1200 and an external device, the CPU 1212 may execute a communication program loaded into the RAM 1214 and instruct the communication interface 1222 to perform communication processing based on the processing described in the communication program. Under the control of the CPU 1212, the communication interface 1222 reads transmission data stored in a transmission buffer area provided in the RAM 1214, the storage device 1224, a DVD-ROM, or a recording medium such as an IC card, and transmits the read transmission data to the network, or writes reception data received from the network to a reception buffer area or the like provided on the recording medium.

[0120] Furthermore, the CPU 1212 may cause all or a necessary portion of a file or database stored in an external recording medium such as the storage device 1224, a DVD drive (DVD-ROM), an IC card, etc. to be read into the RAM 1214, and may perform various types of processing on the data on the RAM 1214. The CPU 1212 may then write back the processed data to the external recording medium.

[0121] Various types of information, such as various types of programs, data, tables, and databases, may be stored on the recording medium and may undergo information processing. The CPU 1212 may perform various types of processing on data read from the RAM 1214, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search / replacement, etc., as described throughout this disclosure and specified by the instruction sequences of the programs, and write the results back to the RAM 1214. The CPU 1212 may also search for information in a file, database, etc. on the recording medium. For example, if multiple entries, each having an attribute value of a first attribute associated with an attribute value of a second attribute, are stored on the recording medium, the CPU 1212 may search for an entry whose attribute value of the first attribute matches a specified condition from among the multiple entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0122] The above-described programs or software modules may be stored in a computer-readable storage medium on or near the computer 1200. A recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can also be used as a computer-readable storage medium, thereby providing the programs to the computer 1200 via the network.

[0123] The blocks in the flowcharts and block diagrams in the present embodiments may represent stages of a process in which an operation is performed or "parts" of an apparatus responsible for performing the operation. Particular stages and "parts" may be implemented by dedicated circuitry, programmable circuitry provided with computer-readable instructions stored on a computer-readable storage medium, and / or a processor provided with computer-readable instructions stored on a computer-readable storage medium. The dedicated circuitry may include digital and / or analog hardware circuits, including integrated circuits (ICs) and / or discrete circuits. The programmable circuitry may include reconfigurable hardware circuits, such as field programmable gate arrays (FPGAs) and programmable logic arrays (PLAs), including AND, OR, XOR, NAND, NOR, and other logical operations, flip-flops, registers, and memory elements.

[0124] A computer-readable storage medium may include any tangible device capable of storing instructions that are executed by an appropriate device, such that a computer-readable storage medium having instructions stored thereon comprises an article of manufacture, including instructions that can be executed to create means for performing the operations specified in the flowcharts or block diagrams. Examples of computer-readable storage media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, etc. More specific examples of computer-readable storage media may include floppy disks, diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), electrically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disc (DVD), Blu-Ray disc, memory stick, integrated circuit card, etc.

[0125] The computer readable instructions may include either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, JAVA®, C++, etc., and conventional procedural programming languages ​​such as the “C” programming language or similar programming languages.

[0126] Computer-readable instructions may be provided to a general-purpose computer, a special-purpose computer, or another programmable data processing device, or a programmable circuit, either locally or via a local area network (LAN), a wide area network (WAN) such as the Internet, so that the processor of the programmable data processing device, such as a computer, or the programmable circuit executes the computer-readable instructions to generate means for performing the operations specified in the flowcharts or block diagrams. Here, the computer may be a personal computer (PC), a tablet computer, a smartphone, a workstation, a server computer, a general-purpose computer, a special-purpose computer, or the like, or may be a computer system in which multiple computers are connected. Such a computer system in which multiple computers are connected is also called a distributed computing system, and is a broad definition of computers. In a distributed computing system, multiple computers collectively execute a program by each executing a portion of the program and passing data between the computers as needed during program execution.

[0127] Examples of processors include a computer processor, a central processing unit (CPU), a processing unit, a microprocessor, a digital signal processor, a controller, a microcontroller, etc. A computer may have one or more processors. In a multiprocessor system with multiple processors, each processor executes a portion of a program and passes data between processors as needed during program execution, allowing the multiple processors to collectively execute a program. For example, in multitasking, each of the multiple processors may execute a portion of each task in small chunks by switching tasks at time slice intervals. In this case, which portion of a program each processor executes changes dynamically. Which portion of a program each of the multiple processors executes may also be statically determined by multiprocessor-aware programming.

[0128] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the claims that such modifications and improvements can also be included within the technical scope of the present invention.

[0129] It should be noted that the execution order of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before," "prior to," etc., and that the processes can be performed in any order unless the output of a previous process is used in a later process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," etc. for convenience, this does not mean that the processes must be performed in this order. [Explanation of symbols]

[0130] 10 performance data, 11 performance data, 12 performance data, 19 performance data, 20 processed performance data, 21 processed performance data, 22 processed performance data, 29 processed performance data, 30 ensemble data, 31 ensemble data, 32 ensemble data, 33 ensemble data, 39 ensemble data, 40 pseudo performance data, 50 user, 51 user, 52 user, 59 user, 80 network, 90 system, 92 ensemble data generation unit, 100 performance data communication device, 101 performance data communication device, 102 performance data communication device, 109 performance data communication device, 110 transmission unit, 120 reception unit, 130 performance input unit, 140 performance output unit, 150 conversion unit, 160 conversion unit, 200 performance data processing device, 201 performance data processing device, 202 Performance data processing device, 209 performance data processing device, 210 transmission unit, 220 reception unit, 230 performance data processing unit, 300 control device, 310 storage unit, 320 reception unit, 330 transmission unit, 340 correspondence unit, 350 destination designation reception unit, 360 ensemble data generation unit, 370 communication status acquisition unit, 380 processing adjustment unit, 400 performance data generation device, 410 learning model generation unit, 420 learning model storage unit, 430 music score data reception unit, 440 simulation generation unit, 500 musical instrument, 600 performance output device, 1200 computer, 1210 host controller, 1212 CPU, 1213 GPU, 1214 RAM, 1216 graphics controller, 1218 display device, 1222 communication interface, 1224 storage device, 1230 ROM, 1240 I / O chip, 1242 I / O controller

Claims

1. a storage unit for storing information about a performance data processing device that processes performance data of a musical instrument performance received via a network to generate processed performance data; a destination specification receiving unit that receives, from a device other than the performance data communication device having the transmitting unit that transmits the performance data, a specification of a destination of the performance data to which the performance data is to be transmitted and a specification of a destination of the processed performance data to which the processed performance data is to be transmitted; a correspondence unit that associates the performance data communication device with the performance data processing device using information about the performance data processing device stored in the storage unit, so that the performance data transmitted by the transmission unit of the performance data communication device is processed by the performance data processing device; Equipped with The correspondence unit controls the performance data to be transmitted via the network to a destination of the performance data whose designation has been accepted by the destination designation acceptance unit, and controls the processed performance data to be transmitted via the network to a destination of the processed performance data whose designation has been accepted by the destination designation acceptance unit.

2. the storage unit stores information on a plurality of the performance data processing devices that process performance data differently from one another; 2. The control device according to claim 1, wherein the destination designation receiving unit receives a designation of a destination to which the performance data is to be transmitted from among the plurality of performance data processing devices.

3. the association unit associates the first performance data communication device with the first performance data processing device, and associates the second performance data communication device with the second performance data processing device; The control device a receiving section for receiving, from the first performance data processing device, first processed performance data obtained by processing the performance data received from the first performance data communication device, and for receiving, from the second performance data processing device, second processed performance data obtained by processing the performance data received from the second performance data communication device; an ensemble data generating unit that generates ensemble data by combining the first processed performance data and the second processed performance data received by the receiving unit; The control device according to claim 1 or 2, comprising:

4. The control device described in Claim 3, wherein the ensemble data generation unit generates ensemble data by synthesizing the first processed performance data and the second processed performance data based on a synthesis method instructed by another device different from the performance data communication device.

5. 3. The control device according to claim 1, wherein the association unit associates the plurality of performance data communication devices with the performance data processing device, and causes the performance data processing device to process the performance data received from each of the plurality of performance data communication devices and synthesize the plurality of processed performance data to generate ensemble data.

6. The control device described in Claim 5, wherein the matching unit causes the performance data processing device to process the performance data received from each of the multiple performance data communication devices based on a synthesis method instructed by another device different from the performance data communication device, and generate ensemble data by synthesizing the multiple processed performance data.

7. The control device according to claim 1 or 2; A performance data generating device A system comprising: The performance data generating device a pseudo-generation unit that generates pseudo-performance data of a specified user playing a specified piece of music; and The system comprises: an ensemble data generating section for generating ensemble data by combining processed performance data obtained by processing the performance data received from the performance data communication device by the performance data processing device associated with the performance data communication device by the associating section, and pseudo performance data pseudo-generated by the pseudo generation section; The system further comprises:

8. The performance data generating device a learning model storage unit that stores performance data generated by a user playing a musical piece on a musical instrument and a learning model that receives musical score data as input and outputs performance data, the learning model being generated by machine learning using learning data including musical score data of the musical piece; a music score data receiving unit for receiving music score data; and The system described in claim 7, wherein the pseudo-generation unit inputs the musical score data received by the musical score data receiving unit into the learning model corresponding to the specified user, and obtains the performance data output from the learning model as pseudo-performance data.

9. A memory unit for storing information about a performance data processing device that processes performance data of a performance using a musical instrument received via a network and generates processed performance data; a destination designation receiving unit that receives designation of a destination to which the processed performance data is to be transmitted; a communication status acquisition unit that acquires a communication status of a communication path from the performance data processing device to the transmission destination; a processing adjustment unit that adjusts processing of the performance data by the performance data processing device based on the communication status; a correspondence unit that associates the performance data communication device with the performance data processing device using information about the performance data processing device stored in the storage unit, in order to have the performance data processing device process the performance data transmitted by the transmission unit of the performance data communication device that has the transmission unit that transmits the performance data; Equipped with The association unit controls the processed performance data to be transmitted via the network to the destination designated by the destination designation receiving unit.

10. The control device according to claim 9 , wherein the processing adjustment section controls the content of the processing so that the amount of data of the processed performance data is less than the amount of data corresponding to the communication delay indicated by the communication status.

11. A memory unit for storing information about a performance data processing device that processes performance data of a performance using a musical instrument received via a network and generates processed performance data; a correspondence unit that associates the performance data communication device with the performance data processing device using information about the performance data processing device stored in the storage unit, in order to have the performance data processing device process the performance data transmitted by the transmission unit of the performance data communication device that has the transmission unit that transmits the performance data; a control device comprising: A performance data generating device A system comprising: The performance data generating device a pseudo-generation unit that generates pseudo-performance data of a specified user playing a specified piece of music; a learning model storage unit that stores ensemble data generated by a first user and a second user playing together, and a learning model that receives performance data by the first user as input and outputs performance data by the second user, the learning model being generated by machine learning using learning data including performance data by the first user in the ensemble; and the pseudo-performance generation unit inputs performance data by the first user into the learning model, and acquires performance data output from the learning model as pseudo-performance data; The system comprises: The system further includes an ensemble data generation unit that generates ensemble data by combining processed performance data, which is generated by the performance data processing device associated with the performance data communication device by the association unit, and performance data received from the performance data communication device, with pseudo performance data that is pseudo-generated by the pseudo generation unit.

12. 1. A computer-implemented control method comprising: a storing step of storing in a storage unit information about a performance data processing device that processes performance data of a performance using a musical instrument, the performance data being received via the network, and generates processed performance data; a destination designation receiving step of receiving, from a device other than the performance data communication device having a transmitting unit for transmitting the performance data, designation of a destination of the performance data to which the performance data is to be transmitted and designation of a destination of the processed performance data to which the processed performance data is to be transmitted; a correspondence step of associating the performance data communication device with the performance data processing device using information about the performance data processing device stored in the storage unit, so that the performance data transmitted by the transmission unit of the performance data communication device is processed by the performance data processing device; Equipped with The correspondence step controls the performance data to be transmitted via the network to a destination of the performance data designated in the destination designation receiving step, and controls the processed performance data to be transmitted via the network to a destination of the processed performance data designated in the destination designation receiving step.

13. A computer-implemented control method comprising: a storing step of storing in a storage unit information about a performance data processing device that processes performance data of a performance using a musical instrument, the performance data being received via the network, and generates processed performance data; a destination designation receiving step of receiving a designation of a destination to which the processed performance data is to be transmitted; a communication status acquisition step of acquiring a communication status of a communication path from the performance data processing device to the transmission destination; a processing adjustment step for adjusting the processing of the performance data by the performance data processing device based on the communication status; and a correspondence step for correlating the performance data communication device with the performance data processing device using information about the performance data processing device stored in the storage unit, so that the performance data transmitted by the transmission unit of a performance data communication device that transmits the performance data is processed by the performance data processing device. Equipped with The control method, wherein the associating step controls the processed performance data so that the processed performance data is transmitted via the network to the destination designated in the destination designation receiving step.

14. A computer-implemented control method comprising: a storing step of storing in a storage unit information about a performance data processing device that processes performance data of a performance using a musical instrument, the performance data being received via the network, and generates processed performance data; a correspondence step of associating the performance data communication device with the performance data processing device using information about the performance data processing device stored in the storage unit, in order to have the performance data processing device process the performance data transmitted by the transmission unit of the performance data communication device, which has a transmission unit for transmitting the performance data; a pseudo-generation step of pseudo-generating performance data of a specified piece of music played by a specified user; Equipped with the pseudo-generation step inputs the performance data of the first user to a learning model that receives ensemble data generated by the first user and a second user playing together and generates training data including the performance data of the first user in the ensemble, and outputs performance data of the second user, and acquires the performance data output from the learning model as pseudo-performance data; The control method includes: The control method further comprises an ensemble data generation step of generating ensemble data by combining processed performance data obtained by processing the performance data received from the performance data communication device by the performance data processing device associated with the performance data communication device in the association step with the pseudo performance data generated in the pseudo generation step.

15. A program for causing a computer to execute the control method according to any one of claims 12 to 14.

Images