Information processing systems, information processing methods, and programs

Abstract

To properly maintain a relation between a sounding period of each sound and a temporal variation in control value throughout before and after a movement of the sounding period.SOLUTION: An information processing system 100A comprises: an acquisition part 31 which acquires musical performance data X1 specifying a plurality of sounding periods, corresponding to a plurality of sounds, in time series and control data Y1 representing control value transition as a temporal variation in control value associated with the plurality of sounds; a first adjustment part 321 which moves the plurality of sounding periods on a time axis respectively; and a second adjustment part 322 which moves an object point of the control value transition on the time axis according to a movement of an end point closest to the object point among end points of the plurality of respective sounding periods before the movement by the first adjustment part 321.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] This disclosure relates to a technique for processing information.

Background Art

[0002] Conventionally, a technique for specifying a time series of control values for a plurality of sounds has been proposed. For example, Patent Document 1 discloses a technique for converting the operation amount of a continuously variable operator into a linear value and converting the linear value into a vendor MIDI value.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, the pronunciation period of each of a plurality of musical sounds may move on the time axis. For example, in the scene of DTM (Desk Top Music), a quantization function is provided that matches the start point of each pronunciation period to a point discretely set on the time axis. However, if the time series of control values is maintained when each pronunciation period moves, the relationship between the pronunciation period of each musical sound and the time series of control values will be different before and after the movement of the pronunciation period. In view of the above circumstances, one aspect of this disclosure aims to appropriately maintain the relationship between the pronunciation period of each sound and the temporal change of the control value before and after the movement of the pronunciation period.

Means for Solving the Problems

[0005] To solve the above problems, an information processing system according to one aspect of the present disclosure includes: an acquisition unit that acquires performance data specifying a plurality of sound durations corresponding to a plurality of sounds in a time series, and control data representing control value transitions, which are temporal changes in control values ​​related to the plurality of sounds; a first adjustment unit that moves each of the plurality of sound durations on a time axis; and a second adjustment unit that moves a target point in the control value transition on a time axis in accordance with the movement of the endpoint closest to the target point among the endpoints in each of the plurality of sound durations before the movement by the first adjustment unit.

[0006] An information processing method according to one aspect of the present disclosure acquires performance data specifying a plurality of sound durations corresponding to a plurality of sounds in a time series, and control data representing control value transitions, which are temporal changes in control values ​​related to the plurality of sounds; moves each of the plurality of sound durations on a time axis; and moves the target point in the control value transition on a time axis in accordance with the movement of the endpoint closest to the target point among the endpoints in each of the plurality of sound durations before the movement by the first adjustment unit.

[0007] A program according to one aspect of this disclosure causes a computer system to function as follows: an acquisition unit that acquires performance data specifying multiple sound durations corresponding to multiple sounds in a time series, and control data representing control value transitions, which are temporal changes in control values ​​related to the multiple sounds; a first adjustment unit that moves each of the multiple sound durations on a time axis; and a second adjustment unit that moves a target point in the control value transition on a time axis in accordance with the movement of the endpoint closest to the target point among the endpoints in each of the multiple sound durations before the movement by the first adjustment unit. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram illustrating the configuration of the information processing system in the first embodiment. [Figure 2] This is a schematic diagram of music data. [Figure 3] This is an explanatory diagram of the music data. [Figure 4]This is a block diagram illustrating the functional configuration of an information processing system. [Figure 5] This is an explanatory diagram of the processing performed by the conversion unit. [Figure 6] This is an explanatory diagram illustrating the shifts in the pronunciation period and control period in Case 1. [Figure 7] This is an explanatory diagram illustrating the shifts in the pronunciation period and control period in Case 2. [Figure 8] This is an explanatory diagram illustrating the shifts in the pronunciation period and control period in Case 3. [Figure 9] This is an explanatory diagram of the shifts in the pronunciation period and control period in Case 4. [Figure 10] This is a flowchart of the conversion process. [Figure 11] This is an explanatory diagram of the problems in proportionality 2. [Figure 12] This is a block diagram of the information processing system in the second embodiment. [Modes for carrying out the invention]

[0009] A: First Embodiment Figure 1 is a block diagram of the information processing system 100A in the first embodiment. An electronic musical instrument 200 is connected to the information processing system 100A by wire or wireless connection. The electronic musical instrument 200 is an electronic keyboard instrument including a keyboard unit 21 and an operation pedal 22. The keyboard unit 21 consists of a plurality of keys corresponding to different pitches. Each of the plurality of keys is operated by the user U. The operation pedal 22 is an operator for controlling the acoustic characteristics of musical tones corresponding to the operation of the keyboard unit 21, and is operated, for example, by pressing it down by the user U. The user U plays a desired musical piece by sequentially operating each key of the keyboard unit 21 while appropriately operating the operation pedal 22. The electronic musical instrument 200 transmits a performance signal x representing the operation of the keyboard unit 21 and a control signal y representing the operation of the operation pedal 22 to the information processing system 100A. The performance signal x and the control signal y are signals representing performance to the electronic musical instrument 200.

[0010] The information processing system 100A is a computer system that records the performance of user U on the electronic musical instrument 200. The information processing system 100A can be implemented on an information device such as a smartphone, tablet terminal, or personal computer. The information processing system 100A comprises a control device 11, a storage device 12, a communication device 13, an operating device 14, a display device 15, a sound source device 16, and a sound emission device 17. The information processing system 100A can be implemented as a single device or as a group of devices configured separately from each other. The information processing system 100A may also be mounted on the electronic musical instrument 200.

[0011] The control device 11 is one or more processors that control each element of the information processing system 100A. Specifically, the control device 11 is composed of one or more types of processors, such as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), SPU (Sound Processing Unit), DSP (Digital Signal Processor), FPGA (Field Programmable Gate Array), or ASIC (Application Specific Integrated Circuit).

[0012] The communication device 13 communicates with an external device such as an electronic musical instrument 200. For example, the communication device 13 receives a performance signal x and a control signal y from the electronic musical instrument 200. The communication between the communication device 13 and the electronic musical instrument 200 may be either wired or wireless.

[0013] The storage device 12 is one or more memories that store the programs executed by the control device 11 and various data used by the control device 11. For example, known recording media such as semiconductor recording media and magnetic recording media, or combinations of multiple types of recording media, are used as the storage device 12. In addition, for example, a portable recording medium that is detachable from the information processing system 100A, or a recording medium (e.g., cloud storage) that the control device 11 can access via a communication network, may be used as the storage device 12.

[0014] FIG. 2 is an explanatory diagram of the data stored in the storage device 12. The storage device 12 stores music data M1. The music data M1 is data representing a music piece played by the user U using the electronic musical instrument 200. Specifically, the music data M1 includes performance data X1 and control data Y1. The performance data X1 is data representing the operations of the user U on the keyboard unit 21 and is generated from the performance signal x. The control data Y1 is data representing the operations of the user U on the operation pedal 22 and is generated from the control signal y. The music data M1 is stored in the storage device 12 as a file conforming to, for example, the MIDI standard.

[0015] FIG. 3 is an explanatory diagram of the music data M1. The performance data X1 specifies a plurality of musical tones constituting the music piece played by the user U in time series. That is, the performance data X1 is score data representing the score of the music piece played by the user U. Specifically, the performance data X1 specifies the pitch and the pronunciation period P (P1, P2) for each of the plurality of musical tones constituting the music piece. Each pronunciation period P is specified by a start point pS and an end point pE. Specifically, one pronunciation period P is specified by a combination of the time of the start point pS and the time length of the pronunciation period P, or a combination of the time of the start point pS and the time of the end point pE. In the following description, the start point pS and the end point pE are collectively referred to as the endpoints p of the pronunciation period P. Note that the performance data X1 also represents a chord composed of a plurality of musical tones. For example, a plurality of pronunciation periods P corresponding to different pitches may overlap each other on the time axis.

[0016] The control data Y1 is data representing the temporal change (hereinafter referred to as "control value transition") V of the control value C for a plurality of musical tones specified by the performance data X1. The control value C is a parameter for controlling the acoustic characteristics of the musical tone. The control value C in the first embodiment is sustain, which means the prolongation of the musical tone. That is, the operation pedal 22 is a damper pedal for instructing the prolongation of the musical tone. The control value C is set to a non - negative value corresponding to the operation amount (i.e., the depression amount) of the operation pedal 22. The control value C changes temporally according to the operation from the user U with respect to the operation pedal 22.

[0017] The control value transition V is represented by a polyline including a plurality of inflection points (hereinafter referred to as "target points") q. The control data Y1 in the first embodiment is data specifying each of the plurality of target points q. Each target point q is specified by a combination of a time on the time axis and the control value C. The control period Q in FIG. 2 is a period during which the control value C is a valid numerical value in the control value transition V. Specifically, the period during which the control value C is a positive number other than zero (i.e., a valid numerical value) is the control period Q. The start point qS in FIG. 3 is the target point q corresponding to the start point of the control period Q among the plurality of target points q, and the end point qE in FIG. 2 is the target point q corresponding to the end point of the control period Q among the plurality of target points q.

[0018] As illustrated in FIG. 2, the control device 11 generates music data M2 from music data M1. The music data M2 includes performance data X2 and control data Y2. The performance data X2 is generated from the performance data X1, and the control data Y2 is generated from the control data Y1. Specifically, the performance data X2 is generated by adjusting the performance data X1, and the control data Y2 is generated by adjusting the control data Y1. The performance data X2 and the control data Y2 will be described later.

[0019] The operation device 14 in FIG. 1 is an input device that receives an instruction from the user U. For example, an operator operated by the user U or a touch panel that detects the contact by the user U is used as the operation device 14. Note that an operation device 14 separate from the information processing system 100A may be connected to the information processing system 100A by wire or wirelessly.

[0020] The display device 15 displays an image under the control of the control device 11. For example, various display panels such as liquid crystal display panels or organic EL (electroluminescence) panels can be used as the display device 15. Alternatively, a separate display device 15 may be connected to the information processing system 100A by wire or wireless connection.

[0021] The sound source device 16 generates an acoustic signal A corresponding to the music data M2. Specifically, the sound source device 16 generates an acoustic signal A that represents the waveform of the musical sound represented by the music data M2. That is, an acoustic signal A is generated that represents a waveform in which the acoustic characteristics of the musical sound specified by the performance data X2 are controlled according to the control value C specified by the control data Y2. An acoustic signal A corresponding to the music data M1 may also be generated.

[0022] Furthermore, the control device 11 may execute a program to realize the functions of the sound source device 16. In other words, the element that generates the acoustic signal A (sound source unit) may be realized by either a software sound source realized by the control device 11, or a hardware sound source (sound source device 16) dedicated to generating the acoustic signal A.

[0023] The sound emission device 17 emits the musical sound represented by the acoustic signal A. For example, a speaker or headphones may be used as the sound emission device 17. Alternatively, a separate sound emission device 17 may be connected to the information processing system 100A by wire or wireless connection.

[0024] Figure 4 is a block diagram illustrating the functional configuration of the information processing system 100A. The control device 11 implements multiple functions (acquisition unit 31, conversion unit 32, display control unit 33) for processing music data M1 by executing a program stored in the storage device 12. The functions of the control device 11 may also be implemented by multiple devices configured separately from each other. Some or all of the functions of the control device 11 may be implemented by dedicated electronic circuits.

[0025] The acquisition unit 31 acquires music data M1, which includes performance data X1 and control data Y1. In the first embodiment, the acquisition unit 31 generates performance data X1 from the performance signal x received by the communication device 13. The acquisition unit 31 also generates control data Y1 from the control signal y received by the communication device 13. The acquisition unit 31 stores the music data M1, which includes the performance data X1 and control data Y1, in the storage device 12.

[0026] The conversion unit 32 generates music data M2 by processing music data M1. That is, the conversion unit 32 converts music data M1 to music data M2. The conversion unit 32 includes a first adjustment unit 321 and a second adjustment unit 322. The first adjustment unit 321 generates performance data X2 by processing performance data X1. The second adjustment unit 322 generates control data Y2 by processing control data Y1. Performance data X2, including performance data X2 and control data Y2, is stored in the storage device 12. The display control unit 33 displays the contents of music data M(M1,M2) on the display device 15.

[0027] Figure 5 is an explanatory diagram of the processing performed by the first adjustment unit 321 and the second adjustment unit 322. Figure 5 illustrates the images G(G1, G2) that the display control unit 33 displays on the display device 15. Image G1 is an image representing the content of the music data M1. Image G2 is an image representing the content of the music data M2. Before processing is performed by the conversion unit 32, image G1 is displayed on the display device 15, and when processing is performed by the conversion unit 32, the display on the display device 15 is updated from image G1 to image G2.

[0028] Multiple reference points R are set discretely on the time axis. Specifically, the multiple reference points R are set discretely on the time axis at predetermined intervals. The first adjustment unit 321 generates performance data X2 by quantizing the performance data X1. Quantization is the process of moving each of the multiple sound durations P(P1, P2) on the time axis so that the starting point pS of each sound duration P specified by the performance data X1 coincides with one of the multiple reference points R.

[0029] Specifically, the first adjustment unit 321 moves the starting point pS of each sound-producing period P to the reference point R closest to the starting point pS among a plurality of reference points R. Therefore, each sound-producing period P moves forward or backward along the time axis. For example, in Figure 5, sound-producing period P1 moves forward, and sound-producing period P2 moves backward. As a result of quantization by the first adjustment unit 321, musical tones can be reproduced in which the listener can perceive a clear rhythm. The first adjustment unit 321 stores performance data X2 representing the sound-producing period P after the quantization in the storage device 12. Note that the interval between each reference point R does not have to be constant.

[0030] The second adjustment unit 322 moves each target point q in the control value transition V specified by the control data Y1 on the time axis. Specifically, the second adjustment unit 322 moves each target point q in the control value transition V on the time axis in accordance with the movement of the endpoint p (hereinafter referred to as "nearest point p") that is closest to the target point q on the time axis during multiple sound generation periods P prior to the movement by the first adjustment unit 321. The second adjustment unit 322 stores the control data Y2 representing the control value transition V corresponding to the target point q after the movement in the storage device 12.

[0031] Specifically, the second adjustment unit 322 moves the target point q in the control value transition V by the amount of movement of the nearest point p corresponding to the target point q, in the direction of movement of the nearest point p. For example, in the example in Figure 5, the nearest point p of target points q11 and q12 in the control period Q1 is the starting point qS of the sound generation period P1. The starting point qS of the sound generation period P1 moves forward on the time axis by a time δ1 due to the quantization described above. Therefore, the second adjustment unit 322 moves target points q11 and q12 forward on the time axis by a time δ1.

[0032] Furthermore, the nearest point p of the target point q13 within the control period Q1 is the end point pE of the sound generation period P1. The end point pE of the sound generation period P1, like the start point pS of the sound generation period P1, moves forward on the time axis by time δ1 due to quantization. Therefore, the second adjustment unit 322 moves the target point q13 forward on the time axis by time δ1.

[0033] The nearest point p of target point q14 in control period Q1 is the starting point pS of the immediately following sound generation period P2. The starting point qS of sound generation period P2 moves backward on the time axis by time δ2 due to quantization. Therefore, the second adjustment unit 322 moves target point q14 backward on the time axis by time δ2. As described above, the nearest points p of target points q11 to q13 in control period Q1 and the nearest point p of target point q14 in control period Q1 are the endpoints p of different sound generation periods P(P1,P2). Therefore, target points q11 to q13 and target point q14 move in different directions on the time axis. That is, the shape of the control value transition V1 changes before and after adjustment by the second adjustment unit 322.

[0034] The nearest point p of all target points q21 to q23 during the control period Q2 is the endpoint pE of the sound generation period P2. The endpoint pE of the sound generation period P2 moves backward on the time axis by time δ2 due to quantization. Therefore, the second adjustment unit 322 moves the target points q21 to q23 backward on the time axis by time δ2. In other words, the control value transition V2 is translated backward on the time axis by time δ2 while maintaining its shape.

[0035] As a result of the operation of the first adjustment unit 321 and the second adjustment unit 322 as described above, it is assumed that the sound generation period P and the control period Q move along the time axis as shown in cases 1 to 4 below. In the following description, a form in which the control period Q does not move along the time axis (hereinafter referred to as "proportionality 1") is assumed for comparison with the first embodiment. Proportionality 1 is a form in which the second adjustment unit 322 is omitted from the first embodiment. In proportionality 1, quantization by the first adjustment unit 321 is performed in the same manner as in the first embodiment.

[0036] [Case 1] As illustrated in Figure 6, before quantization is performed by the first adjustment unit 321, it is assumed that the portion of the sound generation period P including the end point pE and the portion of the control period Q including the start point qS overlap each other over time T1. Case 1 is the case in which, from the above state, the sound generation period P moves forward on the time axis by time T2 due to quantization. Time T2 is longer than time T1.

[0037] In proportionality 1, where the control period Q does not move, the sound production period P after quantization and the control period Q do not overlap on the time axis. That is, before the processing by the conversion unit 32, the control value C within the control period Q was applied to the musical tone in the sound production period P, but as a result of the processing by the conversion unit 32, the state changes so that the control value C within the control period Q is not applied to the musical tone in the sound production period P.

[0038] In contrast to proportionality 1, the second adjustment unit 322 of the first embodiment moves the starting point qS of the control period Q forward on the time axis by time T2. Therefore, the relationship in which the sound production period P and the control period Q overlap over time T1 is maintained in the same way as before the movement. That is, even after the processing by the conversion unit 32 is performed, the state in which the control value C in the control period Q is applied to the musical tone in the sound production period P is maintained.

[0039] [Case 2] As illustrated in Figure 7, before the quantization is performed by the first adjustment unit 321, we assume that the control period Q is located after the sound generation period P by a time T1. That is, the starting point qS of the control period Q is located after the end point pE of the sound generation period P by a time T1. Case 2 is the case in which, from the above state, the sound generation period P moves backward on the time axis by a time T2 (T2 > T1).

[0040] In proportionality 1, where the control period Q does not move, the sound production period P after quantization and the control period Q overlap on the time axis. That is, before the processing by the conversion unit 32, the control value C within the control period Q was not applied to the musical tone in the sound production period P, but as a result of the processing by the conversion unit 32, the state changes so that the control value C within the control period Q is applied to the musical tone in the sound production period P.

[0041] In contrast to proportionality 1, the second adjustment unit 322 of the first embodiment moves the starting point qS of the control period Q backward on the time axis by time T2. Therefore, the relationship in which the sound production period P and the control period Q are separated from each other by time T1 is maintained in the same way as before the movement. That is, even after the processing by the conversion unit 32 is performed, the state in which the control value C in the control period Q is not applied to the musical tone in the sound production period P is maintained.

[0042] [Case 3] As illustrated in Figure 8, before the quantization is performed by the first adjustment unit 321, it is assumed that the portion of the sound generation period P including the starting point pS and the portion of the control period Q including the ending point qE overlap with each other over time T1. Case 3 is the case in which, from the above state, the sound generation period P moves backward on the time axis by time T2 (T2 > T1) due to quantization.

[0043] In proportionality 1, where the control period Q does not move, the sound production period P after quantization and the control period Q do not overlap on the time axis. That is, before the processing by the conversion unit 32, the control value C within the control period Q was applied to the musical tone in the sound production period P, but as a result of the processing by the conversion unit 32, the state changes so that the control value C within the control period Q is not applied to the musical tone in the sound production period P.

[0044] In contrast to proportionality 1, the second adjustment unit 322 of the first embodiment moves the endpoint qE of the control period Q backward on the time axis by time T2. Therefore, the relationship in which the sound production period P and the control period Q overlap over time T1 is maintained in the same way as before the movement. That is, even after the processing by the conversion unit 32 is performed, the state in which the control value C in the control period Q is applied to the musical tone in the sound production period P is maintained.

[0045] [Case 4] As illustrated in Figure 9, before quantization is performed by the first adjustment unit 321, we assume that the control period Q is positioned ahead of the sound generation period P by a time T1. That is, the end point qE of the control period Q is positioned ahead of the start point pS of the sound generation period P by a time T1. Case 4 is the case in which, from the above state, the sound generation period P moves forward on the time axis by a time T2 (T2 > T1).

[0046] In proportionality 1, where the control period Q does not move, the sound production period P after quantization and the control period Q overlap on the time axis. That is, before the processing by the conversion unit 32, the control value C within the control period Q was not applied to the musical tone in the sound production period P, but as a result of the processing by the conversion unit 32, the state changes so that the control value C within the control period Q is applied to the musical tone in the sound production period P.

[0047] In contrast to proportionality 1, the second adjustment unit 322 of the first embodiment moves the endpoint qE of the control period Q forward on the time axis by time T2. Therefore, the relationship in which the sound production period P and the control period Q are separated from each other by time T1 is maintained in the same way as before the movement. That is, even after the processing by the conversion unit 32 is performed, the state in which the control value C in the control period Q is not applied to the musical tone in the sound production period P is maintained.

[0048] In Figures 6 to 9, the sound production period P is an example of the "first sound production period," and the control period Q is an example of the "first control period." Also, time T1 is an example of the "first time," and time T2 is an example of the "second time."

[0049] Figure 10 is a flowchart of the process by which the control device 11 generates music data M2 (hereinafter referred to as the "conversion process"). For example, the conversion process is initiated by an instruction from the user U to the operating device 14.

[0050] When the conversion process begins, the control device 11 (acquisition unit 31) acquires the music data M1 (S1). Specifically, the control device 11 generates performance data X1 from the performance signal x and control data Y1 from the control signal y. The control device 11 (acquisition unit 31) stores the music data M1 in the storage device 12 (S2). The control device 11 (display control unit 33) also displays an image G1 representing the music data M1 on the display device 15 (S3).

[0051] User U can instruct the generation of music data M2 by operating the control device 14. The control device 11 (first adjustment unit 321) waits for instructions from user U (S4: NO). When it receives an instruction from user U to generate music data M2 (S4: YES), the control device 11 performs the following processing to convert music data M1 to music data M2 (S5~S8).

[0052] The control device 11 (first adjustment unit 321) generates performance data X2 by quantizing the performance data X1 (S5). Specifically, the control device 11 aligns the starting point pS of each sound generation period P with one of the multiple reference points R. The control device 11 (first adjustment unit 321) stores the performance data X2 in the storage device 12 (S6).

[0053] Furthermore, the control device 11 (second adjustment unit 322) generates control data Y2 by moving each target point q in the control value transition V specified by the control data Y1 on the time axis (S7). Specifically, the control device 11 moves each target point q in the control value transition V on the time axis in accordance with the movement of the nearest point p. The control device 11 (second adjustment unit 322) stores the control data Y2 in the storage device 12 (S8).

[0054] Through the above process, music data M2, which includes performance data X2 and control data Y2, is stored in the storage device 12. The control device 11 (display control unit 33) displays an image G2 representing the music data M2 on the display device 15 (S9). In response to an instruction from the user U to the operation device 14, the control device 11 causes the sound source device 16 to generate an acoustic signal A corresponding to the music data M2 (S10). When the acoustic signal A is supplied to the sound emission device 17, the musical sound represented by the music data M2 is emitted.

[0055] As can be understood from the above explanation, in the first embodiment, when each sound production period P moves along the time axis, the target point q in the control value transition V moves along the time axis in accordance with the movement of the endpoint p(pS,pE) of the sound production period P that is closest to the target point q. Therefore, the relationship between the sound production period P of each musical tone and the control value transition V (control period Q) can be appropriately maintained before and after the conversion process. In particular, in the first embodiment, the target point q in the control value transition V moves in the direction of the movement of the endpoint p(pS,pE) of the sound production period P by the amount of movement of that endpoint p. Therefore, the temporal relationship between the endpoint p of the sound production period P and the control value transition V can be appropriately maintained. Users can enjoy the customer experience of changing the position of each sound production period P on the time axis while maintaining the temporal relationship between the endpoint p of the sound production period P and the control value transition V.

[0056] Incidentally, in the first embodiment, the control value transition V was exemplified as a polyline including multiple target points q. However, in addition to the quantization of each sound production period P by the first adjustment unit 321, a form in which quantization using a reference point R is also performed on the control value transition V (hereinafter referred to as "proportionality 2") is also envisioned. In proportionality 2, as exemplified in Figure 11, for example, only the control value C at each reference point R on the time axis is extracted from the control value transition V. However, proportionality 2 has the problem that the shape of the control value transition V changes excessively before and after the conversion process. Also, the control value C extracted from the control value transition V may not reach zero, and the effect corresponding to the control value C may continue even after the control period Q has elapsed. In contrast to proportionality 2, in the first embodiment, the continuity of the control value transition V is maintained even after the execution of quantization. Therefore, according to the first embodiment, there is an advantage in that each musical tone can be appropriately controlled according to the control value transition V compared to proportionality 2.

[0057] B: Second Embodiment A second embodiment will now be described. For elements whose function is the same as in the first embodiment in each of the embodiments described below, the same reference numerals as in the first embodiment will be used, and detailed descriptions of each will be omitted as appropriate.

[0058] Figure 12 is a block diagram of the information processing system 100B in the second embodiment. The information processing system 100B is a server device that communicates with an electronic musical instrument 200 via a communication network 300, such as the Internet. The information processing system 100B comprises a control device 11, a storage device 12, and a communication device 13.

[0059] The electronic instrument 200 transmits a performance signal x representing an operation on the keyboard unit 21 and a control signal y representing an operation on the control pedal 22 to the information processing system 100B. The communication device 13 receives the performance signal x and control signal y via the communication network 300.

[0060] The control device 11 functions as an acquisition unit 31 and a conversion unit 32. The acquisition unit 31 acquires musical data M1, which includes performance data X1 and control data Y1, similar to the first embodiment. Specifically, the acquisition unit 31 generates performance data X1 from the performance signal x received by the communication device 13 and generates control data Y1 from the control signal y received by the communication device 13. The conversion unit 32 generates musical data M2 by processing the musical data M1, similar to the first embodiment. The configuration and operation of the conversion unit 32 are the same as in the first embodiment. The communication device 13 transmits the musical data M2 generated by the conversion unit 32 to the electronic instrument 200.

[0061] Similar to the first embodiment, the storage device 12 stores the program executed by the control device 11 and various data used by the control device 11. For example, similar to the first embodiment, the storage device 12 stores the music data M1 generated by the acquisition unit 31 and the music data M2 generated by the conversion unit 32.

[0062] The electronic musical instrument 200 includes a keyboard unit 21 and an operating pedal 22 similar to those in the first embodiment, as well as a sound source device 23 and a sound emission device 24. The sound source device 23 generates an acoustic signal A corresponding to the musical data M2 received from the information processing system 100B. The sound emission device 24 emits the musical sound represented by the acoustic signal A. The same effects as in the first embodiment are achieved in the second embodiment as well.

[0063] In the second embodiment, the acquisition unit 31 of the information processing system 100B generated music data M1 from the performance signal x and the control signal y. However, the acquisition unit 31 of the information processing system 100B may also receive the music data M1 generated by the electronic musical instrument 200 via the communication device 13. As can be understood from the above description, the acquisition unit 31 may be either an element that generates the music data M1 itself or an element that receives the music data M1 from another device. In other words, "acquisition" in this disclosure includes both generation and reception.

[0064] C: Variant Specific modifications added to each of the embodiments exemplified above are shown below. Multiple embodiments arbitrarily selected from the embodiments described above and the modifications exemplified below may be combined as appropriate, within the bounds of which they do not contradict each other.

[0065] (1) In the above-described embodiments, quantization is exemplified by aligning the starting point pS of each sound period P with one of a plurality of reference points R on the time axis. However, the process of moving each sound period P on the time axis is not limited to quantization. For example, the first adjustment unit 321 may move each sound period P on the time axis in response to instructions from the user U to the operating device 14. The first adjustment unit 321 may also move each sound period P on the time axis according to rules other than the rule of aligning the starting point pS of each sound period P with the reference point R. Not limited to the specific method of adjustment by the first adjustment unit 321, the second adjustment unit 322 moves each target point q of the control value transition V on the time axis, similar to the above-described embodiments. As can be understood from the above examples, the first adjustment unit 321 is comprehensively represented as an element that moves each of the plurality of sound periods P specified by the performance data X1 on the time axis.

[0066] (2) In each of the above-described forms, it was assumed that quantization would be performed on all of the sound durations P specified by the performance data X1, but it is also possible to perform quantization on only a portion of the multiple sound durations P specified by the performance data X1. For example, one form is assumed in which only one or more sound durations P that satisfy certain conditions are moved on the time axis from among the multiple sound durations P specified by the performance data X1. For example, the first adjustment unit 321 moves only the sound durations P corresponding to musical tones within a specific pitch range from among the multiple sound durations P specified by the performance data X1 on the time axis. As can be understood from the above explanation, the sound durations P that the first adjustment unit 321 moves on the time axis are all or some of the sound durations P specified by the performance data X1.

[0067] (3) In each of the above forms, sustain (damper pedal), which means the extension of the musical tone, was given as an example of control value C, but the content of control value C is not limited to the above examples. For example, in addition to sustain, the numerical values ​​of control changes such as pitch bend, expression, and effect (filter) can be given as examples of control value C.

[0068] (4) In the above-described embodiments, the control value transition V was exemplified as a polyline, but the control value transition V may also be a time-continuous curve. In the above-described embodiments where the control value transition V is a polyline, control data Y(Y1,Y2) was exemplified by specifying the inflection point on the polyline as the target point q, but in embodiments where the control value transition V is a curve, control data Y(Y1,Y2) is used that specifies the control value C on the curve in a time series.

[0069] (5) In the above-described forms, examples were given in which the control value C changes in a multi-valued manner, but a form in which the control value C changes in a binary manner is also conceivable. For example, the control value C is set to either a first value representing the state in which the user U operates the operation pedal 22, or a second value representing the unoperated state in which the operation pedal 22 is released.

[0070] (6) In each of the above-described embodiments, an electronic keyboard instrument was given as an example of an electronic instrument 200, but the types of electronic instruments used to generate the performance data X1 are not limited to the above examples. For example, any type of electronic instrument such as a string instrument, wind instrument, or percussion instrument may be used to generate the performance data X1. The sound corresponding to each sound generation period P is not limited to musical sounds (e.g., harmonic musical tones). For example, the above-described embodiments also apply when generating non-harmonic sounds such as noise.

[0071] (7) In each of the above-described embodiments, an operating pedal 22 operated by the user U by stepping on it was given as an example, but the operating device used to generate the control data Y1 is not limited to the above examples. Any operating device with any configuration that accepts operation by the user U can be used to generate the control data Y1.

[0072] (8) The functions of the information processing system 100 (100A, 100B) in each of the above-described forms are realized through the cooperation of one or more processors constituting the control device 11 and the program stored in the storage device 12, as described above. The programs exemplified above can be provided in a form stored on a computer-readable recording medium and installed on a computer. The recording medium is, for example, a non-transitory recording medium, such as an optical recording medium (optical disc) like a CD-ROM, but also includes any known form of recording medium such as a semiconductor recording medium or a magnetic recording medium. Note that a non-transitory recording medium includes any recording medium except for transient propagation signals (transitory, propagating signals), and volatile recording media are not excluded. Furthermore, in a configuration in which a distribution device distributes a program via a communication network, the recording medium in which the program is stored in the distribution device corresponds to the non-transitory recording medium described above.

[0073] (9) The notation “the nth” (where n is a natural number) in this disclosure is used solely as a formal or convenient label to distinguish each element in notation and has no substantive meaning. Therefore, there is no room for restrictive interpretation of the position or order of processing of each element based on the notation “the nth.”

[0074] D: Addendum From the forms exemplified above, the following configuration can be understood, for example.

[0075] An information processing system according to one aspect of the present disclosure (Aspect 1) comprises: an acquisition unit that acquires performance data specifying a plurality of sound durations corresponding to a plurality of musical tones in a time series, and control data representing control value transitions, which are temporal changes in control values ​​related to the plurality of musical tones; a first adjustment unit that moves each of the plurality of sound durations on a time axis; and a second adjustment unit that moves a target point in the control value transition on the time axis in accordance with the movement of the endpoint closest to the target point among the endpoints in each of the plurality of sound durations before the movement by the first adjustment unit. In the above embodiment, when each sound duration moves on the time axis, the target point in the control value transition moves on the time axis in accordance with the movement of the endpoint of the sound duration closest to the target point. Therefore, the relationship between the sound duration of each musical tone and the control value transition can be appropriately maintained before and after the movement of the sound duration.

[0076] A "control value" is a numerical value applied to the control of multiple musical tones. Specifically, a numerical value applied to the control of the acoustic characteristics of a musical tone (e.g., musical expression or timbre) is considered a "control value." For example, the numerical values ​​of control changes such as sustain (damper pedal), pitch bend, expression, and effects (filter) in MIDI are examples of "control values."

[0077] A "control value transition" is a time series of control values ​​on a time axis, and can be represented, for example, by a polyline or curve. A "target point" is a point in the control value transition defined by its position on the time axis and the numerical value of the control value.

[0078] The "endpoint of the pronunciation period" is the start or end point of the pronunciation period. Among multiple endpoints, including the start and end points of different pronunciation periods, the second adjustment unit moves the target point in accordance with the movement of the endpoint closest to the target point in the control value transition. Note that multiple pronunciation periods may overlap on the time axis.

[0079] In a specific example of Embodiment 1 (Embodiment 2), the first adjustment unit moves each of the plurality of sound-producing periods on the time axis so that the starting point of each sound-producing period coincides with one of a plurality of reference points discretely set on the time axis. In the above embodiment, the starting point of the sound-producing period of each musical note moves to a reference point on the time axis (quantization). Therefore, the timing of each musical note can be adjusted so that a clear rhythm is perceived. Note that the "reference point" is a time point discretely set on the time axis. For example, a plurality of reference points are set discretely at predetermined intervals.

[0080] In a specific example of Embodiment 1 or Embodiment 2 (Embodiment 3), the second adjustment unit moves the target point in the control value transition by the amount of movement of the endpoint closest to the target point, in the direction of movement of the endpoint closest to the target point. In the above embodiments, the target point in the control value transition moves by the amount of movement of the endpoint in the direction of movement of the endpoint of the sound production period. Therefore, the temporal relationship between the endpoint of the sound production period and the control value transition can be appropriately maintained.

[0081] In any specific example of Embodiments 1 to 3 (Embodiment 4), when the portion of the plurality of sound-producing periods that includes the end point of the first sound-producing period and the portion that includes the start point of the first control period in which the control value is a valid numerical value in the control value transition overlap with each other for a period of 1 hour, and the first sound-producing period moves forward on the time axis by a period of 2 hours that exceeds the period of 1 hour, the second adjustment unit moves the start point of the first control period forward on the time axis by the period of 2 hours.

[0082] If the first sound-producing period is moved forward by two hours on the time axis from a state (premise) where the portion including the end point of the first sound-producing period and the portion including the start point of the first control period overlap over a period of one hour, then in the form where the first control period does not move on the time axis, the first sound-producing period and the first control period after the move will not overlap on the time axis. That is, the control value within the first control period will not be applied to the musical tone of the first sound-producing period. In one embodiment of this disclosure, the second adjustment unit moves the start point of the first control period forward by two hours on the time axis. Therefore, the relationship in which the first sound-producing period and the first control period overlap over a period of one hour is maintained.

[0083] In any specific example of Embodiments 1 to 4 (Embodiment 5), when the first control period in which the control value is a valid numerical value in the control value transition is located one hour after the first sound period among the plurality of sound periods, and the first sound period moves backward on the time axis by two hours, which is longer than the first hour, the second adjustment unit moves the starting point of the first control period backward on the time axis by two hours.

[0084] When the first control period is moved backward on the time axis by 2 hours from a state in which the first sound-producing period is located 1 hour after the first sound-producing period, in the form in which the first control period does not move on the time axis, the moved first sound-producing period and the first control period will overlap on the time axis. That is, the control value within the first control period will be applied to the musical tone of the first sound-producing period. In one embodiment of this disclosure, the second adjustment unit moves the starting point of the first control period backward on the time axis by 2 hours. Therefore, the relationship in which the first sound-producing period and the first control period are separated by 1 hour is maintained.

[0085] In any specific example of Embodiments 1 to 5 (Embodiment 6), when the portion of the plurality of sound-producing periods that includes the start of the first sound-producing period and the portion that includes the end of the first control period where the control value is a valid numerical value in the control value transition overlap with each other for a period of 1 hour, and then the first sound-producing period moves backward on the time axis by a period of 2 hours that exceeds the period of 1 hour, the second adjustment unit moves the end of the first control period backward on the time axis by the period of 2 hours.

[0086] If the first sound-producing period shifts backward on the time axis by two hours from a state where the portion including the start point of the first sound-producing period and the portion including the end point of the first control period overlap over a period of one hour, then in a configuration where the first control period does not shift on the time axis, the shifted first sound-producing period and the first control period will not overlap on the time axis. That is, the control value within the first control period will not be applied to the first sound-producing period. In one embodiment of this disclosure, the second adjustment unit shifts the end point of the first control period backward on the time axis by two hours. Therefore, the relationship in which the first sound-producing period and the first control period overlap over a period of one hour is maintained.

[0087] In any specific example of Embodiments 1 to 6 (Embodiment 7), when the first control period in which the control value is a valid numerical value in the control value transition is located one hour ahead of the first sound period among the plurality of sound periods, and the first sound period moves forward on the time axis by two hours, which is longer than the first hour, the second adjustment unit moves the end point of the first control period forward on the time axis by two hours.

[0088] When the first control period is located one hour ahead of the first sound-producing period, and the first sound-producing period moves forward by two hours on the time axis, in the form where the first control period does not move on the time axis, the first sound-producing period after the move and the first control period overlap on the time axis. That is, the control value within the first control period is applied to the first sound-producing period. In one embodiment of this disclosure, the second adjustment unit moves the endpoint of the first control period forward by two hours on the time axis. Therefore, the relationship in which the first sound-producing period and the first control period are separated by one hour is maintained.

[0089] In any specific example of Embodiments 1 to 7 (Embodiment 8), the acquisition unit receives a performance signal representing a performance on an electronic musical instrument from the electronic musical instrument and acquires the performance data from the performance signal. In any specific example of Embodiments 1 to 8 (Embodiment 9), the acquisition unit receives a control signal representing an operation on an operating pedal and acquires the control data from the control signal.

[0090] An information processing method according to one aspect of this disclosure acquires performance data specifying a plurality of sound durations corresponding to a plurality of musical tones in a time series, and control data representing control value transitions, which are temporal changes in control values ​​related to the plurality of musical tones; moves each of the plurality of sound durations on the time axis; and moves the target point in the control value transition on the time axis in accordance with the movement of the endpoint closest to the target point among the endpoints in each of the plurality of sound durations before the movement by the first adjustment unit. Each aspect illustrated for the information processing system is similarly applied to the information processing method.

[0091] A program according to one aspect of the present disclosure causes a computer system to function as follows: an acquisition unit that acquires performance data specifying multiple sound durations corresponding to multiple musical tones in a time series, and control data representing control value transitions, which are temporal changes in control values ​​related to the multiple musical tones; a first adjustment unit that moves each of the multiple sound durations on a time axis; and a second adjustment unit that moves a target point in the control value transition on a time axis in accordance with the movement of the endpoint closest to the target point among the endpoints in each of the multiple sound durations before the movement by the first adjustment unit. [Explanation of Symbols]

[0092] 100A, 100B... Information processing system, 11... Control device, 12... Memory device, 13... Communication device, 14... Operating device, 15... Display device, 16... Sound source device, 17... Sound emission device, 200... Electronic musical instrument, 21... Keyboard unit, 22... Operating pedal, 31... Acquisition unit, 32... Conversion unit, 321... First adjustment unit, 322... Second adjustment unit, 33... Display control unit.

Claims

1. An acquisition unit that acquires performance data specifying multiple sound durations corresponding to multiple sounds in a time series, and control data representing control value transitions, which are temporal changes in control values ​​related to the multiple sounds. A first adjustment unit moves each of the aforementioned plurality of sound durations along the time axis, The second adjustment unit moves the target point in the control value transition along the time axis by the amount of movement of the endpoint closest to the target point among the endpoints in each of the multiple sound generation periods before the movement by the first adjustment unit, in the direction of movement of that endpoint. An information processing system equipped with the following features.

2. An acquisition unit that acquires performance data specifying multiple sound durations corresponding to multiple sounds in a time series, and control data representing control value transitions, which are temporal changes in control values ​​related to the multiple sounds. A first adjustment unit moves each of the aforementioned plurality of sound durations along the time axis, The second adjustment unit moves on the time axis in accordance with the movement of the endpoint closest to the target point among the endpoints in each of the multiple sound generation periods before the movement by the first adjustment unit, and the target point in the control value transition is moved by the second adjustment unit. It is equipped with, When the portion of the plurality of sound-producing periods that includes the end of the first sound-producing period and the portion that includes the start of the first control period in which the control value is a valid numerical value in the control value transition overlap with each other for a period of one hour, and the first sound-producing period moves forward on the time axis by a second hour that exceeds the first hour, the second adjustment unit moves the start of the first control period forward on the time axis by the second hour. Information processing system.

3. An acquisition unit that acquires performance data specifying multiple sound durations corresponding to multiple sounds in a time series, and control data representing control value transitions, which are temporal changes in control values ​​related to the multiple sounds. A first adjustment unit moves each of the aforementioned plurality of sound durations along the time axis, The second adjustment unit moves on the time axis in accordance with the movement of the endpoint closest to the target point among the endpoints in each of the multiple sound generation periods before the movement by the first adjustment unit, and the target point in the control value transition is moved by the second adjustment unit. It is equipped with, In the control value transition described above, if the first control period, for which the control value is a valid numerical value, is located one hour after the first sound period among the plurality of sound periods, and then moves backward on the time axis by two hours, which is longer than the first hour, the second adjustment unit moves the starting point of the first control period backward on the time axis by two hours. Information processing system.

4. An acquisition unit that acquires performance data specifying multiple sound durations corresponding to multiple sounds in a time series, and control data representing control value transitions, which are temporal changes in control values ​​related to the multiple sounds. A first adjustment unit moves each of the aforementioned plurality of sound durations along the time axis, The second adjustment unit moves on the time axis in accordance with the movement of the endpoint closest to the target point among the endpoints in each of the multiple sound generation periods before the movement by the first adjustment unit, and the target point in the control value transition is moved by the second adjustment unit. It is equipped with, When the portion of the plurality of sound-producing periods that includes the start of the first sound-producing period and the portion that includes the end of the first control period where the control value is a valid numerical value in the control value transition overlap over a period of one hour, and the first sound-producing period moves backward on the time axis by a second hour that exceeds the first hour, the second adjustment unit moves the end of the first control period backward on the time axis by the second hour. Information processing system.

5. An acquisition unit that acquires performance data specifying multiple sound durations corresponding to multiple sounds in a time series, and control data representing control value transitions, which are temporal changes in control values ​​related to the multiple sounds. A first adjustment unit moves each of the aforementioned plurality of sound durations along the time axis, The second adjustment unit moves on the time axis in accordance with the movement of the endpoint closest to the target point among the endpoints in each of the multiple sound generation periods before the movement by the first adjustment unit, and the target point in the control value transition is moved by the second adjustment unit. It is equipped with, In the control value transition described above, if the first control period, for which the control value is a valid numerical value, is located one hour ahead of the first sound period among the plurality of sound periods, and then moves forward on the time axis by two hours, which is greater than the first hour, the second adjustment unit moves the end point of the first control period forward on the time axis by two hours. Information processing system.

6. The first adjustment unit moves each of the plurality of sounding periods on the time axis so that the starting point of each sounding period coincides with one of a plurality of reference points discretely set on the time axis. An information processing system according to any one of claims 1 to 5.

7. The acquisition unit is, The system receives a performance signal from an electronic instrument that represents a performance, The performance data is obtained from the performance signal. An information processing system according to any one of claims 1 to 5.

8. The acquisition unit is, Receives a control signal that represents the operation of the control pedal, The control data is obtained from the control signal. An information processing system according to any one of claims 1 to 5.

9. Performance data is obtained that specifies multiple sound durations corresponding to multiple sounds in a time series, and control data is obtained that represents the temporal change in control values ​​for the multiple sounds. Each of the aforementioned multiple pronunciation periods is moved along the time axis, The target point in the control value transition is moved on the time axis by the amount of movement of the endpoint closest to the target point among the endpoints in each of the multiple sound generation periods before the movement, in the direction of movement of that endpoint. Information processing methods implemented by computer systems.

10. When the portion of the plurality of sounding periods that includes the end point of the first sounding period and the portion that includes the start point of the first control period in the control value transition where the control value is a valid numerical value overlap over a period of one hour, and then the first sounding period moves forward on the time axis by a second hour that exceeds the first hour, the movement of the target point involves moving the start point of the first control period forward on the time axis by the second hour. The information processing method of claim 9.

11. In the aforementioned control value transition, when the first control period, in which the control value is a valid numerical value, is located one hour after the first sound period among the plurality of sound periods, and then moves backward on the time axis by two hours, exceeding the first hour, the movement of the target point involves moving the starting point of the first control period backward on the time axis by two hours. The information processing method of claim 9.

12. When the portion of the plurality of sounding periods that includes the start of the first sounding period and the portion that includes the end of the first control period where the control value is a valid numerical value in the control value transition overlap over a period of one hour, and then the first sounding period moves backward on the time axis by a second hour that exceeds the first hour, the movement of the target point involves moving the end of the first control period backward on the time axis by the second hour. The information processing method of claim 9.

13. In the aforementioned control value transition, when the first control period, in which the control value is a valid numerical value, is located one hour ahead of the first sound period among the plurality of sound periods, and the first sound period moves forward on the time axis by two hours, which is greater than the first hour, the movement of the target point involves moving the end point of the first control period forward on the time axis by two hours. The information processing method of claim 9.

14. In moving each of the aforementioned pronunciation periods, each of the multiple pronunciation periods is moved along the time axis so that the starting point of each pronunciation period coincides with one of the multiple reference points discretely set on the time axis. An information processing method according to any one of claims 9 to 13.

15. In acquiring the aforementioned performance data, The system receives a performance signal from an electronic instrument that represents a performance, The performance data is obtained from the performance signal. An information processing method according to any one of claims 9 to 13.

16. In acquiring the aforementioned control data, Receives a control signal that represents the operation of the control pedal, The control data is obtained from the control signal. An information processing method according to any one of claims 9 to 13.

17. An acquisition unit that acquires performance data specifying multiple sound durations corresponding to multiple sounds in a time series, and control data representing control value transitions, which are temporal changes in control values ​​related to the multiple sounds. A first adjustment unit moves each of the aforementioned plurality of sound durations along the time axis, and The second adjustment unit moves the target point in the control value transition along the time axis by the amount of movement of the endpoint closest to the target point among the endpoints in each of the multiple sound generation periods before the movement by the first adjustment unit, in the direction of movement of the endpoint. A program that makes a computer system function.

Images