Information processing device

Abstract

This information processing device (3) comprises: a control information acquisition unit (366) that acquires a plurality of pieces of sound control information in which a plurality of sound control operations for a musical piece are recorded; a correction unit (370) that is able to correct each of the plurality of pieces of acquired sound control information; a playback unit (371) that plays back the musical piece; and a reproduction unit (372) that reproduces, in the played back musical piece, the plurality of sound control operations based on the corrected plurality of pieces of sound control information.

Description

Information processing device

[0001] The present invention relates to an information processing device.

[0002] Conventionally, a communication system has been known that includes a terminal device and multiple external devices and displays a music playback history and a usage history of operation components mounted on the external devices (see, for example, Patent Document 1). In the communication system described in Patent Document 1, the terminal device assigns IDs to each external device and issues various commands to it. Examples of the external devices include a DJ player and a DJ mixer. The DJ player is equipped with a jog dial, and the DJ mixer is equipped with a channel fader. In addition, the DJ player and DJ mixer are equipped with controls such as knobs, buttons, and selector switches.

[0003] A function selection screen is displayed on the operation panel of the terminal device. The function selection screen displays function buttons including a "Playback History" button and a "Usage History" button. The "Playback History" button is a button for obtaining the playback history of songs played on each DJ player from each DJ player. When the "Playback History" button is selected, a playback history display screen is displayed. The playback history display screen displays history information and ID information. The history information displays the playback history in chronological order. The playback history includes the date and time playback started, song title, artist name, label, and playback time.

[0004] The "Usage History" button is a button for obtaining the usage history of each operating component mounted on the DJ mixer. The usage history includes the cumulative usage time (operation time or power-on time), the number of uses, operation log information, etc. When the "Usage History" button is selected, a remaining life display screen is displayed. The remaining life display screen displays a mixer image and ID information. The mixer image includes a fader image showing the channel fader and a panel image showing the touch panel, and color-coded markers corresponding to the remaining life are superimposed on the fader image and panel image.

[0005] JP 2014-99025 A

[0006] Although the communication system described in Patent Document 1 is capable of recording operation log information, the operation log information is only used to determine the lifespan of the DJ mixer, and there is a problem in that the operation log information is not fully utilized. For this reason, there has been a demand for other ways to utilize the recorded operation log information.

[0007] An information processing device according to one aspect of the present invention includes a control information acquisition unit that acquires a plurality of pieces of audio control information that record a plurality of audio control operations for a piece of music, a modification unit that can modify each of the acquired pieces of audio control information, a playback unit that plays the piece of music, and a reproduction unit that reproduces the plurality of audio control operations for the played piece of music based on the modified pieces of audio control information.

[0008] FIG. 1 is a diagram illustrating the configuration of an operation reproduction system according to an embodiment. FIG. 2 is a diagram illustrating an overview of the functions of the operation reproduction system according to an embodiment. FIG. 3 is a block diagram illustrating the configuration of an information processing device according to an embodiment. FIG. 4 is a block diagram illustrating the functional configuration of a control unit according to an embodiment. FIG. 5 is a diagram illustrating a portion of operation log information included in operation reproduction information according to an embodiment. FIG. 6 is a diagram illustrating an example of an operation screen according to an embodiment. FIG. 7 is a diagram illustrating an example of an operation display image according to an embodiment. FIG. 8 is a diagram illustrating a first operation status display section of an operation display image according to an embodiment. FIG. 9 is a diagram illustrating a second operation status display section of an operation display image according to an embodiment. FIG. 10 is a diagram illustrating a crossfader status display section of an operation display image according to an embodiment. FIG. 11 is a diagram illustrating an effect status display section of an operation display image according to an embodiment. FIG. 12 is a diagram illustrating display contents of a first fader status display section according to an embodiment. FIG. 13 is a diagram illustrating display contents of a first equalizer status display section according to an embodiment. FIG. 14 is a diagram illustrating display contents of an effect status display section according to an embodiment. FIG. 15 is a diagram illustrating a jog operation correction image according to an embodiment. FIG. 16 is a diagram illustrating a template image according to an embodiment. FIG. 17 is a diagram illustrating an example of a mix content image according to an embodiment. FIG. 18 is a diagram illustrating an example of an operation reproduction screen according to an embodiment.

[0009] An embodiment of the present invention will be described below with reference to the accompanying drawings. [Configuration of Operation Reproduction System] FIG. 1 is a diagram illustrating the configuration of an operation reproduction system 1A according to this embodiment. As shown in FIG. 1, the operation reproduction system 1A according to this embodiment includes an operation device 2, an information processing device 3 that inputs operation information corresponding to input operations on the operation device 2, and a display device 4 connected to the information processing device 3. The information processing device 3 is configured to be connectable to a network N such as the Internet and can communicate with servers SV1 and SV2 connected to the network N. The servers SV1 and SV2 are not limited to computers but also include storage that can be used as a file server. An example of the server SV2 is a server for a music distribution service. Note that, in this specification, DJ operations correspond to audio control operations and include, for example, operations to control the playback position of music, operations to modify the tone, and operations to control the volume of music. Examples of playback position control operations include scratching, variable speed playback, reverse playback, loop playback, and moving the playback position using the controls on the operation device. Examples of tone processing operations include equalizer (EQ) operations and effect (Fx) operations. Examples of music volume operations include fader operations such as channel faders and cross faders.

[0010] [Overview of Operation Reproduction System] FIG. 2 is a diagram illustrating an overview of the functions of the operation reproduction system 1A. In the operation reproduction system 1A, as shown in FIG. 2, the information processing device 3 records the details of DJ operations as operation reproduction information based on operation information and song identification information input from the operation device 2 in response to DJ operations by the user US on the operation device 2. The information processing device 3 is capable of editing the recorded operation reproduction information and uploading the operation reproduction information to the server SV1 shown in FIG. 1. The operation reproduction information includes operation log information indicating the details of DJ operations by the user US on the controls of the operation device 2 and song identification information for identifying the target song that is the target of the DJ operation. The song identified by the song identification information is the song that was the target of the operation when the operation reproduction information was recorded. Hereinafter, the song identified by the song identification information will be referred to as the target song.

[0011] The information processing device 3 can reproduce the DJ operations of the user US by acquiring pre-recorded operation reproduction information or operation reproduction information generated by another information processing device 3. Reproducing a DJ operation involves reading the operation reproduction information and executing a process similar to the process corresponding to the operation information received from the operation device 2 during the DJ operation. For example, reproducing a DJ operation involves, when a controller is operated a predetermined time after the start of recording of the operation reproduction information, executing a process corresponding to the operation of the controller after the same time has elapsed since the start of operation reproduction processing based on the operation reproduction information. When reproducing a DJ operation, the information processing device 3 generates an image showing the details of the DJ operation based on the operation log information of the operation reproduction information using the second image generation unit 369 (described later) and displays the image on the display device 4. This visualizes the recorded DJ operation of the user. Furthermore, if the information processing device 3 holds a target song identified by song identification information, the information processing device 3 plays the target song according to the details of the DJ operation based on the operation log information. This reproduces the playback of the target song according to the recorded DJ operation. In this way, the operation reproduction system 1A can be said to be a DJ operation visualization system that visualizes the DJ operations of the user. Each component of the operation reproduction system 1A will be described below.

[0012] 1 and 2 is a DJ device such as a DJ controller or a DJ mixer, and is an audio control device having operators that allow DJ operations to be performed. Although detailed illustration is omitted, for example, the operation device 2 has various operators such as a jog dial, a crossfader, a channel fader, etc., as well as a rotary operator that can adjust the volume level of the high frequency band, the volume level of the mid frequency band, the volume level of the low frequency band, the volume level of the input signal, and the application rate of effects.

[0013] 1 is configured with various display panels such as liquid crystal displays. The display device 4 displays an image according to an image signal input from the information processing device 3. In this embodiment, the information processing device 3 is configured with a notebook PC (Personal Computer), and therefore the display device 4 is integrated with the information processing device 3. However, the present invention is not limited to this, and the display device 4 may be a display separate from the information processing device 3.

[0014] [Configuration of Information Processing Device] Figure 3 is a block diagram showing the configuration of the information processing device 3. The information processing device 3 is configured, for example, by a PC capable of executing music management software. The information processing device 3 executes the music management software and plays pre-stored music based on operation signals input from the operation device 2. The information processing device 3 generates and stores operation reproduction information that records the details of DJ operations performed by the user US on the operation device 2. Furthermore, the information processing device 3 reproduces the DJ operations performed by the user US based on the operation reproduction information. As shown in Figure 3, the information processing device 3 includes an input unit 31, an output unit 32, a communication unit 33, a storage unit 34, a memory 35, and a control unit 36, which are electrically connected by a bus line BL.

[0015] The input unit 31 and the output unit 32 have terminals connectable to external devices. The input unit 31 outputs signals input from external devices to the control unit 36. Examples of external devices connected to the input unit 31 include the operation device 2, a pointing device, a keyboard, and an external storage device storing music data. The external storage device is included in the storage unit of the present invention. An example of a terminal constituting the input unit 31 is a USB (Universal Serial Bus) terminal. The output unit 32 outputs information input from the control unit 36 ​​to the external device. Examples of external devices connected to the output unit 32 are the display device 4, speakers, and headphones. Examples of terminals constituting the output unit 32 are a USB terminal, a video output terminal, and an audio output terminal.

[0016] The communication unit 33 communicates with external devices such as servers SV1 and SV2 on the network N under the control of the control unit 36. For example, the communication unit 33 receives operation reproduction information from the server SV1 and outputs it to the control unit 36, and also transmits operation reproduction information generated by the control unit 36 ​​to the server SV1. The communication unit 33 can also receive music data from a server SV2 that is different from the server SV1. The information and data received by the communication unit 33 are stored in the storage unit 34 by the control unit 36. For example, music data received from the server SV2 is stored in the storage unit 34.

[0017] The storage unit 34 is configured with storage devices such as an HDD (Hard Disk Drive) and an SSD (Solid State Drive), and stores programs and data for operating the information processing device 3. For example, the storage unit 34 stores an OS (Operating System) and music management software that control the information processing device 3, as well as an operation reproduction program for executing the operation reproduction process described below. In other words, the storage unit 34 is a recording medium that stores the operation reproduction program. Furthermore, for example, the storage unit 34 can store multiple pieces of music data and operation reproduction information. The memory 35 is a work memory for the control unit 36.

[0018] [Configuration of Control Unit] Fig. 4 is a block diagram showing the functional configuration of the control unit 36. The control unit 36 ​​is configured with at least one processor such as a CPU (Central Processing Unit) and controls the operation of the information processing device 3. The control unit 36 ​​has multiple functional units that are realized when the processor executes programs stored in the storage unit 34. For example, as shown in Fig. 4, the control unit 36 ​​has an operation acquisition unit 361, a first music acquisition unit 362, a first music playback unit 363, a reproduction information generation unit 364, a first image generation unit 365, a reproduction information acquisition unit 366, an identification information acquisition unit 367, a second music acquisition unit 368, a second image generation unit 369, a modification unit 370, a second music playback unit 371, a reproduction unit 372, and a conversion unit 373.

[0019] The operation acquisition unit 361 acquires an operation signal from among signals input from an external device connected to the input unit 31. For example, the operation acquisition unit 361 acquires an operation signal input from an operation device 2 connected to the information processing device 3. The first music acquisition unit 362 acquires, from the storage unit 34, music data corresponding to the music selected by the user, based on the acquired operation signal.

[0020] The first music playback unit 363 plays the music data acquired by the first music acquisition unit 362 based on the operation signal acquired by the operation acquisition unit 361. For example, when an operation signal corresponding to an operation on the jog dial of the controller device 2 is acquired, the first music playback unit 363 changes the playback speed and playback direction of the music based on the acquired operation signal. Also, for example, when an operation signal corresponding to an operation on the channel fader of the first channel on the controller device 2 is acquired, the first music playback unit 363 increases or decreases the volume level of the music being played on the first channel based on the acquired operation signal.

[0021] The reproduction information generating unit 364 generates operation reproduction information that records the content of the user's DJ operations on the operation device 2. Specifically, the reproduction information generating unit 364 generates operation reproduction information by recording the operation content corresponding to the operation signal input from the operation device 2 in chronological order from the start of recording. In other words, the reproduction information generating unit 364 generates operation reproduction information by recording the user's DJ operations on the operation device 2 in chronological order from the start of recording. The generated operation reproduction information is stored in the storage unit 34.

[0022] [Contents of Operation Reproduction Information] The operation reproduction information includes operation log information and music identification information, as shown in Fig. 2. The operation reproduction information is a file in a format that can be transmitted to the server SV1 or the like.

[0023] [Contents of Operation Log Information] FIG. 5 shows a portion of operation log information LG included in the operation reproduction information. The operation log information is information that records, for each operator, the operation details of operators operated by the user among the operators of the operation device 2, along with the elapsed time since the start of recording of the DJ operation. That is, the operation log information includes a plurality of pieces of audio control information that record each of a plurality of audio control operations. Specifically, as shown in FIG. 5, the operation log information LG is information in which records RC, each of which includes a record number RN, an elapsed time ET from the start of recording, an operator type SC indicating the operated operator, a channel number CN of the channel on which the operated operator is located, and an operator position SP indicating the position of the operator after operation, are arranged in the order of the record number RN. The operator type SC, the channel number CN, and the operator position SP correspond to operation content information.

[0024] The record number RN is, for example, an integer value. The record number RN is incremented each time an operation signal is input. A record RC including the record number RN is generated by the reproduction information generator 364, and the generated record RC is added to the operation log information LG. In other words, the record number indicates the order of the recorded operations. However, a record number is not necessary. The channel number CN of a controller shared by each channel is "0." The channel number CN of the controller for the first channel is "1," and the channel number CN of the controller for the second channel is "2." An example of a controller shared by each channel is a crossfader. The controller position SP of a controller other than the jog dial is set to an absolute value within the operable range of the controller. On the other hand, since the jog dial can be continuously rotated clockwise and counterclockwise, the absolute position of the jog dial cannot be recorded. Therefore, if the operated operator is a rotary operator that can be rotated infinitely, such as a jog dial, the operator position SP is set to a relative position based on the position before the operation.

[0025] The operation log information includes time adjustment information TA, which aligns the playback position of the song with the operation timing of the controls when reproducing the DJ operations. The time adjustment information TA is information for aligning the position on the time axis based on the elapsed time ET of the operation reproduction information with the playback position of the song. In other words, the time adjustment information TA is information that associates the elapsed time ET from the start of recording the DJ operations with the playback position of the song. Specifically, the time adjustment information TA is at least one record RC that includes a record number RN, the elapsed time ET, and the playback position PP of the target song corresponding to the elapsed time ET. The playback position PP of the target song is, for example, the playback position from the beginning of the song and is indicated by time. The time adjustment information TA is recorded every time a predetermined amount of time has elapsed since the start of recording the DJ operations, and is also recorded every time an operation that changes the playback position of the song is performed. For example, the time adjustment information TA is recorded when a jog dial that changes the playback speed and playback direction of the song is operated, and when an operation that changes the playback state of the song is performed. Examples of operations that change the playback state of a song include starting playback, stopping playback, reverse playback, and changing the tempo.

[0026] [Song Identification Information] Song identification information is information for identifying the song whose playback is controlled by the DJ operation indicated in the operation log information LG. In other words, song identification information is information that can uniquely identify the song that was being played when the DJ operation was recorded. Examples of song identification information include song metadata. Song metadata includes the song title, artist name, genre, album name, track order on the album, lyricist name, composer name, arranger name, performance time, production code, and product number. In this embodiment, song identification information is information including the song title, artist name, and album name. Additionally, song identification information may be parameter values ​​based on the analysis results of song data, or parameter values ​​calculated using a predetermined function from information such as the song title and artist name. Note that the operation replay information includes song identification information for the song used when the operation replay information was recorded. Therefore, if multiple songs were used when the operation replay information was recorded, the operation replay information includes song identification information for each target song.

[0027] 4 generates an image to be displayed on the display device 4. The first image generation unit 365 outputs the generated image to the display device 4 via the output unit 32, thereby displaying the generated image on the display device 4. For example, the first image generation unit 365 generates an image of the runtime screen G1 of the music management software shown in FIG.

[0028] FIG. 6 is a diagram showing an example of the operation screen G2. For example, the first image generation unit 365 generates an image of the operation screen G2 that is displayed when recording or reproducing DJ operations. As shown in FIG. 6 , the operation screen G2 includes a record start button G21, a record stop button G22, a reset button G23, and a play button G24. The operation screen G2 also includes display areas for the elapsed time from the start of recording, the playback time of the song, and the currently playing deck and the BPM of that deck. The record start button G21 is a button for starting recording of DJ operations. The record stop button G22 is a button for stopping recording of DJ operations. The reset button G23 is a button for erasing recorded DJ operations. In other words, the reset button G23 is a button for erasing generated operation reproduction information. The play button G24 is a button for loading recorded operation reproduction information and reproducing DJ operations based on the operation reproduction information. When the play button G24 is pressed and any operation reproduction information is selected by the user from the operation reproduction information stored in the storage unit 34, the operation reproduction process is executed.

[0029] The reproduction information acquisition unit 366, identification information acquisition unit 367, second music acquisition unit 368, second image generation unit 369, correction unit 370, second music playback unit 371, and reproduction unit 372 shown in FIG. 4 constitute an operation reproduction processing unit RP. The operation reproduction processing unit RP executes an operation reproduction process in which the control unit 36 ​​executes an operation reproduction program to reproduce DJ operations based on operation reproduction information. The reproduction information acquisition unit 366 corresponds to a control information acquisition unit that acquires multiple pieces of sound control information included in the operation log information of the operation reproduction information. The reproduction information acquisition unit 366 acquires operation reproduction information selected by the user from the operation reproduction information stored in the storage unit 34. Alternatively, the reproduction information acquisition unit 366 acquires operation reproduction information from the server SV1 via the communication unit 33. The identification information acquisition unit 367 acquires music identification information included in the operation reproduction information acquired by the reproduction information acquisition unit 366. The second music acquisition unit 368 acquires music data of a target song identified by the song identification information acquired by the identification information acquisition unit 367 from the music data stored in the storage unit 34. Note that there are cases where music data of the target song is not stored in the storage unit 34. In such cases, the second music acquisition unit 368 may acquire the music data of the target song from the server SV2, or may output a signal indicating that the music data of the target song cannot be acquired to the second music playback unit 371. The music data acquired from the server SV2 by the second music acquisition unit 368 is stored in the storage unit 34. The storage of music data here includes temporary storage by the storage unit 34.

[0030] The second image generation unit 369 reads the operation reproduction information acquired by the reproduction information acquisition unit 366 and generates various images, as well as generating various reproduction images of DJ operations based on the operation reproduction information by the reproduction unit 372 described below.

[0031] [Operation Display Image] FIG. 7 is a diagram showing an example of an operation display image P1 showing the details of DJ operations based on operation reproduction information for each operator provided on the DJ equipment. For example, the second image generation unit 369 reads out the operation reproduction information stored in the storage unit 34 and generates the operation display image P1, an example of which is shown in FIG. 7, as an image showing the details of the DJ operations included in the operation reproduction information. The operation display image P1 is an image showing the operation status of each operator on the DJ equipment, the contents of the target songs loaded when the DJ operations are reproduced, and the connections between the target songs. For example, the operation display image P1 is an image showing the operation status of each operator according to the elapsed time since the start of recording of the operation reproduction information. In other words, the operation display image P1 is an operation content display image showing the contents of each of the multiple sound control information included in the operation log information of the operation reproduction information. More specifically, the operation display image P1 is an image showing the operation status of each operator on the DJ equipment and the playback status of the target songs. 7 includes a first song information display area P11, a second song information display area P12, and a details display area P13. In the operation display image P1, a straight line PL indicating the playback position of the target song is set when the DJ operation based on the operation reproduction information is reproduced by the reproduction unit 372 (described later). The straight line PL moves along the horizontal axis, on which the elapsed time of the operation log information in the operation reproduction information is set, to indicate the playback position of the target song.

[0032] [Configuration of First Song Information Display Unit and Second Song Information Display Unit] The first song information display unit P11 displays information about a first song being played on a first channel. Specifically, the first song information display unit P11 displays the name of the first song, the BPM of the first song, the playback position of the first song, the remaining time of the first song, the waveform of the first song, and the song structure of the first song. The second song information display unit P12 displays information about a second song being played on a second channel. Specifically, the second song information display unit P12 displays the name of the second song, the BPM of the second song, the playback position of the second song, the remaining time of the second song, the waveform of the second song, and the song structure of the second song.

[0033] When the reproduction unit 372 (described later) reproduces DJ operations based on operation reproduction information, lines PL1 and PL2 indicating the playback position of each song are set on the waveform of the first song in the first song information display area P11 and the waveform of the second song in the second song information display area P12. Like the line PL, the lines PL1 and PL2 move on the waveform of the target song to indicate the playback position of the target song. Note that the information about songs displayed on each song information display area P11 and P12 is not limited to the above. For example, each song information display area P11 and P12 only needs to display at least the name of the song, and may not display at least one of the other pieces of information. Furthermore, each song information display area P11 and P12 may display information about the song other than the above.

[0034] [Configuration of the Detail Display Section] The detail display section P13 displays the operation details of the controls based on the operation log information of the operation reproduction information and the playback status of music on each channel, with the elapsed time included in the operation log information of the operation reproduction information represented on the horizontal axis. The detail display section P13 is located below the first music information display section P11 and the second music information display section P12 in the operation display image P1. The detail display section P13 includes a first operation status display section P14, a second operation status display section P15, a crossfader status display section P16, and an effect status display section P17. In the detail display section P13, the first operation status display section P14, the crossfader status display section P16, the second operation status display section P15, and the effect status display section P17 are arranged in order from top to bottom, and each of the display sections P14 to P17 extends along the horizontal axis representing the elapsed time.

[0035] [Configuration of First Operation Status Display Unit] Fig. 8 is a diagram showing the first operation status display unit P14. The first operation status display unit P14 displays the playback status of the first music piece and the operation status of each operator provided for the first channel along with the elapsed time in the operation reproduction information. As shown in Figs. 7 and 8, the first operation status display unit P14 includes a time display unit TD that indicates the elapsed time included in the operation log information, as well as a first playback status display unit P14A, a first fader status display unit P14B, and a first equalizer status display unit P14C.

[0036] The first playback status display area P14A displays the playback status of the first song. As shown in FIG. 15 , the first playback status display area P14A displays the playback period of the first channel using a rectangle P141. The timing at which the song was loaded is indicated by the elapsed time corresponding to the left edge of the rectangle P141 representing the playback period. The rectangle P141 also contains characters indicating the track number of the song. The first playback status display area P14A displays the timing at which the CUE button was pressed using a downward-pointing black triangle symbol P142. The first playback status display area P14A also displays the duration for which the CUE button was pressed using a line symbol P143 connecting the two symbols P142. Pressing the CUE button during playback of a song stops playback of that song. The first playback status display area P14A displays the timing at which the START / PAUSE button was pressed using a downward-pointing white triangle symbol P144.

[0037] The first fader status display section P14B displays the operation status of the first channel fader. As shown in FIG. 8 , the first fader status display section P14B has an effective range set along the vertical direction, with the upper end of the effective range indicating the position of the first channel fader when the volume level of the first channel is maximum, and the lower end of the effective range indicating the position of the first channel fader when the volume level of the first channel is minimum. The first fader status display section P14B then displays the position of the first channel fader according to elapsed time using a line P146. The line P146 indicates changes in the volume level of the first channel and corresponds to a volume level display line along the time series. In the example of FIG. 8 , during the period from 0 minutes 0 seconds to 2 minutes 40 seconds, which is the elapsed time indicated by the horizontal axis, the first fader status display section P14B displays that the first channel fader is positioned at the position where the volume level of the first channel is maximum. Also, in the example of Figure 15, during the period from 3 minutes 25 seconds to 6 minutes 0 seconds, the first fader status display section P14B indicates that the first channel fader is positioned at the position where the volume level of the first channel is at its minimum.

[0038] The first equalizer status display section P14C displays the operation status of the controls for adjusting the volume level of the high-frequency band, the mid-frequency band, and the low-frequency band. These controls are, for example, rotary controls such as rotary volume controls. As shown in FIG. 8 , the first equalizer status display section P14C has an effective range set along the vertical direction, with the upper end of the effective range indicating the position of the control when the volume level of the corresponding band is maximum, and the lower end of the effective range indicating the position of the control when the volume level of the corresponding band is minimum. The first equalizer status display section P14C then displays the position of each control over elapsed time using lines P147, P148, and P149. Specifically, in the first equalizer state display section P143, the position of the operator for adjusting the volume level of the high frequency band is displayed by a line P147, the position of the operator for adjusting the volume level of the mid frequency band is displayed by a line P148, and the position of the operator for adjusting the volume level of the low frequency band is displayed by a line P149. The colors of the lines P147, P148, and P149 may be different from each other, and the line types of the lines P147, P148, and P149 may be different from each other.

[0039] [Configuration of Second Operation Status Display Section] FIG. 9 is a diagram showing the second operation status display section P15. As shown in FIGS. 7 and 9, the second operation status display section P15 displays the playback status of the second music piece and the operation status of each operator provided for the second channel in chronological order from the start of reading of the operation reproduction information. Specifically, the second operation status display section P15 includes a time display section TD, as well as a second playback status display section P15A, a second fader status display section P15B, and a second equalizer status display section P15C. The second playback status display section P15A displays the playback status of the second music piece. The configuration of the second playback status display section P15A is the same as the configuration of the first playback status display section P14A. That is, as shown in FIG. 16 , the second playback status display section P15A displays the playback period of the second channel using a rectangle P151, the timing when the CUE button was pressed using a symbol P152 similar to the symbol P142, the period during which the CUE button was continuously pressed using a symbol P153 similar to the symbol P143, and the timing when the START / PAUSE button was pressed using a symbol P154 similar to the symbol P144. Note that in the second playback status display section P15A of FIG. 9 , a hollow rectangle P155 is set in the rectangle P151 indicating the playback period of the second channel. The rectangle P155 indicates the period during which the jog dial for the second channel was operated during the playback period of the second music piece. When the user clicks or taps the rectangle P155, a jog operation correction image P2 (described later) is displayed, allowing the correction unit 370 to correct and edit the jog dial operation information.

[0040] The second fader status display section P15B displays the operating status of the second channel fader. Similar to the first fader status display section P14B, the second fader status display section P15B displays the position of the second channel fader over time using a line P156. The line P156 indicates changes in the volume level of the second channel and corresponds to a volume level display line along the time series. The second equalizer status display section P15C displays the operating status of the controls for adjusting the volume level of the high frequency band, the mid frequency band, and the low frequency band. Similar to the configuration of the first equalizer status display section P14C, the second equalizer status display section P15C displays the position of the control for adjusting the volume level of the high frequency band using a line P157, the position of the control for adjusting the volume level of the mid frequency band using a line P158, and the position of the control for adjusting the volume level of the low frequency band using a line P159.

[0041] [Configuration of the Crossfader Status Display Unit] FIG. 10 is a diagram illustrating the crossfader status display unit P16. As shown in FIG. 7 , the crossfader status display unit P16 is disposed between the first operation status display unit P14 and the second operation status display unit P15 and extends along the horizontal axis. As shown in FIGS. 7 and 10 , the crossfader status display unit P16 includes a time display unit TD and a position display unit P16A that displays the position of the crossfader. Here, the crossfader is configured to be slidable, for example, in one direction. By sliding the crossfader toward one end of the sliding range, the proportion of the first track in the volume of the currently playing music becomes greater than the proportion of the second track. When the crossfader is positioned at one end of the sliding range, the proportion of the first track becomes the largest. By sliding the crossfader toward the other end, the proportion of the second track in the volume of the currently playing music becomes greater than the proportion of the first track. When the crossfader is positioned at the other end of the sliding range, the proportion of the second track becomes the largest.

[0042] To display the operation status of the crossfader operated in this manner, the position display section P16A displays the position of the crossfader based on the operation log information LG using a line P161 within an effective range along the vertical direction, as shown in FIG. 10 . The position display section P16A indicates that the crossfader is being operated toward one end of the sliding range as the position moves upward relative to the crossfader status display section P16, where the first operation status display section P14 is located. The position display section P16A also indicates that the crossfader is being operated toward the other end of the sliding range as the position moves downward relative to the position display section P16A, where the second operation status display section P15 is located. In the examples of FIGS. 7 and 10 , the volume ratio between the first and second music tracks is adjusted using the first and second channel faders, so the crossfader position remains unchanged and set to the intermediate value of the effective range of the position display section P16A. In this way, by arranging the position display section P16A and displaying the position of the crossfader as described above, it becomes easier to understand whether the crossfader is being operated to the first channel or the second channel.

[0043] [Configuration of Effect Status Display Section] Fig. 11 is a diagram showing the effect status display section P17. As shown in Figs. 7 and 11, the effect status display section P17 displays the type of effect, the channel to which the effect is applied, the period during which the effect is on, and the application rate of the effect. As shown in Fig. 11, the effect status display section P17 includes a time display section DP, as well as an on status display section P17A, a rate display section P17B, a type display section P17C, and a target display section P17D.

[0044] The on-state display section P17A uses a rectangle P171 to indicate the period during which the effect is on. The percentage display section P17B displays the application percentage, generally expressed as "Dry / Wet." The percentage display section P17B displays the application percentage of the effect in the vertical effective range using a line P172. The higher the line P172 is positioned within the effective range, the greater the application percentage of the effect, and the lower the line P172 is positioned within the effective range, the smaller the application percentage of the effect.

[0045] The type display section P17C displays the type of effect. Specifically, the type display section P17C displays a name indicating the type of effect that has been set within a rectangle P173 that indicates the setting period for the effect. In the example of FIG. 18 , it is shown that "ECHO" was set during the period from 0 minutes 0 seconds to 6 minutes 20 seconds of the elapsed time indicated by the horizontal axis, and "FLANGER" was set from 6 minutes 20 seconds onwards. The target display section P17D displays the channel to which the effect is to be set. Specifically, the target display section P17D displays a name indicating the channel to which the effect is to be set within a rectangle P174 that indicates the setting period for the setting target. In the example of FIG. 11 , it is shown that the first channel was set as the setting target during the period from 0 minutes 0 seconds to 7 minutes 50 seconds of the elapsed time indicated by the horizontal axis, and the master was set as the setting target from 7 minutes 50 seconds onwards. Note that "master" indicates the master output.

[0046] By checking the operation display image P1 described above, the content of the DJ operations recorded in the operation reproduction information can be easily understood. In addition, by checking the operation display image P1, it is possible to easily understand what songs were mixed and how they were mixed.

[0047] [Configuration of the Correction Unit] The correction unit 370 corrects the operation reproducibility information stored in the storage unit 34 based on an operation signal input from the input unit 31 when the operation display image P1 is generated and displayed by the second image generation unit 369. More specifically, the correction unit 370 corrects, as the target sound control information for correction, sound control information selected in the operation display image P1 from among multiple pieces of sound control information included in the operation reproducibility information. In other words, the correction unit 370 can also be considered an editing unit capable of editing operation reproducibility information including sound control information. The correction unit 370 corrects, for example, first channel fader operation information, second channel fader operation information, crossfader operation information, jog dial operation information, and effect setting information included in the operation reproducibility information.

[0048] FIG. 12 shows the display content of the first fader status display section P14B when correcting the first channel fader operation information. The upper part of FIG. 12 shows the first fader status display section P14B, including a line P146 indicating the position of the first channel fader. As shown in the lower part of FIG. 12, when the line P146 is operated by the user, the second image generation section 369 changes the line P146 to a curved or linear shape in accordance with the user's operation. The correction section 370 corrects the first channel fader operation information in the operation log information in accordance with the changed line P146. In this way, the correction section 370 corrects or edits the first channel fader operation information indicating the audio control operation for the first channel fader in accordance with the user's operation on the line P146. Although not shown, when the user operates a line P156 that is included in the second fader state display section P15B and indicates the position of the second channel fader, the second image generation section 369 changes the line P156 in accordance with the user's operation. Then, the correction section 370 corrects or edits the second channel fader operation information, which is the acoustic control information for the second channel fader in the operation log information, in accordance with the changed line P156.

[0049] FIG. 13 shows the display content of the first equalizer status display unit P14C when correcting the first equalizer operation information. The upper part of FIG. 13 shows the first equalizer status display unit P14C including the lines P147 to P149. The lower part of FIG. 13 shows the first equalizer status display unit P14C when line P149 is selected as the target of correction or editing among the lines P147 to P149. For example, when a user performs an operation to align the lines P147 and P148 with line P149, the second image generation unit 369 aligns the lines P147 and P148 with line P149 in accordance with the user's operation. Then, the correction unit 370 corrects the first equalizer operation information, which is acoustic control information for the controls that adjust the volume levels of each frequency band of the first channel in the operation log information, in accordance with the changed lines P147 and P148. That is, the second image generation unit 369 changes the lines P147 to P149 into a curved or linear shape in response to the user's operation, and the correction unit 370 corrects or edits the first equalizer operation information in response to the changed lines P147 to P149. Although not shown, when the user operates lines P157 to P159, which are included in the second equalizer status display unit P15C and indicate the positions of controls that adjust the volume levels of each frequency band, the second image generation unit 369 changes the lines P157 to P159 in response to the user's operation. Then, the correction unit 370 corrects or edits the second equalizer operation information, which is acoustic control information for the controls that adjust the volume levels of each frequency band of the second channel in the operation log information, in response to the changed lines P157 to P159.

[0050] FIG. 14 illustrates the display contents of the effect status display section P17 when modifying effect setting information. As described above, the effect status display section P17 includes the time display section DP, the on-state display section P17A, the ratio display section P17B, the type display section P17C, and the target display section P17D. As shown in FIG. 14 , when the user selects a rectangle P173 indicating the effect setting period in the type display section P17C, the second image generation unit 369 displays a submenu SM1 for selecting an effect type. In the example of FIG. 14 , the submenu SM1 includes ECHO, DELAY, SPIRAL, HELIX, and REVERB as options. When the user selects an effect in the submenu SM1, the second image generation unit 369 sets the selected effect to the selected effect setting period. The modification unit 370 then modifies or edits the operation log information so that the selected effect is set to the selected effect setting period. If the length and position of the effect setting period indicated by the rectangle P173 are changed by the user, the second image generation unit 369 displays the type display area P17C reflecting the changed effect setting period. Then, the modification unit 370 modifies or edits the effect setting period in the operation log information in accordance with the changed effect setting period.

[0051] As shown in FIG. 14 , when the user selects a rectangle P174 indicating a setting period in the object display section P17D, the second image generation unit 369 displays a submenu SM2 for selecting a setting target. In the example of FIG. 14 , the submenu SM2 includes options for Channel 1, Channel 2, Channel 3, Channel 4, and Master. When the user selects a setting target in the submenu SM2, the second image generation unit 369 sets the selected setting target to the selected setting period. The correction unit 370 then corrects or edits the operation log information so that the selected setting target is set to the selected setting period. Note that if the user changes the length and position of the setting period of the setting target indicated by the rectangle P174, the second image generation unit 369 displays the object display section P17D reflecting the changed setting period. The correction unit 370 then corrects or edits the setting period of the effect setting target in the operation log information according to the changed setting period.

[0052] Although not shown, when the user performs an operation to add or delete a rectangle P171 indicating the period during which an effect is on in the on state display section P17A, the second image generation unit 369 adds or deletes the rectangle P171 from the on state display section P17A. Furthermore, when the user performs an operation to change the position and length of the rectangle P171, the second image generation unit 369 changes the position and length of the rectangle P171 in the on state display section P17A. Furthermore, although not shown, when the user performs an operation to change the line P172 indicating the effect application rate in the rate display section P17B, the second image generation unit 369 changes the line P172 to a curved or linear form. The correction unit 370 then corrects the operation log information in accordance with the user's changes to the on state display section P17A and the rate display section P17B. In this way, the correction unit 370 corrects the effect setting information, including the effect on period, the effect application rate, the effect type, and the channel to which the effect is applied.

[0053] 15 is a diagram showing a jog operation correction image P2 that is displayed when a period during which the jog dial was operated is selected. As shown in FIG. 9, the period during which the jog dial for the second channel was operated is set by a rectangle P155 in the second playback state display area P15A of the second operation state display area P15. When the user selects the rectangle P155, the second image generation unit 369 displays the jog operation correction image P2 shown in FIG. 15. The jog operation correction image P2 is included in the operation display image P1. The jog operation correction image P2 includes a song information display area P21 that displays information about the target song operated by the jog dial, a partial waveform display area P22 that displays a partial waveform including a portion of the overall waveform of the target song that changes due to operation of the jog dial, and a jog dial state display area P23 that displays the operation state of the jog dial.

[0054] The song information display section P21 displays the same content as the first song information display section P11 or the second song information display section P12. Specifically, when the jog dial operation period selected by the user is set in the first playback status display section P14A, the song information display section P21 displays information about the first song. Furthermore, when the jog dial operation period selected by the user is set in the second playback status display section P15A, the song information display section P21 displays information about the second song. That is, the song information display section P21 displays the name, BPM, playback position, remaining time, overall waveform, song structure, etc. of the target song. The partial waveform display section P22 displays an enlarged portion of the overall waveform of the target song within a predetermined period that includes the jog dial operation period.

[0055] The jog dial status display section P23 displays the operation status of the jog dial using a representation method known as scratch music notation. Specifically, the jog dial status display section P23 indicates the beat position on the horizontal axis and the rotation direction of the jog dial on the vertical axis. More specifically, the jog dial status display section P23 indicates clockwise rotation of the jog dial with an upward line P231 and counterclockwise rotation of the jog dial with a downward line P232. The jog dial status display section P23 also indicates the point at which the sound of the channel operated by the jog dial in playback mode is cut off by the crossfader with a line P233 along the horizontal axis.

[0056] FIG. 16 illustrates a template image P3 displayed together with the jog operation correction image P2. The user can freely change the start positions, stop positions, and inclination of the lines P231 and P232, as well as the position of the line P233, in the jog dial state display area P23. The second image generation unit 369 changes the start positions, stop positions, and inclination of the lines P231 and P232, as well as the position of the line P233, in response to the user's operation. In other words, the user can freely change the jog dial operation state based on the operation log information. Additionally, the jog dial operation state displayed in the jog dial state display area P23 can be replaced with the jog dial operation state displayed by a template TP1, TP2, TP3, etc. selected from the template image P3 displayed by the second image generation unit 369, as shown in FIG. 16. In this case, the second image generation unit 369 changes the jog dial operation state displayed in the jog dial state display area P23 to the jog dial operation state displayed by the selected template. Then, the correction unit 370 corrects or edits the jog dial operation information indicating the operation state of the jog dial to be corrected in the operation log information, according to the operation state of the jog dial changed in the jog dial state display unit P23.

[0057] The second image generating unit 369 generates the above-described operation display image P1, and the operation display image P1 is displayed on the display device 4, thereby making it possible to visualize the contents of the operation reproduction information. Furthermore, the user operates the displayed operation display image P1, which allows the modifying unit 370 to modify and edit the operation contents of each operator, and ultimately allows the user to modify and edit the multiple pieces of sound control information included in the operation reproduction information.

[0058] 4 corresponds to the playback unit of the present invention, and plays back the music data acquired by the second music acquisition unit 368. At this time, the second music playback unit 371 changes the playback state of the target music in accordance with the user's DJ operation on the operation device 2 reproduced by the reproduction unit 372. Note that if the second music acquisition unit 368 cannot acquire the music data of the target music, the second music playback unit 371 does not play back the music data.

[0059] [Configuration of Reproduction Unit] The reproduction unit 372 reproduces DJ operations based on the operation reproduction information acquired by the reproduction information acquisition unit 366. That is, if the operation reproduction information has been corrected by the correction unit 370, the reproduction unit 372 reproduces the DJ operations, which are sound control operations, based on the corrected operation reproduction information. Specifically, the reproduction unit 372 analyzes operation log information of the operation reproduction information and controls playback of the target song by the second music playback unit 371 based on the operation log information. More specifically, the reproduction unit 372 reproduces the DJ operations of the user on the operation device 2 at the time the operation reproduction information was recorded, for the target song being played by the second music playback unit 371, based on the operation log information of the acquired operation reproduction information. For example, when the reproduction unit 372 reproduces DJ operations based on the operation reproduction information while the above-described operation display image P1 is displayed, the reproduction unit 372 reproduces the DJ operations based on the operation reproduction information, and the second music playback unit 371 plays the target song under the control of the reproduction unit 372. At this time, the second image generating unit 369 moves the line PL indicating the elapsed time of the operation log information and the lines PL1 and PL2 indicating the playback positions of each song in the operation display image P1 to the right as time passes.

[0060] [Configuration of the Conversion Unit] The conversion unit 373 converts the operation replay information into operation replay information in a common format. Specifically, the conversion unit 373 converts the operation replay information including multiple pieces of acoustic control information into a format usable by other devices. The operation replay information converted by the conversion unit 373 may be uploaded to the server SV1 or the like on the network N by the communication unit 33, as described above.

[0061] [Mix Content Image] FIG. 17 is a diagram showing an example of a mix content image P4 based on operation reproduction information. Specifically, FIG. 17 is an example of a mix content image P4 showing the connection between multiple target songs based on the operation log information LG. The second image generation unit 369 generates a mix content image P4, an example of which is shown in FIG. 17, as an image showing the content of DJ operations read from the operation reproduction information by the reproduction unit 372. Here, the above-mentioned operation display image P1 displays the operation status of each operator as well as the playback status of songs on each channel. In contrast, the mix content image P4 is an image showing the mix content of multiple songs based on the operation reproduction information. In other words, the mix content image P4 is an image showing the mix status of songs based on the result of reproducing the DJ operations by the reproduction unit 372.

[0062] The mix content image P4 includes a first deck display section P4A, a second deck display section P4B, a third deck display section P4C, and a fourth deck display section P4D. In this embodiment, the music loaded onto the first and second decks is played on the first channel, and the music loaded onto the third and fourth decks is played on the second channel. Therefore, the first deck display section P4A and the second deck display section P4B display the playback status of the music played on the first channel, and the third deck display section P4C and the fourth deck display section P4D display the playback status of the music played on the second channel.

[0063] Each deck display unit P4A, P4B, P4C, and P4D displays the playback period of a song using a rectangle P41 that follows the elapsed time. When a song is being looped, each deck display unit P4A, P4B, P4C, and P4D displays a rectangle P411 indicating the loop playback period within the rectangle P41, and also displays a loop playback mark P412 within the rectangle P411. For example, in the example of FIG. 17 , there is a loop playback period at the end of the song loaded on the second deck, and there is also a loop playback period at the end of the song loaded on the fourth deck, so the deck display units P4B and P4D have the rectangle P411 and the loop playback mark P412 set within the rectangle P41 indicating the playback period of the song.

[0064] Each deck display section P4A, P4B, P4C, and P4D displays a waveform P42 of a song loaded on the corresponding deck. For example, if a DJ operation is performed to change the playback direction and playback speed of a song, the song's waveform P42 is an edited version of the song's original waveform to match the DJ operation. In other words, the waveform P42 is a waveform edited from the song's original waveform based on operation log information. Note that in each deck display section P4A, P4B, P4C, and P4D in the example of FIG. 17 , the waveform P42 is positioned above the rectangle P41. However, this is not a limitation, and the waveform P42 may be positioned below the rectangle P41 or displayed overlapping the rectangle P41. Furthermore, as long as the rectangle P411 and the loop playback mark P412 indicating the loop playback period are visibly displayed, only one of the rectangle P41 and the waveform P42 may be displayed.

[0065] Like the fader status display sections P142 and P152, each deck display section P4A, P4B, P4C, and P4D displays the position of the channel fader for the channel associated with the deck using a line P43 in accordance with the playback period of the music. Each deck display section P4A, P4B, P4C, and P4D has an effective range set along the vertical direction in the area above the rectangle P41 that overlaps with the waveform P42. The upper end of the effective range indicates the position of the channel fader when the volume level of the corresponding channel is maximum, and the lower end of the effective range indicates the position of the channel fader when the volume level of the corresponding channel is minimum. Based on the operation reproduction information, each deck display section P4A, P4B, P4C, and P4D displays the position of the channel fader when the music loaded on the corresponding deck is being played using a line P43.

[0066] By checking such a mix content image P4, it is possible to easily understand how the music loaded onto each deck was mixed when the operation reproduction information was recorded.

[0067] Fig. 18 is a diagram showing an example of the operation reproduction screen G3. The second image generation unit 369 generates an image of the operation reproduction screen G3 shown in Fig. 18. Specifically, the second image generation unit 369 generates an image of the operation reproduction screen G3 showing the content of the DJ operation performed on the operation device 2 and reproduced by the reproduction unit 372 based on the operation reproduction information, and causes the generated image to be displayed on the display device 4. In other words, the second image generation unit 369 generates an image that visualizes the DJ operation reproduced by the reproduction unit 372.

[0068] The operation reproduction screen G3 includes a first song information display section G31, a second song information display section G32, an operation display section G33, a menu button G34, a speed adjustment button G35, a reproduction start button G36, and a reproduction position display bar G37. The first song information display section G31 and the second song information display section G32 are located at the top of the operation reproduction screen G3. The first song information display section G31 is located on the left side and displays information about the first song among the target songs. Specifically, the first song information display section G31 displays the title, playback time, remaining playback time, and BPM of the first song, as well as waveform information about the first song. The waveform information is a graph of signal strength according to the playback time of the song, with the horizontal axis representing the playback time of the song and the vertical axis representing the signal strength of the song. The second song information display section G32 is located on the right side and displays information about the second song among the target songs. The display content of the second music information display section G32 is the same as that of the first music information display section G31.

[0069] The operation display unit G33 displays a device image G331 in which the operation device 2 is viewed from above. Specifically, the operation display unit G33 displays a plurality of operators that the operation device 2 has at the same positions as the respective positions of the plurality of operators on the operation device 2. The second image generation unit 369 draws the content of the DJ operation on the operation device 2 reproduced by the reproduction unit 372 and displays it on the operation display unit G33. That is, the second image generation unit 369 visualizes the DJ operation reproduced by the reproduction unit 372 on the operation display unit G33.

[0070] For example, the second image generation unit 369 generates an image in which the position of a partial image indicating a control of the deck indicated by the deck number DN, among the controls indicated by the control type SC in the record RC of the operation log information LG corresponding to the elapsed time since the start of the operation reproduction process, is moved to the control position SP. In other words, the second image generation unit 369 moves the position of a partial image indicating a control corresponding to the control type SC and deck number DN, among the controls included in the device image G331, to a position corresponding to the control position SP according to the elapsed time. As a specific example, if the control indicated by the control type SC in the acquired record RC is a channel fader and the deck number DN indicated by the deck number DN of the record RC is "2," the second image generation unit 369 generates an operation reproduction screen G3 in which the partial image indicating the operation knob of the channel fader of the second deck is moved to a position corresponding to the control position SP of the record RC.

[0071] Furthermore, for example, the second image generation unit 369 generates an image indicating the movement direction or rotation direction of a controller of the deck indicated by the deck number DN, among the controllers indicated by the controller type SC in the record RC of the operation log information LG corresponding to the elapsed time since the operation reproduction process was started, using a graphic FG such as an arrow based on the controller position SP. In other words, the second image generation unit 369 displays a graphic FG such as an arrow indicating the operation direction and rotation direction of the controller near a controller corresponding to the controller type SC and deck number DN among the controllers in the device image G331, based on the controller position SP corresponding to the elapsed time. As a specific example, if the controller indicated by the controller type SC in the acquired record RC is a jog dial and the deck number DN indicated by the deck number DN of the record RC is "1," the second image generation unit 369 generates an operation reproduction screen G3 in which a graphic FG, which is an arrow indicating the operation direction of the rotary controller, is superimposed on a partial image indicating the jog dial of the first deck, based on the controller position SP of the record RC.

[0072] Furthermore, for example, the second image generation unit 369 highlights or displays a graphic such as a circle on the operator of the deck indicated by the deck number DN among the operators indicated by the operator type SC in the record RC of the operation log information LG corresponding to the elapsed time since the start of the operation reproduction process. That is, the second image generation unit 369 highlights or displays a graphic such as a circle on the operator corresponding to the operator type SC and deck number DN among the operators in the device image G331 in accordance with the elapsed time, thereby distinguishing the operator from the other operators. As a specific example, if the operator indicated by the operator type SC in the acquired record RC is the first sampler pad and the deck number DN indicated by the deck number DN in the record RC is "2," the second image generation unit 369 generates an operation reproduction screen G3 in which a partial image indicating the first sampler pad of the second deck is highlighted, or an operation reproduction screen G3 in which a graphic FG such as a circle is superimposed on the partial image indicating the first sampler pad.

[0073] The operation display unit G33 can also display information related to the reproduced DJ operation. For example, the second image generation unit 369 displays a speech bubble SB containing information related to the DJ operation on the operation display unit G33. Examples of information related to the DJ operation include the amount of operation of the controller, the value of a parameter changed by the reproduced operation, information identifying the DJ device used when the operation reproduction information was recorded, and the settings of the DJ application when the operation reproduction information was recorded. Examples of the amount of operation of the controller include the amount of movement of the controller, a numerical value indicating the position of the controller, and the rotation angle of the controller. In the example of FIG. 7 , the speech bubble SB indicates the amount of movement of the crossfader in the direction of the arrow. If the operation of a controller that sets the value of a specific parameter is recorded in the operation log information LG, the parameter value may be the value of the parameter changed in response to the operation of the controller. For example, if the operation log information LG records a press of the loop button, the parameter value is the "number of bars of the loop" set by pressing the loop button. Displays including this information are sequentially set in the operation display section G33 by the second image generation section 369. As a result, the contents of the reproduced DJ operation and information related to the reproduced DJ operation are set in the operation display section G33, and the operation reproduction screen G3 including the operation display section G33 is sequentially displayed.

[0074] The menu button G34 is a button that displays a menu screen (not shown). The menu screen includes, for example, a selection item for selecting a device image G331 of an operation device displayed on the operation display unit G33. When reproducing another user's DJ operation based on the acquired operation reproduction information, for example, a device image G331 that resembles the model of the operation device 2 used by the user is displayed, making it easier to understand the other user's DJ operation. For this reason, the second image generation unit 369 displays the device image G331 selected by the user on the operation display unit G33 and visualizes the operations performed on the controls included in the device image G331 based on the reproduced DJ operation. This makes it easier for the user to understand the content of the DJ operation performed by the other user.

[0075] The speed adjustment button G35, reproduction start button G36, and reproduction position display bar G37 allow the content of the DJ operations displayed on the operation display unit G33 to be manipulated like a video. The speed adjustment button G35 is a button that synchronizes and increases or decreases the playback speed of the target music by the second music playback unit 371 and the reproduction speed of the DJ operations by the reproduction unit 372. In the example of the operation reproduction screen G3 shown in FIG. 7 , the speed adjustment button G35 includes buttons for adjusting the speed to 0.5x, 0.75x, 1x, 1.25x, and 1.5x. The reproduction unit 372 reproduces the DJ operations at a speed corresponding to the input button and causes the second music playback unit 371 to play the target music at the speed corresponding to the input button. When the reproduction speed of the DJ operations is increased or decreased using the speed adjustment button G35, the display update speed of the operation display unit G33 also increases or decreases in synchronization with the reproduction speed of the DJ operations. In this way, the reproduction unit 372 synchronizes and increases or decreases the speed at which the DJ operation is performed, which is reproduced based on the operation reproduction information, and the playback speed of the target song played by the second music playback unit 371. Note that the reproduction speed of the DJ operation and the playback speed of the target song may be configured so that the user can set any desired speed.

[0076] The reproduction start button G36 is a button that starts the operation reproduction process of the DJ operation based on the operation reproduction information. When the reproduction start button G36 is pressed, the operation reproduction process starts, and the second music playback unit 371 starts playing the target music, and the reproduction unit 372 starts reproducing the DJ operation based on the operation reproduction information. Note that when the reproduction start button G36 is pressed again, the operation reproduction process is paused. The reproduction position display bar G37 is a bar provided with a slider G371 that indicates the current reproduction position of the DJ operation based on the operation reproduction information. The user can adjust the reproduction position by sliding the slider G371 along the reproduction position display bar G37. Based on the position of the slider G371, the reproduction unit 372 changes the reproduction position of the DJ operation and also changes the playback position of the target music by the second music playback unit 371. Note that the operation reproduction screen G3 may also be provided with a button that moves the reproduction position back a predetermined time from the current reproduction position and a button that moves the reproduction position forward a predetermined time from the current reproduction position.

[0077] [Effects of the Embodiment] The operation reproduction system 1A according to the present embodiment described above has the following effects. The operation reproduction system 1A includes an operation device 2, an information processing device 3, and a display device 4. The information processing device 3 includes a reproduction information acquisition unit 366, a correction unit 370, a second music playback unit 371, and a reproduction unit 372. The reproduction information acquisition unit 366 corresponds to the control information acquisition unit and acquires multiple pieces of sound control information included in the operation log information of the operation reproduction information. That is, the reproduction information acquisition unit 366 acquires multiple pieces of sound control information that record multiple sound control operations for audio data. The correction unit 370 can correct each of the acquired multiple pieces of sound control information. The second music playback unit 371 corresponds to the playback unit and plays the target music. The reproduction unit 372 reproduces multiple sound control operations based on the operation reproduction information including the corrected multiple pieces of sound control information for the music played by the second music playback unit 371.

[0078] With this configuration, the reproduction unit 372 reproduces multiple sound control operations for the music played by the second music playback unit 371 based on multiple pieces of sound control information. Therefore, by acquiring multiple pieces of sound control information, the music can be played in the same way as when the sound control operations were recorded. Therefore, for example, an automatic performance of a music piece involving multiple sound control operations can be performed. Furthermore, even if an incorrect operation is performed when the sound control operations are recorded, the correction unit 370 can correct the incorrect operation for each of the multiple sound control operations. Therefore, music can be played with more appropriate sound control operations.

[0079] In the information processing device 3, the modifying unit 370 modifies the selected piece of sound control information to be modified from among the plurality of pieces of sound control information. With this configuration, the modifying unit 370 modifies the selected piece of sound control information to be modified from among the plurality of pieces of sound control information, so that each piece of sound control information can be modified individually.

[0080] The information processing device 3 includes a second image generating unit 369 that generates an operation display image P1 that displays the content of each of the multiple pieces of sound control information. The second image generating unit 369 corresponds to an image generating unit. The modifying unit 370 modifies the sound control information selected in the operation display image P1 from among the multiple pieces of sound control information as the sound control information to be modified. With this configuration, the user can confirm the content of each piece of sound control information, i.e., the content of each sound control operation, by checking the operation display image P1 generated by the second image generating unit 369. The modifying unit 370 then modifies the sound control information selected by the user in the operation display image P1 as the sound control information to be modified, allowing the user to easily select the sound control information to be modified. This makes it easy to check the content of the sound control information and select the sound control information to be modified.

[0081] In the information processing device 3, the plurality of acoustic control operations are operations on a plurality of different operators. With this configuration, the modifying unit 370 modifies the acoustic control information, thereby making it possible to modify the operations on the operators when the plurality of acoustic control information was recorded.

[0082] In the information processing device 3, one of the plurality of pieces of acoustic control information is first channel fader operation information indicating an operation on a first channel fader that adjusts the volume level of a first music piece, and another of the plurality of pieces of acoustic control information is second channel fader operation information indicating an operation on a second channel fader that adjusts the volume level of a second music piece. The correction unit 370 corrects each piece of channel fader operation information. With this configuration, the correction unit 370 can correct the recorded operations on each channel fader.

[0083] In the information processing device 3, the operation display image P1 includes a first fader state display section P14B that indicates changes in the volume level of the first channel using a line P146 that follows a time series, and a second fader state display section P15B that indicates changes in the volume level of the second channel using a line P156 that follows a time series. The lines P146 and P156 correspond to volume level display lines, and the fader state display sections P14B and P15B correspond to channel fader state display sections. The correction unit 370 corrects the first channel fader operation information in accordance with the line P146 adjusted by the user, and corrects the second channel fader operation information in accordance with the line P156 adjusted by the user. With this configuration, the correction unit 370 corrects the first channel fader operation information by the user adjusting the line P146 in the first fader state display section P14B of the operation display image P1. Furthermore, the user adjusts the line P156 in the second fader state display portion P15B of the operation display image P1, and the modifying portion 370 modifies the second channel fader operation information, thereby allowing the user to easily modify the operation information of each channel fader.

[0084] In the information processing device 3, one of the plurality of pieces of acoustic control information is crossfader operation information indicating an operation on a crossfader that adjusts the volume balance between a first channel and a second channel that can play a target song. The correction unit 370 corrects the crossfader operation information. With this configuration, the recorded crossfader operation can be corrected by the correction unit 370.

[0085] In the information processing device 3, the operation display image P1 includes a crossfader state display section P16 that indicates changes in the volume balance between the first channel and the second channel over time using a line P161. The line P161 corresponds to a volume balance display line. The correction unit 370 corrects the crossfader operation information in accordance with the line P161 adjusted by the user. With this configuration, the user adjusts the line P161 in the crossfader state display section P16 of the operation display image P1, and the correction unit 370 corrects the crossfader operation information. This allows the user to easily correct the crossfader operation information.

[0086] In the information processing device 3, one of the plurality of pieces of acoustic control information is jog dial operation information indicating an operation on a jog dial that adjusts the playback direction and playback speed of a target song. The correction unit 370 corrects the jog dial operation information. With this configuration, the correction unit 370 can correct the recorded operation on the jog dial.

[0087] In the information processing device 3, the operation display image P1 includes a jog operation correction image P2, which includes a jog dial state display portion P23 that indicates the operation state of the jog dial. The correction portion 370 corrects the jog dial operation information in accordance with the operation state of the jog dial adjusted by the user in the jog dial state display portion P23. With this configuration, the user adjusts the operation state of the jog dial in the jog dial state display portion P23 of the operation display image P1, and the correction portion 370 corrects the jog dial operation information. This allows the user to easily correct the jog dial operation information.

[0088] In the information processing device 3, one of the multiple pieces of audio control information is effect setting information indicating an operation on a control that sets an effect to be applied to the played target song. The modifying unit 370 modifies the effect setting information, which includes at least one of the effect type, the effect on-duration, the effect application rate, and the effect target channel. In this embodiment, the effect setting information includes the effect type, the effect on-duration, the effect application rate, and the effect target channel. With this configuration, the recorded effect setting operation can be modified by the modifying unit 370.

[0089] In the information processing device 3, the operation display image P1 includes an effect status display section P17 that indicates the setting status of an effect. The modifying section 370 modifies the effect setting information in accordance with the setting status of the effect adjusted by the user. With this configuration, the user adjusts the setting status of the effect in the effect status display section P17 of the operation display image P1, and the modifying section 370 modifies the effect setting information. This allows the user to easily modify the effect setting information.

[0090] The information processing device 3 includes a conversion unit 373 that converts operation replay information including multiple pieces of audio control information into a format that can be used by other devices. With this configuration, by acquiring the operation replay information, the other devices can, for example, play music based on the operation replay information. This improves the convenience of the operation replay information.

[0091] [Modifications of the Embodiments] The present invention is not limited to the above-described embodiments, and modifications and improvements within the scope of achieving the object of the present invention are included in the present invention. In the above-described embodiments, each of the plurality of pieces of audio control information included in the operation reproduction information is an operation for a corresponding operator and indicates an audio control operation applied to a song. The plurality of pieces of audio control information includes channel fader operation information, crossfader operation information, jog dial operation information, and effect setting information. However, this is not limited to this, and the plurality of pieces of audio control information does not have to include at least one of the channel fader operation information, crossfader operation information, jog dial operation information, and effect setting information. On the other hand, the audio control information may include control information indicating an operation or setting operation for another operator. For example, one of the plurality of pieces of audio control information may be information indicating the operation state of a tempo slider that adjusts the tempo of a song.

[0092] In the above embodiment, the operation display image P1 on which the user modifies and edits the sound control information is an image having the layout shown in Fig. 7. However, the layout and content of the operation display image P1 are not limited to this. For example, the operation display image P1 may further include a display section showing the operation content of other controls.

[0093] In the above embodiment, the modifying unit 370 modifies or edits the corresponding sound control information in response to a user's operation on the operation display image P1. However, the modification is not limited to this, and the modifying unit 370 may modify or edit the sound control information by replacing a numerical value indicated by the sound control information included in the operation log information of the operation reproduction information with a numerical value input by the user.

[0094] In the above embodiment, the conversion unit 373 converts the operation reproducibility information including the recorded and corrected acoustic control information into operation reproducibility information in a format usable by other devices. However, this is not limiting, and the conversion unit 373 may be omitted. Alternatively, the reproducibility information generation unit 364 may generate operation reproducibility information in a format usable by other devices.

[0095] [Summary of the Invention] The summary of the invention is as follows: [1] An information processing device comprising: a control information acquisition unit that acquires a plurality of pieces of audio control information that record a plurality of audio control operations for a piece of music; a modification unit that can modify each of the acquired pieces of audio control information; a playback unit that plays the piece of music; and a reproduction unit that reproduces, for the played piece of music, the plurality of audio control operations based on the modified pieces of audio control information.

[0096] According to this configuration, the reproduction unit reproduces multiple sound control operations for a piece of music played by the reproduction unit based on multiple pieces of sound control information. Therefore, by acquiring multiple pieces of sound control information, the music can be played in the same way as when the sound control operations were recorded. Therefore, for example, an automatic performance of a piece of music involving multiple sound control operations can be performed. Furthermore, even if an incorrect operation is performed when the sound control operations are recorded, the correction unit can correct the incorrect operation for each of the multiple sound control operations. Therefore, music can be played with more appropriate sound control operations.

[0097] [2] The information processing device according to [1], wherein the correction unit corrects selected acoustic control information to be corrected from the plurality of pieces of acoustic control information. With this configuration, the correction unit corrects selected acoustic control information to be corrected from the plurality of pieces of acoustic control information, so that each of the plurality of pieces of acoustic control information can be corrected individually.

[0098] [3] The information processing device according to [2] further comprises an image generation unit that generates an operation display image showing the content of each of the plurality of pieces of sound control information, and the correction unit corrects the sound control information selected in the operation display image from among the plurality of pieces of sound control information as the sound control information to be corrected. With this configuration, the content of each piece of sound control information, i.e., the content of each sound control operation, can be confirmed by checking the operation display image generated by the image generation unit. The correction unit corrects the sound control information selected in the operation display image as the sound control information to be corrected, allowing the user to easily select the sound control information to be corrected. This makes it easy to check the content of the sound control information and select the sound control information to be corrected.

[0099] [4] The information processing device according to [3], wherein the plurality of acoustic control operations are operations for a plurality of mutually different operators. With this configuration, the correction unit corrects the acoustic control information, thereby correcting the operations for each operator when the plurality of acoustic control information is recorded.

[0100] [5] The information processing device according to [4], wherein one of the plurality of pieces of acoustic control information is channel fader operation information indicating an operation on a channel fader that adjusts the volume level of the music, and the correction unit corrects the channel fader operation information. With this configuration, the recorded operation on the channel fader can be corrected by the correction unit.

[0101] [6] The information processing device according to [5], wherein the operation display image includes a channel fader state display section that indicates changes in the volume level using a volume level display line along a time series, and the correction section corrects the channel fader operation information in accordance with the volume level display line adjusted by the user. With this configuration, the correction section corrects the channel fader operation information when the user adjusts the volume level display line in the channel fader state display section of the operation display image. This allows the user to easily correct the channel fader operation information.

[0102] [7] The information processing device according to any one of [4] to [6], wherein one of the plurality of pieces of acoustic control information is crossfader operation information indicating an operation on a crossfader that adjusts the volume balance between a first channel and a second channel that can play the music, and the correction unit corrects the crossfader operation information. With this configuration, the recorded crossfader operation can be corrected by the correction unit.

[0103] [8] The information processing device according to [7], wherein the operation display image includes a crossfader state display section that indicates changes in the volume balance using a volume balance display line along a time series, and the correction section corrects the crossfader operation information in accordance with the volume balance display line adjusted by the user. With this configuration, the correction section corrects the crossfader operation information when the user adjusts the volume balance display line in the crossfader state display section of the operation display image. This allows the user to easily correct the crossfader operation information.

[0104] [9] The information processing device according to any one of [4] to [8], wherein one of the plurality of pieces of acoustic control information is jog dial operation information indicating an operation on a jog dial for adjusting the playback direction and playback speed of the music piece, and the correction unit corrects the jog dial operation information. With this configuration, the recorded operation on the jog dial can be corrected by the correction unit.

[0105]

[10] The information processing device according to [9], wherein the operation display image includes a jog dial state display section that indicates the operation state of the jog dial, and the correction section corrects the jog dial operation information according to the operation state of the jog dial adjusted by the user. With this configuration, the correction section corrects the jog dial operation information when the user adjusts the operation state of the jog dial in the jog dial state display section of the operation display image. This allows the user to easily correct the jog dial operation information.

[0106]

[11] The information processing device according to any one of [4] to

[10] , wherein one of the plurality of pieces of acoustic control information is effect setting information indicating an operation on an operator that sets an effect to be applied to the played music piece, and the modifying unit modifies the effect setting information including at least one of the type of the effect, the on-duration of the effect, the application rate of the effect, and the target channel of the effect. With this configuration, the recorded effect setting operation can be modified by the modifying unit.

[0107]

[12] The information processing device according to

[11] , wherein the operation display image includes an effect status display section that indicates the setting status of the effect, and the modifying section modifies the effect setting information in accordance with the setting status of the effect adjusted by the user. With this configuration, the modifying section modifies the effect setting information when the user adjusts the setting status of the effect in the effect status display section of the operation display image. This allows the user to easily modify the effect setting information.

[0108]

[13] The information processing device according to any one of [1] to

[12] , further comprising a conversion unit that converts the plurality of pieces of audio control information into a format that can be used by other devices. With this configuration, by acquiring the plurality of pieces of audio control information by other devices, for example, the other devices can play music based on the plurality of pieces of audio control information. This improves the convenience of the audio control information.

[0109] 3...information processing device, 369...second image generation unit (image generation unit), 370...modification unit, 371...second music playback unit (playback unit), 372...reproduction unit, 373...conversion unit, P1...operation display image, P14B...first fader status display unit (channel fader status display unit), P146...line (volume level display line), P15B...second fader status display unit (channel fader status display unit), P156...line (volume level display line), P16...cross fader status display unit, P161...line (volume balance display line), P17...effect status display unit, P23...jog dial status display unit.

Claims

1. An information processing device comprising: a control information acquisition unit that acquires a plurality of pieces of audio control information that record a plurality of audio control operations for a piece of music; a modification unit that can modify each of the acquired plurality of pieces of audio control information; a playback unit that plays the piece of music; and a reproduction unit that reproduces, for the played piece of music, the plurality of audio control operations based on the modified plurality of pieces of audio control information.

2. An information processing apparatus according to claim 1, wherein the correction section corrects a piece of sound control information to be selected from the plurality of pieces of sound control information.

3. An information processing device according to claim 2, further comprising an image generating unit which generates an operation display image showing the contents of each of the plurality of pieces of sound control information, and wherein the correction unit corrects, from among the plurality of pieces of sound control information, sound control information selected in the operation display image as the sound control information to be corrected.

4. An information processing apparatus according to claim 3, wherein each of the plurality of sound control operations is an operation on a plurality of mutually different operators.

5. An information processing device according to claim 4, wherein one of the plurality of pieces of acoustic control information is channel fader operation information indicating an operation on a channel fader that adjusts the volume level of the music, and the correction unit corrects the channel fader operation information.

6. An information processing device according to claim 5, wherein the operation display image includes a channel fader state display section which shows the change in the volume level by a volume level display line along a time series, and the correction section corrects the channel fader operation information in accordance with the volume level display line adjusted by a user.

7. An information processing device as claimed in any one of claims 4 to 6, wherein one of the plurality of pieces of acoustic control information is crossfader operation information indicating an operation on a crossfader that adjusts the volume balance between a first channel and a second channel capable of playing the music, and the correction unit corrects the crossfader operation information.

8. An information processing device according to claim 7, wherein the operation display image includes a crossfader status display section which shows the change in the volume balance by a volume balance display line along a time series, and the correction section corrects the crossfader operation information in accordance with the volume balance display line adjusted by a user.

9. An information processing device according to any one of claims 4 to 8, wherein one of the plurality of pieces of acoustic control information is jog dial operation information indicating an operation on a jog dial for adjusting the playback direction and playback speed of the music, and the correction unit corrects the jog dial operation information.

10. An information processing device according to claim 9, wherein the operation display image includes a jog dial status display section that indicates the operation status of the jog dial, and the correction section corrects the jog dial operation information in accordance with the operation status of the jog dial adjusted by a user.

11. An information processing device as claimed in any one of claims 4 to 10, wherein one of the plurality of pieces of acoustic control information is effect setting information indicating an operation on an operator for setting an effect to be applied to the played music piece, and the modification unit modifies the effect setting information including at least one of the type of effect, the on period of the effect, the application rate of the effect, and the target channel of the effect.

12. An information processing device according to claim 11, wherein the operation display image includes an effect status display section that indicates the setting status of the effect, and the modification section modifies the effect setting information in accordance with the setting status of the effect adjusted by the user.

13. An information processing device according to any one of claims 1 to 12, further comprising a conversion unit that converts the plurality of pieces of acoustic control information into a format that can be used by other devices.

Images