Information processing device and music playback program

Abstract

Before exporting the edited song as a music file, shoot a video using the edited song. [Solution] The information processing unit includes a display control unit that displays an operation screen on a display unit that includes a music editing screen for editing music and a playback control screen for controlling the playback conditions of music; an editing unit that edits music in response to operations on the music editing screen; a playback unit that plays the music edited by the music editing screen according to the playback conditions on the playback control screen; and a video recording unit that plays the music using the playback unit while recording a video that includes the music as audio.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus and a music playback program.

Background Art

[0002] Conventionally, while playing music and outputting sound through speakers, headphones, etc., video shooting of dancing in accordance with the music has been performed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, for example, users related to dancing such as dancers may edit the music so as to be suitable for dancing, such as applying effects to the music or changing the length of the music. Also, when practicing dancing in accordance with the music, users related to dancing may want to change the tempo of the music, play from a specific part of the music, or repeatedly play a specific part. Further, users related to dancing may shoot a video of dancing in accordance with the music, check the shot video, and repeatedly review the choreography of the dance or edit the music.

[0005] Conventionally, music editing software and video shooting software are separate software. For this reason, when a user related to dancing shoots a dance video in accordance with the music edited by the editing software, for example, the edited music is written out and saved as music data in a predetermined music file format by the editing software, and the video shooting is performed by the shooting software while playing the saved music data, which is troublesome.

[0006] One aspect of this project is to provide an information processing device and music playback program that allows users to shoot video with an edited song before exporting the edited song as a music file. [Means for solving the problem]

[0007] The information processing device relating to one aspect includes a display control unit that displays an operation screen on a display unit that includes a music editing screen for editing music and a playback control screen for controlling the playback conditions of music; an editing unit that edits music in response to operations on the music editing screen; a playback unit that plays the music edited by the music editing screen according to the playback conditions on the playback control screen; and a video recording unit that plays the music using the playback unit while recording a video that includes the music as audio. [Effects of the Invention]

[0008] According to one embodiment, video recording can be performed using the edited music before exporting the edited music as a music file. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 shows an example of the functional configuration of an information processing device according to the embodiment. [Figure 2A] Figure 2A shows an example of a music editing screen according to the embodiment. [Figure 2B] Figure 2B shows an example of a music editing screen according to the embodiment. [Figure 3A] Figure 3A shows an example of the advanced mode screen according to this embodiment. [Figure 3B] Figure 3B shows an example of the advanced mode screen according to this embodiment. [Figure 4A] Figure 4A shows an example of the basic mode screen according to the embodiment. [Figure 4B] Figure 4B shows an example of the basic mode screen according to the embodiment. [Figure 4C] Figure 4C shows an example of the basic mode screen according to the embodiment. [Figure 5] Figure 5 shows an example of a video recording screen according to the embodiment. [Figure 6] Figure 6 shows an example of the process a user goes through when editing music and shooting a video. [Figure 7] Figure 7 is a flowchart showing an example of the procedure for a control process including music playback processing according to the embodiment. [Figure 8] Figure 8 is a block diagram showing an example of the functional configuration of an information processing device according to the embodiment. [Figure 9] Figure 9 shows an example of a hardware configuration. [Modes for carrying out the invention]

[0010] The following describes the embodiments for implementing the information processing device and music playback program related to this disclosure (hereinafter referred to as "Embodiments") with reference to the attached drawings. It should be noted that these embodiments represent only one example or aspect, and the following description does not limit the configuration, operation, function, properties, characteristics, methods, and applications related to this disclosure.

[0011] <Embodiment 1> <Configuration of the information processing device 10> Figure 1 is a diagram showing an example of the functional configuration of an information processing device 10 according to an embodiment. For example, the information processing device 10 is a user terminal, such as a smartphone or other mobile terminal, on which music editing and video recording software is installed. The information processing device 10 has an editing function for editing music, a playback control function for playing music, and a video recording function for recording videos. Figure 1 schematically shows the blocks related to the editing function, playback control function, and video recording function of the information processing device 10. As shown in Figure 1, the information processing device 10 has a display unit 11, an input unit 12, a communication unit 13, a camera unit 14, a storage unit 15, and a control unit 16. Note that Figure 1 only shows a selection of the functional units related to the music playback function described above, and the information processing device 10 may have functional units other than those shown.

[0012] The display unit 11 is a functional unit that displays various types of information. As an example, the display unit 11 is realized by a liquid crystal display, an organic EL (Electro Luminescence) display, or the like. For example, the display unit 11 displays various operation screens related to an editing function, a playback control function, and a video shooting function.

[0013] The input unit 12 is a functional unit that inputs various operations. When the information processing apparatus 10 is realized by a mobile terminal, the display unit 11 and the input unit 12 are integrated to be realized as a touch panel. For example, the input unit 12 receives various inputs for the various operation screens displayed on the display unit 11.

[0014] The communication unit 13 is a functional unit that is connected to a network (not shown) either wired or wirelessly and transmits and receives information to and from other devices via the network. As an example, the communication unit 13 is realized by a communication module, a NIC (Network Interface Card), or the like.

[0015] The camera unit 14 is a functional unit that captures images. For example, the camera unit 14 incorporates an image sensor, captures an image with the image sensor, and outputs image data of the captured image.

[0016] The storage unit 15 is a functional unit that stores various types of data. As one aspect, the storage unit 15 is realized by a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage. For example, the storage unit 15 stores project data 15a.

[0017] The information processing apparatus 10 can edit music through an editing function and manages music as projects. The project data 15a is data that stores music edited by the editing function.

[0018] The control unit 16 is a functional unit that performs overall control of the information processing device 10. For example, the control unit 16 can be implemented by a hardware processor. As shown in Figure 1, the control unit 16 has a display control unit 16a, an editing unit 16b, a playback unit 16c, a video recording unit 16d, and a generation unit 16e. It should be noted that the control unit 16 may be implemented using hardwired logic or the like.

[0019] The display control unit 16a controls the display of various types of information on the display unit 11. For example, the display control unit 16a displays an operation screen, including a music editing screen and a playback control screen, on the display unit 11.

[0020] Here, we will describe an example of the music editing screen and playback control screen displayed on the display unit 11. First, we will describe an example of the music editing screen. Figures 2A and 2B show an example of the music editing screen 50 according to the embodiment.

[0021] The music editing screen 50 includes a switching control 51, a footer 60, an editing operation section 70, and a track display section 80.

[0022] The toggle control 51 displays and allows selection of "Player" and "Edit". When "Player" is selected, the playback control screen described later is displayed, and when "Edit" is selected, the music editing screen 50 is displayed.

[0023] The footer 60 includes an add track button 61, an undo button 62, a play / stop button 63, a redo button 64, and a fade button 65.

[0024] The Add Track button 61 is a button that instructs the user to add track 81 to the track display unit 80. The Undo button 62 is a button that instructs the user to cancel the operation performed in the previous step and return to the state before the operation performed in the previous step. The Play / Stop button 63 is a button that instructs the user to play and stop the music being edited. The Play / Stop button 63 displays "Play" when the music is not playing and displays "Stop" when the music is playing. The Redo button 64 is a button that instructs the user to redo the operation that was canceled by the Undo button 62. The Fade button 65 is a button that instructs the user to add a fade effect to the music.

[0025] The editing operation unit 70 includes a copy button 71, a paste button 72, a split button 73, and a delete button 74. The editing operation unit 70 is displayed when a song displayed on track 81 is touched.

[0026] The copy button 71 is a button that instructs the user to copy the song. The paste button 72 is a button that instructs the user to paste the song that was copied by the copy button 71. The split button 73 is a button that instructs the user to split the song at the specified position. The delete button 74 is a button that instructs the user to delete the specified song.

[0027] The music editing screen 50 manages the sound sources used in the music using track 81. Track 81 is added to the track display section 80 by touching the add track button 61. Track 81 is then displayed on the track display section 80.

[0028] Track 81 contains one or more songs to be used as sound sources. For example, a song specified by the user is downloaded and registered to track 81 by accessing a web service that provides music via a network (not shown). Track 81 displays an audio waveform showing the changes in the audio of the song along the timeline of the registered song.

[0029] The songs registered on each track 81 can be edited using the copy button 71, paste button 72, split button 73, and delete button 74 on the editing control unit 70. For example, when a song registered on track 81 is touched, the editing control unit 70 can perform copy, paste, split, and delete operations, and these operations can be combined to perform editing. For example, the length of song c registered on track 81 can be changed by performing editing that cuts out and extracts a portion of it.

[0030] Furthermore, each track 81 allows for editing by adding effects to the music. For example, when a song registered on track 81 is touched, the fade button 65 allows for the addition of fade-in and fade-out effects. Fade-in gradually increases the volume, and fade-out gradually decreases the volume. The fade effect can be used when (1) you want to gradually increase the volume, (2) you want to seamlessly connect two songs, or (3) you want to gradually end a song. For example, (1) to gradually increase the volume, set a fade-in. (2) to seamlessly connect two songs, set a fade-out on the preceding song and a fade-in on the following song. (3) to gradually end a song, set a fade-out. When an effect is added to a song, a mark indicating the added effect will be displayed on track 81. In addition, each track 81 allows for editing to adjust the volume balance, tone, etc. For example, with track 81, after a touch operation and a predetermined operation is performed, a sub-screen for specifying the maximum volume is displayed, allowing the maximum volume to be adjusted.

[0031] The music editing screen 50 allows users to register multiple songs to one or more tracks 81 and edit each registered song individually to combine them. Furthermore, the music editing screen 50 allows users to mix multiple songs by registering each song to multiple tracks 81.

[0032] The music editing screen 50 shown in Figure 2A displays two tracks 81a and 81b. Track 81a has song A registered to it. Track 81b has a portion of song B followed by a portion of song C registered to it.

[0033] Figure 2B shows an example of the result of editing the music tracks 81a and 81b shown in Figure 2A and adding effects. In Figure 2B, a portion of song A from track 81a is extracted, song B from track 81b is deleted, and the extracted portion of song A from track 81a is joined with song C from track 81b. Also in Figure 2B, an effect has been added to the end of song A from track 81a that allows the listener to select a start and end point for a fade-out. On song A from track 81a, a triangular mark 82 is displayed corresponding to the positions of the start and end points of the fade-out. Additionally, an effect has been added to song C from track 81b that allows the listener to select a start and end point for a fade-in. On song C from track 81b, a triangular mark 83 is displayed corresponding to the positions of the start and end points of the fade-in. In marks 82 and 83, the hypotenuse of the triangle represents the change in volume level.

[0034] The information processing device 10 can play the music edited on the music editing screen 50 on the playback control screen by switching the switching control 51 of the music editing screen 50 to "player".

[0035] Next, an example of a playback control screen will be described. The information processing device 10 provides an advanced mode screen and a basic mode screen as playback control screens, and displays either the advanced mode screen or the basic mode screen on the display unit 11 according to a switching instruction.

[0036] Figures 3A and 3B show an example of the advanced mode screen 100 according to the embodiment. The advanced mode screen 100 includes a switching control 101, a header 110, a footer 120, a waveform display unit 130, a tempo operation unit 140, a marker operation unit 150, and a loop operation unit 160. The advanced mode screen 100 can be switched to the basic mode screen from the menu button 102.

[0037] The toggle control 101 displays and allows selection of "Player" and "Edit". When "Player" is selected, the advanced mode screen 100 is displayed, and when "Edit" is selected, the music editing screen 50 is displayed.

[0038] Header 110 displays song information, such as the title of the song being played.

[0039] The footer 120 includes a play / stop button 121, a start button 122, a forward button 123, a back button 124, and a record button 125.

[0040] The play / stop button 121 is used to instruct the playback and stopping of a song. When the song is not playing, "Play" is displayed on the play / stop button 121, and "Stop" is displayed while the song is playing.

[0041] The start button 122 is a button that instructs the playback position to return to the beginning of the song. The forward button 123 is a button that instructs the playback position of the song to advance to a predetermined number of seconds later. The forward button 123 is a button that instructs the playback position of the song to return to a predetermined number of seconds earlier. The predetermined number of seconds can be specified from a settings screen (not shown). In this embodiment, the case where the predetermined number of seconds is set to 5 seconds is shown.

[0042] The record button 125 is used to initiate video recording. Video recording will be explained later.

[0043] The waveform display unit 130 displays an audio waveform that shows the changes in the sound of the entire song along the song's timeline. The waveform display unit 130 also displays a seek bar 131 on the timeline indicating the current playback position. Furthermore, the waveform display unit 130 displays the playback time at the current playback position and the remaining time. The seek bar 131 is slide-operable. The waveform display unit 130 accepts changes to the song's playback position when the seek bar 131 is slid.

[0044] The tempo control unit 140 displays the BPM (Beats Per Minute) value representing the tempo of the song. The tempo control unit 140 also has a tempo up button 141 for instructing the player to increase the tempo of the song, a tempo down button 142 for instructing the player to decrease the tempo of the song, and a slider bar 143 that accepts changes to the song's tempo by sliding it. The tempo control unit 140 accepts changes to the playback speed of the song by changing the tempo of the song using the tempo up button 141, the tempo down button 142, and the slider bar 143.

[0045] The marker operation unit 150 displays four markers 151, numbered "1" through "4". When a marker 151 is touched during music playback, the playback position at the time of the touch is registered to the marker 151. When a marker 151 with a registered playback position is touched, the music starts playing from the playback position registered to the touched marker 151. In the example in Figure 3A, marker "1" 151 has no registered playback position, while markers "2" through "4" 151 have registered playback positions. The timeline on the waveform display unit 130 displays the numbers "2" through "4" indicating the playback positions of markers "2" through "4". The marker operation unit 150 accepts registration of the music's playback position and specification of the playback position when a marker 151 is touched. The marker operation unit 150 also displays a trash can button 152. After the trash can button 152 is touched, the playback position registered in marker 151 is deleted when marker 151 is touched.

[0046] The loop operation unit 160 has a start position button 161 for specifying the start position of loop playback, an end position button 162 for specifying the end position of loop playback, and a loop button 163 for specifying loop playback. When the start position button 161 is touched during music playback, the playback position at the time of the touch is registered as the start position in the start position button 161. Also, when the end position button 162 is touched during music playback after the start position, the playback position at the time of the touch is registered in the end position button 162. The loop button 163 switches between selected and unselected states by touch. When the loop button 163 is selected and music is played, loop playback occurs between the start position registered in the start position button 161 and the end position registered in the end position button 162. In the example in Figure 3A, the timeline of the waveform display unit 130 displays "A" indicating the start position of the loop and "B" indicating the end position of loop playback. The loop operation unit 160 accepts registration of the start and end positions for loop playback and specification of loop playback when the start position button 161, end position button 162, and loop button 163 are touched. The loop operation unit 160 displays a trash can button 164. After the trash can button 164 is touched, the loop operation unit 160 deletes the playback positions registered to the start position button 161 and end position button 162 when the start position button 161 and end position button 162 are touched.

[0047] Furthermore, the waveform display unit 130 is provided with a toggle button 132 for instructing the display of the audio waveform in an enlarged view. When the toggle button 132 is selected, the audio waveform is displayed in an enlarged view on the waveform display unit 130.

[0048] Figure 3B shows an example of the result when the toggle button 132 shown in Figure 3A is selected. As shown in Figure 3B, the waveform display unit 130 displays the audio waveform of the song for a predetermined time before and after the current playback position. The waveform display unit 130 also plots a beat grid 133 corresponding to the beat position of the song. The waveform display unit 130 is provided with a toggle button 134 to indicate the display of the entire audio waveform. When the toggle button 134 is selected, the display returns to Figure 3A, and the waveform display unit 130 displays the audio waveform of the entire song.

[0049] The advanced mode screen 100 has a menu button 102 with three dots arranged vertically to the right of the switching control 101, and when the menu button 102 is selected, menu 170 is displayed. Menu 180 has a switching button 181 that instructs switching to the basic mode screen and an export button 182 that instructs exporting the music as an audio file. When the basic mode screen is to be displayed on the display unit 11 as the playback control screen, the switching button 181 is selected.

[0050] Figures 4A to 4C show an example of the basic mode screen 200 according to the embodiment. The basic mode screen 200 includes a footer 120, a switching control 201, an artwork display section 210, a tag section 220, an operation section 230, and a timeline display section 240. The basic mode screen 200 can be switched to the advanced mode screen 100 from the menu button 102.

[0051] The toggle control 201 displays and allows selection between "Player" and "Edit". When "Player" is selected, the basic mode screen 200 is displayed, and when "Edit" is selected, the music editing screen 50 is displayed.

[0052] The artwork display unit 210 displays artwork such as the album cover for the song. For example, if a song is created by combining multiple songs using the song editing screen 50, the artwork display unit 210 will display the artwork of the song currently being played.

[0053] The tag unit 220 has a tempo 221, a marker 222, and a loop 223 as tags. The display of the operation unit 230 switches according to the tag selected in the tag unit 220.

[0054] Figure 4A shows the state where tempo 221 is selected in the tag section 22. When tempo 221 is selected, the operation section 230 functions as the tempo operation section 140 of the advanced mode screen 100 described above, and displays a tempo up button 141, a tempo down button 142, and a slider bar 143. The operation section 230 accepts changes to the playback speed of the song by changing the tempo of the song using the tempo up button 141, the tempo down button 142, and the slider bar 143.

[0055] Figure 4B shows the state where marker 222 is selected in the tag unit 220. When marker 222 is selected, the operation unit 230 functions as the marker operation unit 150 of the advanced mode screen 100 described above, and the numbers "1" to "4" are displayed as four markers 151. The operation unit 230 accepts registration of the playback position of the song and specification of the playback position when a marker 151 is touched. A trash can button 152 is also displayed on the operation unit 230. After the trash can button 152 is touched, touching a marker 151 deletes the playback position registered in marker 151.

[0056] Figure 4C shows the state where loop 223 is selected in the tag unit 220. When loop 223 is selected, the operation unit 230 functions as the loop operation unit 160 of the advanced mode screen 100 described above, and the start position button 161, end position button 162, and loop button 163 are displayed. The operation unit 230 accepts registration of the start position and end position of loop playback, and specification of loop playback, when the start position button 161, end position button 162, and loop button 163 are touched. The operation unit 230 displays a trash can button 164. After the trash can button 164 is touched, the operation unit 230 deletes the playback positions registered in the start position button 161 and end position button 162 when the start position button 161 and end position button 162 are touched.

[0057] The timeline display unit 240 displays the timeline of the song, and a seek bar 241 is displayed on the timeline at the current playback position. The timeline display unit 240 also displays the playback time at the current playback position of the song and the total playback time of the song. If a playback position is registered to a marker 151, and if the start and end positions of loop playback are registered, the start positions of each are displayed on the timeline of the timeline display unit 240. In Figure 4C, "A" indicating the start position of the loop and "B" indicating the end position of the loop playback are displayed.

[0058] The basic mode screen 200 has a menu button 202 with three dots arranged vertically to the right of the switching control 201, and when the menu button 202 is selected, menu 270 is displayed. Menu 180 has a switching button 183 that instructs switching to the advanced mode screen 100 and an export button 182 that instructs exporting the music as an audio file. When the advanced mode screen 100 is displayed on the display unit 11 as the playback control screen, the switching button 183 is selected.

[0059] The Advanced Mode screen 100 and Basic Mode screen 200 allow you to set the tempo for playing songs and playback conditions such as loop playback.

[0060] Returning to Figure 1, the display control unit 16a controls the display of the music editing screen 50 on the display unit 11 according to the operation. The display control unit 16a also controls the display of either the advanced mode screen 100 or the basic mode screen 200 as the music editing screen on the display unit 11 according to the operation.

[0061] The editing unit 16b edits the music in response to operations on the music editing screen 50. The editing unit 16b stores the data of the music edited on the music editing screen 50 as project data 15a in the storage unit 15. For example, for each track 81 on the music editing screen 50, the editing unit 16b stores the data of the music registered on track 81 in the project data 15a. In addition, if the music is edited on the music editing screen 50, the editing unit 16b stores data indicating the edited content in the project data 15a. For example, if a portion of the music is cut out and extracted, the editing unit 16b stores the data of the extracted portion of the music in the project data 15a. In addition, if an effect is added to the music, the editing unit 16b stores data related to the added effect in the project data 15a. For example, if a fade-in is added to the music, the editing unit 16b stores the start position of the fade-in in the music, the end position of the fade-in, and data indicating that a fade-in is being applied to the music in the project data 15a. As a result, the music and editing content edited by the editing department 16b are stored in the project data 15a in the memory unit 15.

[0062] When playback of a song is instructed using the play / stop button 121, the playback unit 16c starts playing the song edited on the song editing screen 50. The playback unit 16c plays the song according to the playback conditions on the playback control screen. Playing according to the playback conditions on the playback control screen means playing the song on the playback unit 16c before exporting it in a specific format. For example, the playback unit 16c plays the song stored in the project data 15a according to the playback conditions set on the advanced mode screen 100 or the basic mode screen 200. For example, the playback unit 16c plays the song stored in the project data 15a at the tempo set on the advanced mode screen 100 or the basic mode screen 200. The playback unit 16c plays the songs of each track stored in the project data 15a in the order of their playback positions, reflecting the effects stored in the project data 15a. The playback unit 16c loops the music at the start and end positions of the loop playback if loop playback is specified on the advanced mode screen 100 or the basic mode screen 200.

[0063] The video recording unit 16d controls the camera unit 14 to record video in response to a recording instruction. For example, when the recording button 125 on the footer 120 of the advanced mode screen 100 or the basic mode screen 200 is touched, the display control unit 16a controls the display to show the video recording screen on the display unit 11.

[0064] Figure 5 shows an example of a video recording screen 300 according to the embodiment. The video recording screen 300 has a video display area 301 and a recording button 310.

[0065] The video display area 301 shows a live view. For example, when the recording button 125 is touched, the video recording unit 16d controls the camera unit 14 to record a video. For example, the video recording unit 16d controls the camera unit 14 to capture images as a video at a predetermined frame rate. The video recorded by the camera unit 14 is displayed as a live view in the video display area 301.

[0066] The recording button 310 is a button used to start and stop video recording.

[0067] When the shooting button 310 is touched, the playback unit 16c starts playing the song edited on the song editing screen 50. The playback unit 16c plays the song according to the playback conditions on the playback control screen. For example, the playback unit 16c plays the song stored in the project data 15a according to the playback conditions set on the advanced mode screen 100 or the basic mode screen 200. For example, the playback unit 16c plays the song stored in the project data 15a at the tempo set on the advanced mode screen 100 or the basic mode screen 200. The playback unit 16c plays the songs of each track stored in the project data 15a in the order of their playback positions, reflecting the effects stored in the project data 15a. If loop playback is specified on the advanced mode screen 100 or the basic mode screen 200, the playback unit 16c loops the song at the start and end positions of the loop playback.

[0068] When the shooting button 310 is touched, the video recording unit 16d records a video containing the music played by the playback unit 16c as audio, and saves the recorded video as video data in a predetermined video file format in the storage unit 15. Examples of video file formats include MPEG-4 (Moving Picture Experts Group phase 4), M4V, and MOV.

[0069] The generation unit 16e generates music data for a predetermined music file from the project data 15a stored in the storage unit 15 in response to an export instruction on the operation screen. Examples of music file formats include MP3 (MPEG Audio Layer-3) and AAC (Advanced Audio Coding). For example, when the export button 182 is touched from the menu 180 on the advanced mode screen 100 or the basic mode screen 200, the generation unit 16e generates music data for a predetermined music file from the project data 15a. The generation unit 16e saves the generated music data in the storage unit 15.

[0070] Next, we will describe an example of how a user involved in dance, such as a dancer, might use the information processing device 10 according to this embodiment. First, we will describe an example of a process in which the user edits music to be used for dancing and then films a video of themselves dancing to the edited music.

[0071] The user registers a song to track 81 on the song editing screen 50 and edits the song using the song editing screen 50, such as applying effects to the song or changing the length of the song. The editing unit 16b stores the data of the song edited on the song editing screen 50 as project data 15a in the storage unit 15.

[0072] When the user practices dancing to a song, they can use the advanced mode screen 100 or the basic mode screen 200 to change the tempo of the song, start playback from a specific part of the song, or repeatedly play a specific part. The playback unit 16c plays the song stored in the project data 15a according to the playback conditions set on the advanced mode screen 100 or the basic mode screen 200.

[0073] When a user wants to film a video of themselves dancing to an edited song, they select the record button 125 from the advanced mode screen 100 or the basic mode screen 200 to display the video recording screen 300, and then touch the record button 310. The playback unit 16c plays the song stored in the project data 15a according to the playback conditions set on the advanced mode screen 100 or the basic mode screen 200. The video recording unit 16d films a video that includes the song played by the playback unit 16c as audio, and saves the filmed video as video data in a predetermined video file format to the storage unit 15.

[0074] In this way, the information processing device 10 can record a video using the edited music before writing the edited music as a music file.

[0075] Next, we will explain an example of the process a user goes through when repeatedly editing music and shooting videos. The user reviews the recorded video and, if necessary, revise the dance choreography or edit the music again. For example, the user re-edits the music using the music editing screen 50. Then, the user shoots a video of themselves dancing to the re-edited music. For example, the user shoots a video using the video shooting screen 300.

[0076] The information processing device 10 displays the music editing screen 50 when the user selects "Edit" using the switching control 101 on the advanced mode screen 100 or the switching control 201 on the basic mode screen 200. This allows the user to easily re-edit a song using the music editing screen 50. When the user wants to film a video of themselves dancing to the re-edited song, they display the video filming screen 300 and touch the filming button 310. When the filming button 310 is touched, the information processing device 10 films a video that includes the re-edited song as audio and saves the filmed video as video data in a predetermined video file format in the storage unit 15.

[0077] Figure 6 shows an example of the process by which a user edits music and shoots a video. Figure 6(A) shows a time-series example of the process by which a user edits music using editing software and shoots a video using shooting software, as in the conventional method. Figure 6(B) shows a time-series example of the process by which a user edits music and shoots a video using the information processing device 10 according to the embodiment.

[0078] In the conventional method of using editing software and recording software, as shown in Figure 6(A), the user edits the music using the editing software and then exports and saves the edited music as music data in the form of a music file using the editing software. Music edited with editing software cannot be played by other software unless it is exported and saved as music data in the form of a music file, such as an MP3. Therefore, the user exports and saves the edited music as music data in the form of a music file using the editing software. When recording video, the user plays the saved music data using other software with playback capabilities while recording video with the recording software. The other software with playback capabilities may also be recording software.

[0079] Thus, when using editing and recording software, users need to export and save the edited music as music data in the form of a music file, and then record the video while playing back the saved music data, which is time-consuming.

[0080] After editing a song and recording a video, if, for example, the song and video don't match, the user re-edits the song, exports it, and repeats the video recording process. The music data that forms the music file is data that records the audio of the edited song. The user cannot re-edit the song without using editing software. Therefore, when repeatedly re-editing the song and recording a video, the user re-edits the song using editing software, and then exports and saves the re-edited song as music data that forms a music file using the editing software. Then, the user plays the saved music data using other software while recording a video with recording software.

[0081] Thus, when repeatedly re-editing a song and shooting a video, the user has to re-export and save the re-edited song as music data to form a music file each time they re-edit it, which is time-consuming.

[0082] On the other hand, when performing music editing and video recording using the information processing device 10 according to this embodiment, as shown in Figure 6(B), the user edits the music using the music editing screen 50. Then, the user switches the screen to the video recording screen 300 and records a video with the edited music.

[0083] The information processing device 10 stores the edited music on the music editing screen 50 as project data 15a in the storage unit 15. When recording a video using the video recording screen 300, the information processing device 10 plays the edited music stored in the project data 15a while recording the video. In this way, the information processing device 10 allows users to edit music using the music editing screen 50 and record a video with the edited music using the video recording screen 300 before writing the music as a music file. This reduces the workload on the information processing device 10 when a user edits music and records a video.

[0084] After the user shoots a video with the edited music, if, for example, the music and video do not match, the user re-edits the music and shoots the video again. When the user re-edits the music and shoots the video again, the user switches to the music editing screen 50 to re-edit the music. During re-editing, the music editing screen 50 displays the music and editing content stored in the project data 15a. The user can use the music editing screen 50 to re-edit the music that has already been edited, such as applying effects to the music or changing the length of the music. The user switches to the video shooting screen 300 and shoots a video with the re-edited music.

[0085] The information processing device 10 stores the re-edited music on the music editing screen 50 as project data 15a in the storage unit 15, and when shooting a video, it plays the re-edited music stored in the project data 15a. In this way, the information processing device 10 allows users to re-edit music on the music editing screen 50, and even after re-editing music, users can shoot a video with the re-edited music on the video shooting screen 300 before writing the re-edited music as a music file. This reduces the burden on the information processing device 10 when users repeatedly re-edit music and shoot videos.

[0086] When a user wants to export a song edited using the song editing screen 50 as a music file, they touch the export button 182 from the menu 180 on the advanced mode screen 100 or the basic mode screen 200. When the export button 182 is touched, the generation unit 16e generates music data for a predetermined music file from the project data 15a and saves the generated music data to the storage unit 15.

[0087] The music data stored in the memory unit 15 is data for forming music files, and therefore can be played on other devices. This allows the user to copy or move the music data stored in the memory unit 15 to another device, and then play the music edited using the music editing screen 50 on that other device.

[0088] <Processing flow> Next, the flow of control processing, including music playback processing, of the information processing device 10 according to this embodiment will be described. Figure 7 is a flowchart showing an example of the procedure for control processing, including music playback processing, according to this embodiment. The music playback processing shown in Figure 7 is performed when the shooting button 310 on the video shooting screen 300 is touched to instruct the start of video recording.

[0089] The display control unit 16a determines whether or not the display of the playback control screen is instructed (step S10). For example, the display control unit 16a determines that the display of the playback control screen is instructed if "Player" is selected in the switching controls 51, 101, and 201.

[0090] If the display of the playback control screen is not instructed (step S10: No), the display control unit 16a controls the display of the music editing screen 50 because "Edit" is selected in the switching controls 51, 101, and 201 (step S11). The editing unit 16b edits the music in response to the operation on the music editing screen 50. The editing unit 16b stores the data of the music edited on the music editing screen 50 as project data 15a in the storage unit 15.

[0091] If the music editing on the music editing screen 50 is not completed (step S12: No), the display control unit 16a proceeds to step S10 described above, and if the music editing on the music editing screen 50 is completed (step S12: Yes), it proceeds to step S21 described later.

[0092] On the other hand, if the display of the playback control screen is instructed (step S10: Yes), the display control unit 16a controls the display of either the advanced mode screen 100 or the basic mode screen 200 as the music editing screen (step S13). The advanced mode screen 100 and the basic mode screen 200 accept the setting of playback conditions when playing a song. When the playback unit 16c is instructed to play a song using the play / stop button 121, it plays the song stored in the project data 15a according to the playback conditions set on the advanced mode screen 100 or the basic mode screen 200.

[0093] The display control unit 16a determines whether or not the display of the video recording screen has been instructed (step S14). For example, the display control unit 16a determines that the display of the video recording screen has been instructed if the recording button 125 is selected on the advanced mode screen 100 or the basic mode screen 200. If the display of the video recording screen has not been instructed (step S14: No), the system proceeds to step S21, which will be described later.

[0094] On the other hand, if the display of the video recording screen is instructed (step S14: Yes), the display control unit 16a performs control to display the video recording screen 300 (step S15).

[0095] The video recording unit 16d determines whether or not it has been instructed to start video recording (step S16). For example, the video recording unit 16d determines that it has been instructed to start video recording if the recording button 310 is touched on the video recording screen 300. If it has not been instructed to start video recording (step S16: No), it proceeds back to step S15 to wait for an instruction to start.

[0096] If the system is instructed to start video recording (step S16: Yes), the playback unit 16c starts playing the music edited on the music editing screen 50 according to the playback conditions on the playback control screen (step S17). For example, the playback unit 16c plays the music stored in the project data 15a according to the playback conditions set on the advanced mode screen 100 or the basic mode screen 200.

[0097] The video recording unit 16d records a video that includes the music played by the playback unit 16c as audio (step S18).

[0098] The video recording unit 16d determines whether or not it has been instructed to end recording (step S19). For example, if the recording button 310 on the video recording screen 300 is touched after video recording has started, the video recording unit 16d determines that it has been instructed to end recording. If it has not been instructed to end recording (step S19: No), it proceeds to step S17 and continues recording the video.

[0099] On the other hand, if the recording is instructed to end (step S19: Yes), the playback unit 16c stops playing the music. The video recording unit 16d stops recording the video and saves the recorded video as video data in the storage unit 15 in the form of a predetermined video file (step S20).

[0100] The display control unit 16a determines whether or not to terminate the process (step S21). For example, if a predetermined operation to instruct the termination of the process is performed, the display control unit 16a determines that the process is finished (step S21: Yes) and terminates the process. On the other hand, if the termination of the process is not instructed (step S21: No), the process proceeds to step S10 described above.

[0101] <Summary> As described above, the information processing device 10 according to this embodiment plays the music edited on the music editing screen 50 according to the playback conditions on the playback control screen (advanced mode screen 100 and basic mode screen 200), and records a video that includes the played music as audio. This allows the information processing device 10 to record a video with the edited music before writing the edited music as a music file.

[0102] <Embodiment 2> Now, although Embodiment 1 of this disclosure has been described, various applications are possible, and furthermore, it may be implemented in various different forms other than Embodiment 1 described above.

[0103] <Exhibiting creative abilities> The details described in Embodiment 1 above, such as the screen configurations of the music editing screen 50, advanced mode screen 100, basic mode screen 200, and video recording screen 300, are merely examples and can be changed. Similarly, the content of music editing on the music editing screen 50 is also merely an example and can be changed. Furthermore, the playback conditions that can be set on the basic mode screen 200 and video recording screen 300 are also merely examples and can be changed. Additionally, the flowchart described in Embodiment 1 above can also have its processing order altered within a consistent range.

[0104] <Application to other usage scenarios> In the above embodiment 1, the information processing device 10 was given as an example of a mobile terminal such as a smartphone, but the editing function, playback control function, and video recording function of the information processing device 10 may be provided by a cloud service such as a web service.

[0105] Figure 8 is a block diagram showing an example of the functional configuration of an information processing device 400 according to an embodiment. For example, Figure 8 shows an information processing device 400 that provides cloud services.

[0106] The information processing device 400 may be implemented by a server device. For example, the information processing device 400 can provide editing functions, playback control functions, and video recording functions as a cloud service by running a PaaS (Platform as a Service) type middleware or a SaaS (Software as a Service) type application.

[0107] As shown in Figure 8, the information processing device 400 can be connected to a user terminal 420 via a network NW in a communicative manner. For example, the network NW may be any type of communication network, such as the Internet or a LAN (Local Area Network), whether wired or wireless.

[0108] The user terminal 420 is a terminal device used by a user receiving cloud services. The user terminal 420 may be implemented using any computer, such as a smartphone, tablet, wearable device, or personal computer.

[0109] Under this overall configuration, the information processing device 400 shown in Figure 8 differs from the information processing device 10 shown in Figure 1 in that various requests are received via the network NW, and the results of the processing are output via the network NW.

[0110] For example, the information processing device 400 transmits screen information of operation screens, including the music editing screen and the playback control screen, to the user terminal 420 via the communication unit 413 to display the operation screen on the user terminal 420, and receives operation information for the operation screen from the user terminal 420 via the communication unit 413. In addition, the information processing device 400 controls the camera of the user terminal 420 via the communication unit 413 to record video, and receives the recorded video data from the user terminal 420 via the communication unit 413.

[0111] The information processing device 400 provides editing functions, playback control functions, and video recording functions to the user terminal 420. For example, the display control unit 416a, editing unit 416b, playback unit 416c, video recording unit 416d, and generation unit 416e of the control unit 416 of the information processing device 400 may have the same functions as the display control unit 16a, editing unit 16b, playback unit 16c, video recording unit 16d, and generation unit 16e of the control unit 16 of the information processing device 10. Also, the project data 415a of the storage unit 415 of the information processing device 400 may have the same functions as the project data 15a of the storage unit 15 of the information processing device 10.

[0112] Even with the information processing device 400 described above, it is possible to shoot a video using the edited music before writing the edited music as a music file.

[0113] <System> The processing procedures, control procedures, specific names, and various data and parameters shown in the above documents and drawings may be changed at will unless otherwise specified. For example, one or more of the functions of the information processing device 10, such as the display control unit 16a, editing unit 16b, playback unit 16c, video recording unit 16d, and generation unit 16e, may be configured as separate devices.

[0114] Furthermore, the components of each illustrated device are functionally conceptual and do not necessarily need to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown. That is, all or part of them can be functionally or physically distributed and integrated in any units according to various loads and usage conditions. Note that each configuration may also be a physical configuration.

[0115] Furthermore, the processing performed by the illustrated device can be implemented, in whole or in part, by programs executed by hardware processors such as MPUs (Micro-Processing Units) or CPUs (Central Processing Units), or by hardware using wired logic.

[0116] <Hardware> Next, an example of the hardware configuration of the information processing device described in the above embodiment will be explained. Figure 9 is a diagram showing an example of the hardware configuration. As shown in Figure 9, the information processing device 10 includes a communication device 10a, a storage device 10b, a memory 10c, and a processor 10d. Note that the parts shown in Figure 9 may be interconnected by a bus or the like.

[0117] The communication device 10a is a network interface card, etc. The storage device 10b is a storage device such as an HDD (Hard Disk Drive) or SSD (Solid State Drive). For example, the storage device 10b stores programs and databases that operate the functions shown in Figure 1.

[0118] The processor 10d runs a process that performs the functions described in Figure 1 by reading a program that performs the same processing as the processing unit shown in Figure 1 from the storage device 10b or the like and loading it into memory 10c.

[0119] Such a process implements functions similar to those of the processing unit of the information processing device 10. For example, the processor 10d reads a program having functions similar to those of the display control unit 16a, editing unit 16b, playback unit 16c, video recording unit 16d, and generation unit 16e from the storage device 10b, etc. Then, the processor 10d executes a process that performs the same processing as the display control unit 16a, editing unit 16b, playback unit 16c, video recording unit 16d, and generation unit 16e, etc.

[0120] In this way, the information processing device 10 realizes the same functions as the embodiments described above by reading and executing a program. Furthermore, the information processing device 10 can also realize the same functions as the embodiments described above by reading the program from a recording medium using a media reader and executing the read program. It should be noted that the program referred to in these other embodiments is not limited to being executed by the information processing device 10. For example, the functions of this disclosure can be similarly applied when another computer or server executes a program, or when they collaborate to execute a program.

[0121] The above program can be distributed via a network such as the Internet. Furthermore, the program can be recorded on any storage medium and executed by reading it from the medium by a computer. For example, the storage medium can be a hard disk, flexible disk (FD), CD-ROM, MO (Magneto-Optical disk), DVD (Digital Versatile Disc), etc. [Explanation of Symbols]

[0122] 10,400 Information Processing Devices 10a Communication device 10b Storage device 10c memory 10d processor 11 Display section 12 Input section 13,413 Communications Department 14 Camera Section 15,415 storage section 15a,415a Project Data 16,416 Control Unit 16a, 416a Display Control Unit 16b,416b Editorial Department 16c,416c playback section 16d, 416d Video Recording Department 16e,416e generation part

Claims

1. A display control unit that displays an operation screen on the display unit, which includes a music editing screen for editing music and a playback control screen for controlling the playback conditions of music, An editing department that edits a song in response to operations performed on the aforementioned song editing screen, A playback unit that plays the music edited on the aforementioned music editing screen according to the playback conditions on the playback control screen, A video recording unit plays a song using the aforementioned playback unit while simultaneously recording a video that includes the song as audio. An information processing device characterized by having the following features.

2. A storage unit that stores music data edited by the aforementioned editorial department, The system further includes a generation unit that generates music data for forming a predetermined music file from music data stored in the storage unit in response to an output instruction on the operation screen, The information processing apparatus according to claim 1, characterized in that the playback unit plays back the music data stored in the storage unit.

3. The information processing device according to claim 1, characterized in that the playback unit starts playing the music edited on the music editing screen in response to the operation to start video recording.

4. The playback control screen allows setting the tempo for playing the music as a playback condition. The information processing apparatus according to claim 1, characterized in that the playback unit plays the music at the tempo set on the playback control screen.

5. The information processing device according to claim 1, characterized in that the music editing screen allows for editing by adding effects to the music.

6. The information processing device according to claim 1, characterized in that the music editing screen allows for the extraction of a portion of one or more songs and the editing of the extracted portions by joining them together.

7. The information processing device according to claim 1, characterized in that the music editing screen displays the audio waveform of the entire music and allows editing by specifying a start point and an end point for the audio waveform to add fade-in and fade-out effects to the music.

8. The information processing device according to claim 1, characterized in that the music editing screen allows for editing multiple music tracks together.

9. The information processing device according to claim 1, characterized in that the aforementioned music editing screen allows for the editing of multiple music tracks by mixing them.

10. The music edited on the music editing screen is played according to the playback conditions on the playback control screen, which controls the playback conditions of the music. Record a video that includes the aforementioned song as audio. A music playback program characterized by having a computer perform the processing.

Images