Music game system, music game data generation program, and music game data generation method

Abstract

An object of the present invention is to provide a technology capable of generating musical scores for music games that can reproduce music through user operations. [Solution] In order to solve the above problems, the present invention provides a music game system that generates sheet music for a music game from a music sound source, the music game system comprising a part separation unit, a beat detection unit, an onset detection unit, and a sheet music generation unit, the part separation unit separates music parts based on the music sound source and acquires music part information, the beat detection unit detects beat information based on the music sound source, the onset detection unit detects onset information for each music part based on the music part information, and the sheet music generation unit generates sheet music data in which note objects of the music game are arranged based on the beat information and the onset information.

Description

[Technical Field]

[0001] The present invention relates to a music game system, a music game data generation program, and a music game data generation method. [Background technology]

[0002] 2. Description of the Related Art Conventionally, there is known a technique for automatically generating musical scores from musical sound sources by using audio analysis technology. An example of such an automatic music transcription technique is proposed in Patent Document 1, for example.

[0003] For example, Patent Document 1 describes a method of receiving an audio signal, extracting frequency information from the audio signal, extracting amplitude information from the audio signal, extracting pitch information from the frequency information, generating note onset events from the information related to the frequency and amplitude, determining note lengths from the audio signal using the extracted frequency information and amplitude information and the generated note onset events, determining rests, determining tempo over a time window, determining key, and determining instrumentation, and generating musical score display data using the note lengths, rests, tempo, key, and instrumentation. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Special Publication No. 2010-518428 Summary of the Invention [Problem to be solved by the invention]

[0005] However, although Patent Document 1 can generate a score for playing a piece of music from a music sound source, it cannot generate a score for a music game that allows a piece of music to be played by one person.

[0006] In view of the above problems, an object of the present invention is to provide a technology capable of generating musical scores for music games that can reproduce music through user operations. [Means for solving the problem]

[0007] In order to solve the above problems, the present invention provides a music game system that generates sheet music for a music game from a music sound source, the music game system comprising a part separation unit, a beat detection unit, an onset detection unit, and a sheet music generation unit, the part separation unit separates music parts based on the music sound source and acquires music part information, the beat detection unit detects beat information based on the music sound source, the onset detection unit detects onset information for each music part based on the music part information, and the sheet music generation unit generates sheet music data in which note objects of the music game are arranged based on the beat information and the onset information.

[0008] With this configuration, it is possible to generate sheet music data in which note objects for a music game are arranged using beat information and onset information obtained from a music source. This allows a user to generate sheet music data for their favorite music source. Furthermore, in the present invention, onset information is detected based on music part information obtained from the music source. If the music parts were separated after identifying onsets, the onset timing of each music part would be inaccurate, impairing the game's playability. However, the present invention achieves the remarkable effect of being able to generate sheet music data while maintaining the playability of the game by identifying onsets after separating the music parts from the music source.

[0009] In a more preferred embodiment, the music game system further includes a game data generation unit, which generates music game data in which the sounds of each musical part are linked to the note objects based on the musical part information and the score data.

[0010] With this configuration, sounds close to the sound source can be output when playing a music game, allowing the user to enjoy the music game even more.

[0011] In a more preferred embodiment, the music game system further includes a pitch detection unit, the music game has a predetermined number of lanes, the pitch detection unit detects pitch information for each musical part based on the musical part information, and the score generation unit generates the score data in which the note objects that can distinguish the musical parts are arranged in the lanes based on the pitch information.

[0012] This configuration allows for the automatic generation of highly visible music score data according to the type of musical part and the pitch of each musical part, allowing note objects that identify musical parts to be placed, allowing users to enjoy the music game even more.

[0013] In a more preferred embodiment, the musical score generating section determines the placement timing of the note objects based on the beat information and the onset information.

[0014] In a more preferred embodiment, the musical score generating unit determines synchronization timing based on beat timing and onset timing as the placement timing of the note objects.

[0015] With this configuration, the placement of note objects can be automatically determined in accordance with the beat and onset obtained from the music source, allowing the user to play the music game in sync with the rhythm of the music source, making the music game even more enjoyable.

[0016] In a more preferred embodiment, the musical score generating section determines the timing of placing the note objects further based on difficulty level information.

[0017] With this configuration, the placement of note objects can be determined according to the difficulty level, allowing a variety of users to enjoy the music game according to the difficulty level.

[0018] In a more preferred embodiment, the music game system further includes an output processing unit and a judgment unit, wherein the output processing unit receives pressing of the note object and outputs a sound based on the music part information, the judgment unit judges a pressing timing level based on the pressing timing of the note object, and the output processing unit adjusts the output timing of the sound and outputs the sound when the pressing timing level is equal to or higher than a predetermined level.

[0019] With this configuration, sounds can be output according to the quality of the user's performance, which can increase the user's motivation to play the music game. [Effects of the Invention]

[0020] The present invention has the effect of providing a technique capable of generating musical scores for a music game that can reproduce music through user operations. [Brief explanation of the drawings]

[0021] [Figure 1] FIG. 1 is a block diagram illustrating a configuration of a system according to an embodiment. [Figure 2] FIG. 1 is a hardware configuration diagram of a system according to an embodiment. [Figure 3] FIG. 2 is a functional block diagram of an apparatus according to one embodiment. [Figure 4] 1 is a process flowchart of a system according to an embodiment. [Figure 5] 10 is an example of a method for identifying note timing in one embodiment. [Figure 6] 10 is an example of a game screen according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0022] The present invention will now be described more fully with reference to the accompanying drawings, in which preferred embodiments are shown, but which may be embodied in many different forms and are not limited to the embodiments set forth herein.

[0023] For example, while the configuration, operation, etc. of a music game system are described in this embodiment, methods, devices, computer programs, etc. with similar configurations can also achieve similar effects. Furthermore, the program may be stored on a recording medium or provided as a downloadable program from an external server. Furthermore, using this recording medium, for example, the program can be installed on a computer, thereby configuring a music game data generation device and a music game system. Here, the recording medium storing the program may be a non-transitory recording medium, such as a CD-ROM.

[0024] The present invention relates to a music game system that generates musical scores for music games. By accepting an uploaded music source, the system detects the beat of the music based on the music source. The system also separates the music parts contained in the music source based on the music source and detects the onset and pitch for each part. The system then generates music score data in which note objects for the music game are arranged using the beat of the music and the onset and pitch for each part.

[0025] In a typical music game, the user selects one song from multiple songs registered in a database, and the game uses that song as background music and begins outputting score data associated with that song.The game then accepts presses from the user on note objects that move based on the score data, and the total score is determined based on the timing of the presses, and the game is competed for.

[0026] In the music game of the present invention, when a user selects a song, the sound of a specific part from among the song parts separated from the sound source of the song is output as background music, and the sound of the specific part is output in response to the pressing of a note object. In this embodiment, the song parts whose sound is output as background music are rhythm parts such as drums and bass, and the song parts whose sound is output in response to the pressing of a note object are melody or harmony parts such as vocals, piano, and guitar, but this is not limited thereto. For example, if the song parts of the song sound source include orchestral instruments or percussion instruments, the output can be changed as desired.

[0027] The music game of the present invention has a predetermined number of lanes on the musical score, and accepts presses from the user on note objects that move from one side to the other along the lanes. In this embodiment, the predetermined number of lanes is four, but there is no limit to the number.

[0028] <1. System configuration> Fig. 1 is a block diagram showing the configuration of a system according to one embodiment. As shown in Fig. 1, the music game system 0 includes a music game data generation device 1 and a user terminal device 3. In this embodiment, the music game data generation device 1 and the user terminal device 3 are configured to be able to communicate with each other via a communication network NW. The communication network NW in this embodiment is an IP (Internet Protocol) network, but there are no restrictions on the type of communication protocol, and there are also no restrictions on the type or scale of the network.

[0029] In this embodiment, a general-purpose server computer or a personal computer can be used as the music game data generation device 1. Furthermore, a smartphone, tablet terminal, personal computer, wearable device, or the like can be used as the user terminal device 3. Furthermore, the music game data generation device 1 may be configured with multiple computers that are capable of sending and receiving information via a communication network NW or another network.

[0030] <1.1. Hardware configuration of the present invention> 2 is a hardware configuration diagram of the music game system 0. As shown in FIG. 2(a), the server 10 (music game data generation device 1) includes a processing unit 101, a storage unit 102, and a communication unit 103.

[0031] The processing unit 101 has one or more processors, such as a CPU, that can execute an instruction set, and controls the overall operation and processing of the music game data generation device 1 by executing the music game data generation program of the present invention, an OS, and other applications. The storage unit 102 has a volatile memory such as a RAM capable of storing an instruction set, and a non-volatile recording medium such as an HDD or SSD capable of recording an OS and the music game data generation program according to the present invention. The communication unit 103 has a communication interface device with the communication network NW, and controls communication with the communication network NW to input and output information.

[0032] As shown in FIG. 2(b), the terminal device 9 (user terminal device 3) includes a processing unit 91, a storage unit 92, a communication unit 93, an input unit 94, and an output unit 95.

[0033] The processing unit 91 has one or more processors, such as a CPU, that can execute an instruction set, and controls the overall operation and processing of the terminal device 9 by executing a music game program, an OS, and other applications for playing music scores generated by the music game data generation device 1. The storage unit 92 includes a volatile memory such as a RAM capable of storing an instruction set, and a non-volatile recording medium such as an HDD or SSD capable of recording an OS and the like. The communication unit 93 has a communication interface device for connecting to a network, and controls communication with the communication network NW to input and output information. The input unit 94 has an input device capable of input processing, such as a keyboard or a touch panel. The output unit 95 has a display device capable of display processing, such as a display, and an audio device capable of outputting sound.

[0034] <1.2. System Functional Configuration> FIG. 3 is a functional block diagram of the music game system 0 according to this embodiment.

[0035] <1.2.1. Functional Configuration of Music Game Data Generation Device 1> 3, the music game data generation device 1 includes a sound source acquisition unit 11, a part separation unit 12, a beat detection unit 13, an onset detection unit 14, a pitch detection unit 15, a music score generation unit 16, a data compression unit 17, a game data generation unit 18, a distribution unit 19, and a database 2. This is a specific implementation of the music game data generation program stored in a storage unit 102 by a processing unit 101.

[0036] <1.2.2.Database 2> The generated music game data is stored in database 2. The music game data includes a music game data ID for uniquely identifying the music game data, compressed part audio information, and music score data, and is registered in association with information (user ID) for uniquely identifying the user who requested the generation of the music game data.

[0037] <1.2.2.1. Compressed part audio information> Compressed part audio information is information about the audio of a musical part separated from a musical sound source (music part information) that has been reduced in data size using well-known data compression technology. As the musical part information, BGM musical part information and performance musical part information are set. The BGM music part information is information relating to the audio of a music part that is output as BGM. As the BGM music part information, BGM music part audio is set for each BGM part ID for uniquely identifying a BGM music part. The performance musical part information is information about the audio of a musical part that is output when a note object is pressed. As the performance musical part information, a performance musical part audio is set for each performance part ID that uniquely identifies the performance musical part.

[0038] In this embodiment, compressed audio data is stored as the compressed part audio information, but uncompressed audio data may also be stored.

[0039] <1.2.2.2. Musical score data> The music score data is information about the music score of the generated music piece. As the music score data, note arrangement information is set for each difficulty level of the music game. The note placement information is information about note objects that are placed on a musical score, and the note placement information includes a performance part ID, an audio start time, and an audio end time set for each note object.

[0040] <1.2.3. Sound source acquisition section 11> The sound source acquisition unit 11 acquires sound source information of a musical piece sound source. The sound source acquisition unit 11 acquires the sound source information by accepting upload of the sound source information from the user.

[0041] <1.2.4. Part Separation Section 12> The part separation unit 12 separates the music parts based on the sound source information and acquires the music part information. The part separation unit 12 separates the music parts from the acquired sound source information and acquires BGM music part information and performance music part information as the music part information.

[0042] <1.2.5. Beat detection unit 13> The beat detection unit 13 detects beat information of a song based on the sound source information. The beat information is information about the beat of a song, and the beat information is a combination of beats and beat timing (time). Here, a beat is a unit of rhythm that is beaten at equal intervals.

[0043] <1.2.6. Onset detection unit 14> The onset detection unit 14 detects onset information based on the musical piece part information. The onset detection unit 14 detects onset information based on the performance musical piece part information from the acquired musical piece part information. The onset information is information related to the onset (the attack portion of a sound, i.e., the moment when a sound is suddenly generated) of each performance musical piece part, and the onset timing (vocalization time) of each performance musical piece part is detected as the onset information.

[0044] <1.2.7. Pitch detection unit 15> The pitch detection unit 15 detects pitch information based on the musical piece part information. The pitch detection unit 15 detects pitch information based on the performance musical piece part information from the acquired musical piece part information. The pitch information is information related to the pitch (pitch) of each performance musical piece part, and the pitch information is a combination of pitch and time for each performance musical piece part.

[0045] <1.2.8. Musical notation generator 16> The score generation unit 16 generates score data in which note objects are arranged based on sound source information. The score generation unit 16 generates score data in which note objects are arranged based on beat information and onset information.

[0046] <1.2.9. Data Compression Unit 17> The data compression unit 17 compresses the audio data based on the music part information, and generates compressed part audio information.

[0047] <1.2.10. Game Data Generation Unit 18> The game data generation unit 18 generates data for implementing a music game in which the sound of a musical part is output in response to the user pressing a note object. The game data generation unit 18 generates music game data in which the sound of each musical part is linked to a note object based on the compressed part audio information and the score data.

[0048] <1.2.11.Distribution Section 19> The distribution unit 19 distributes the music game data stored in the database 2 to the user terminal device 3, and the data storage unit 31 stores the distributed music game data in the memory unit 92. The output processing unit 32 performs output processing based on the distributed music game data and sends the output processing results to the output unit 95, whereby various screens and sounds are output on the user terminal device 3.

[0049] <1.3. Functional configuration of user terminal device 3> 3, the user terminal device 3 includes a data storage unit 31, an output processing unit 32, a determination unit 33, and an input processing unit 34. This is a specific implementation of a music game program stored in a memory unit 92 by the processing unit 91.

[0050] <1.3.1. Output processing unit 32> The output processing unit 32 outputs a music game screen, which will be described later.

[0051] The output processing unit 32 also receives a press of a note object and outputs a sound. The output processing unit 32 receives a press of a note object from the input processing unit 34 and outputs a sound based on the performance musical piece part information associated with the note object.

[0052] <1.3.2. Judgment unit 33> The determination unit 33 determines the deviation in the timing of pressing the note object. The determination unit 33 determines the press timing level based on the note arrangement information and the press timing of the note object pressed by the user. Specifically, the determination unit 33 detects the deviation between the timing at which the user presses the note object and the start time of the audio of the note object, and determines the press timing level, such as GREAT, GOOD, or BAD, depending on the magnitude of the deviation.

[0053] <2. Processing flow> Next, a method for generating music game data using the music game system 0 of the present invention will be described with reference to Figures 4 to 6. Figure 4 is a flowchart showing the processing performed by the music game data generation device 1 from acquiring sound source information to generating music game data.

[0054] <2.1. Acquiring audio source information> In step S1 (hereinafter, "step SX" will be simply referred to as "SX"), the sound source acquisition unit 11 acquires sound source information. In this embodiment, the sound source acquisition unit 11 accepts uploading of audio data such as WAV, MP3, and FLAC (Free Lossless Audio Codec) as sound source information from a user, and acquires the sound source information.

[0055] <2.2. Acquiring song part information> In S2, the part separation unit 12 acquires musical piece part information based on the sound source information. In this embodiment, the part separation unit 12 separates musical piece parts included in the musical piece sound source using well-known techniques (signal processing (frequency analysis, spectrum analysis, etc.), machine learning, sound source separation algorithms (non-negative matrix factorization, independent component analysis, etc.)) on the musical piece sound source information acquired in S1. Then, the part separation unit 12 acquires, as musical piece part information, BGM musical piece part information and performance musical piece part information including the audio for each musical piece part.

[0056] <2.3. Acquiring beat information> In S3, the beat detection unit 13 detects beat information based on the sound source information. In this embodiment, the beat detection unit 13 detects beat information including a combination of beats and beat timings of the sound source information by using well-known techniques (signal processing, machine learning, etc.) on the sound source information acquired in S1. Note that, although the beat detection unit 13 detects beat information based on the sound source information in this embodiment, it may also detect beat information based on BGM music part information.

[0057] 2.4. Detection of onset information In S4, the onset detection unit 14 detects onset information based on the music part information. In this embodiment, the onset detection unit 14 detects onset information including onset timing for each performed music part using a well-known method (such as an onset detection algorithm (detection based on energy change, detection based on spectrum difference, detection based on phase difference, tempo-synchronized detection, etc.) and machine learning, or a combination thereof) for the performed music part information among the music part information acquired in S2.

[0058] 2.5. Detecting pitch information In S5, the pitch detection unit 15 detects pitch information based on the music part information. In this embodiment, the pitch detection unit 15 detects pitch information including a combination of pitch and time for each performance music part using a well-known method (time domain pitch detection (zero crossing method, autocorrelation method, AMDF method, etc.), frequency domain pitch detection (Fourier transform, cepstrum method, harmonic peak detection method, etc.), machine learning, etc.) for the performance music part information among the music part information acquired in S2.

[0059] <2.6. Creating Musical Score Data> In S6, the score generation unit 16 generates score data based on beat information, onset information, and pitch information. The score generation unit 16 determines the placement timing of note objects based on at least the beat information and onset information. In this embodiment, the score generation unit 16 determines the audio start time of the note objects as the placement timing based on the beat information, onset information, and difficulty level information. Specifically, the score generation unit 16 determines the placement timing of note objects for each difficulty level based on the beat timing and onset timing. Note that in this embodiment, the difficulty levels are, for example, difficult, normal, and easy, but the number and notation thereof are not limited to these.

[0060] 5A and 5B are diagrams showing an example of a process for determining the placement timing of note objects based on beat information, onset information, and difficulty level information. From the top, Fig. 5A shows the sound source information acquired in S1, the music part information acquired in S2, and the onset information detected in S4.

[0061] 5(b) shows, from top to bottom, the onset information detected in S4, the beat information detected in S3, and the placement timing of the note objects. As shown in FIG. 5(b), the score generation unit 16 specifies a synchronization timing based on the onset timing and the beat timing as the placement timing of the note objects. Specifically, the score generation unit 16 specifies an onset timing that perfectly or incompletely matches the beat timing as the synchronization timing, and specifies this synchronization timing as the placement timing of the note objects. Here, an onset timing that perfectly or incompletely matches the beat timing refers to a timing at which the onset timing and the beat timing overlap, or, among multiple onset timings and beat timings, the onset timing that is closest to the beat timing.

[0062] Furthermore, if an onset appears after a predetermined period of time without an onset, the score generation unit 16 identifies the synchronization timing based on the timing at which the onset appears. Specifically, the score generation unit 16 identifies the synchronization timing as the timing at which the onset appears and the beat timing completely or incompletely match, and identifies this synchronization timing as the placement timing of the note object. For example, in Figure 5(b), an onset appears after a predetermined period of time without an onset (such as an introductory or interlude), and the synchronization timing based on the timing at which the first onset appears and the beat timing is identified as the placement timing.

[0063] Furthermore, the score generation unit 16 specifies the placement timing so that there is one note object within a predetermined interval. Specifically, the score generation unit 16 specifies the placement timing so that there is one note object within a predetermined interval according to the difficulty level. For example, in FIG. 5(b), the predetermined interval is set to two beats, and the placement timing is specified so that one note object is placed within this predetermined interval.

[0064] Furthermore, the score generation unit 16 specifies the lane in which the note objects are to be placed based on the pitch information. In this embodiment, the score generation unit 16 specifies the lane in which the note objects are to be placed for each musical part based on the pitch information for each musical part detected in S5, so that note objects associated with the same pitch are placed in the same lane and in order of increasing or decreasing pitch.

[0065] In a preferred embodiment of the present invention, for a note object in one musical part, the lane in which the note object is placed is identified based on the pitches associated with successively placed note objects and the number of lanes. Specifically, in this embodiment, since note objects are placed in ascending or descending order of pitch, it may not be possible to place the note objects in ascending or descending order depending on the pitches associated with successive note objects and the number of lanes. In this case, the score generation unit 16 identifies the lane in which the note object is placed based on the pitches associated with one or more note objects that are not associated with the same pitch as the note object placed in each lane but are placed immediately after the note object.

[0066] Specifically, the score generation unit 16 identifies the placement lanes for note objects by associating pitches associated with note objects with placement lanes from left to right in ascending or descending order based on the pitches associated with consecutive note objects until the number of types of pitches associated with consecutive note objects matches the number of lanes. Then, when pitches have been associated with all placement lanes, if there is a next consecutive note object whose associated pitch does not match the pitch associated with the placed note object, the score generation unit 16 identifies the placement lanes for note objects by associating pitches associated with note objects with placement lanes from left to right in descending or descending order based on the pitches associated with consecutive note objects until the number of types of pitches associated with note objects after that note object matches the number of lanes.

[0067] For example, if there are four lanes and consecutive note objects are associated with the pitches (international style) "A3," "C4," "F4," "A4," "C4," "G4," "E4," "B4," and "C5," the pitch types are first associated with "A3," "C4," "F4," and "A4" in order from the leftmost lane (arrangement in order of increasing pitch from left to right). In this case, the arrangement lanes for the note objects are specified so that the pitches are "E4," "G4," "B4," and "C5" in order from left to right, based on the number of pitch types associated with note objects subsequent to the note object associated with a pitch (G4) that does not match the pitch already arranged.

[0068] Furthermore, the score generation unit 16 places distinguishable note objects for each musical part. In this embodiment, the score generation unit 16 places distinguishable note objects for each musical part by linking them to a performance part ID based on the musical part being played that is linked to the note object. In a preferred embodiment of the present invention, distinguishable note objects for each musical part are placed in the same lane.

[0069] <2.7. Generation of compressed part audio information> In S7, the data compression unit 17 generates compressed part audio information based on the music part information. In this embodiment, the data compression unit 17 uses a well-known data compression technique on the music part information acquired in S2 to generate compressed part audio information by compressing the audio data of the BGM music part and the performance music part for each music part.

[0070] <2.8. Generating music game data> In S8, the game data generation unit 18 generates music game data in which the sounds of each musical part are linked to note objects based on the compressed part audio information and the score data. In this embodiment, the game data generation unit 18 generates music game data for a music game in which the compressed performance musical part audio is linked to note objects based on the compressed part audio information generated in S7 and the score data generated in S6, the BGM musical part information and performance musical part information (compressed part audio information) for each compressed musical part, and the arrangement timing (audio start time) of the note arrangement information. The game data generation unit 18 then registers the music game data in database 2.

[0071] <3. An example of a music game screen> 6 is an example of a music game screen displayed on the user terminal device 3 based on the music game data distributed by the distribution unit 19. In the illustrated example, instrument note objects INT and vocal note objects VNT are arranged as note objects for each musical part being performed, and four lanes L1 to L4, a beat bar BB, a timing bar TB, and a press reception unit PI are displayed.

[0072] In this embodiment, the output processing unit 32 outputs note objects (in the illustrated example, instrument note objects INT and vocal note objects VNT arranged in four lanes L1 to L4) that vary depending on the musical part, so that they move from the top to the bottom of the screen, and accepts presses of the note objects in the corresponding lanes by accepting presses from the user at the press acceptance units PI corresponding to each of the lanes L1 to L4. The closer the presses are accepted when the center of the note object is closer to the timing bar TB, the higher the press timing level becomes, and a score according to the press timing level is added.

[0073] In addition, the output processing unit 32 outputs the BGM music part audio based on the BGM music part information, and by accepting the pressing of a note object, outputs the performance music part audio based on the pressing timing level and the performance music part information linked to the note object.

[0074] In this embodiment, for a first note object and a second note object linked to the same musical performance part, after receiving a press of the first note object, the output processing unit 32 outputs the sound of the musical performance part linked to the first note object based on the placement timing of the second note object and the press timing level of the second note object.The output processing unit 32 then outputs the sound of the musical performance part linked to the second note object based on the press timing level of the second note object.

[0075] <3.1. Example 1 when the press timing level is less than GREAT> Specifically, for a first note object and a second note object linked to the same musical performance part, if the output processing unit 32 receives a press of the first note object and then receives a press of the second note object a predetermined time or more before the placement timing of the second note object (a press timing at which the press timing level is less than GREAT), the output processing unit 32 outputs the sound of the musical performance part linked to the first note object until the press timing.The output processing unit 32 then outputs the sound of the musical performance part linked to the second note object at the press timing.

[0076] <3.2. Example 2 when the press timing level is less than GREAT> Furthermore, for a first note object and a second note object linked to the same musical performance part, if the output processing unit 32 receives a press of the first note object and then receives a press of the second note object a predetermined time or more after the placement timing of the second note object (a press timing at which the press timing level is less than GREAT), the output processing unit 32 outputs the sound of the first note object until the audio end time linked to the first note object.The output processing unit 32 then outputs the sound of the musical performance part linked to the second note object at the press timing of the second note object.

[0077] With these configurations, if the timing of pressing the note object is significantly different from the timing of placing the note object, a sound is output at a timing different from the sound source information, thereby motivating the user to press the note object at the right time.

[0078] <3.3. Example 1 when the press timing level is GREAT> Furthermore, for a first note object and a second note object linked to the same musical performance part, if the output processing unit 32 receives a press of the first note object and then a press of the second note object within a predetermined time period from before the placement timing of the second note object to the placement timing (a press timing at which the press timing level becomes GREAT), the output processing unit 32 outputs the sound of the musical performance part linked to the first note object until the placement timing of the second note object.The output processing unit 32 then outputs the sound of the musical performance part linked to the second note object at the placement timing of the second note object.

[0079] <3.4. Example 2 when the press timing level is GREAT> Furthermore, for a first note object and a second note object linked to the same musical performance part, if the output processing unit 32 receives a press of the first note object and then receives a press of the second note object within a predetermined time period from the placement timing of the second note object to the placement timing and after the placement timing (a press timing at which the press timing level becomes GREAT), the output processing unit 32 outputs the sound of the musical performance part linked to the first note object until the placement timing of the second note object, attenuates the sound of the musical performance part linked to the second note object until the press timing, and cancels the attenuation process at the press timing of the second note object and subsequently outputs the sound of the musical performance part linked to the second note object. In this case, specifically, the output processing unit 32 outputs the sound of the musical performance part linked to the first note object from after receiving a press of the first note object until the placement timing of the second note object, and attenuates the sound of the musical performance part linked to the second note object from the placement timing of the second note object to the press timing of the second note object. Then, the attenuation is released at the timing when the second note object is pressed, and the sound of the performed musical piece part linked to the second note object is output until the audio end time of the second note object.

[0080] With these configurations, even if the timing of pressing the note object is slightly off from the sound source information, the sounds can be connected and played back without any sense of incongruity.

[0081] In this embodiment, the sound output timing is adjusted when the press timing level is GREAT, but the sound output timing may also be adjusted arbitrarily for other press timing levels. For example, the sound output timing may be adjusted when the press timing level is GOOD or higher.

[0082] By executing the above steps S1 to S8, the user can generate music game data simply by uploading sound source information. Furthermore, in the present invention, by accurately pressing note objects, it is possible to reproduce sounds that are close to the sound source, which increases the user's motivation to play the music game.

[0083] In this embodiment, the music game system 0 installs a music game program in the user terminal device 3, and the music game data generation device 1 is used by the music game program, but some or all of the functional components (units) of the music game data generation device 1 may also be provided in the user terminal device 3, which is the client. [Explanation of symbols]

[0084] 0: Music Game System 1: Music game data generation device 2: Database 3: User terminal device 10: Server 101: Processing section 102: Storage section 103: Communications Department 9: Terminal device 91: Processing section 92: Storage section 93: Communications Department 94: Input section 95: Output section 11: Sound source acquisition section 12: Part separation section 13: Beat detection section 14: Onset detection section 15: Pitch detection section 16: Music notation generation section 17: Data compression section 18: Game data generation unit 19: Distribution Department 31: Data storage section 32: Output processing section 33: Judgment section 34: Input processing section NW: Communication network

Claims

1. A music game system that generates a musical score for a music game from a music source, the music game system includes a part separation unit, a beat detection unit, an onset detection unit, and a score generation unit; the part separation unit separates music parts based on the music sound source and acquires music part information; the beat detection unit detects beat information based on the music sound source; the onset detection unit detects onset information including a plurality of onset timings for each of the musical piece parts based on the musical piece part information; The music score generation unit sets a synchronization timing based on the detected beat timing and onset timing as a placement timing of the note object of the music game, and generates music score data using an onset timing that completely matches or does not completely match the beat timing among the plurality of onset timings as the synchronization timing. Music game system.

2. The musical score generating unit specifies the timing of arranging the note objects among the synchronization timings based on the difficulty level information. The music game system according to claim 1 .

3. the music game system further comprises a game data generation unit; The game data generation unit generates music game data in which the sounds of each of the musical part are linked to the note objects based on the musical part information and the score data. The music game system according to claim 1 .

4. The music game system further includes a pitch detection unit, the music game has a predetermined number of lanes; the pitch detection unit detects pitch information for each of the musical piece parts based on the musical piece part information; The musical score generating unit generates the musical score data in which the note objects that can distinguish the musical part are arranged in the lanes based on the pitch information. The music game system according to claim 1 .

5. the music game system further includes an output processing unit and a determination unit; the output processing unit receives a press of the note object and outputs a sound based on the music part information at the timing when the note object is pressed; the determination unit determines a press timing level based on the press timing of the note object; The output processing unit adjusts the output timing of the sound and outputs the sound when the pressing timing level is equal to or higher than a predetermined level. The music game system according to claim 1 .

6. A music game data generation program that generates a score for a music game from a music source, the music game data generation program causes a computer to function as a part separation unit, a beat detection unit, an onset detection unit, and a score generation unit; the part separation unit separates music parts based on the music sound source and acquires music part information; the beat detection unit detects beat information based on the music sound source; the onset detection unit detects onset information including a plurality of onset timings for each of the musical piece parts based on the musical piece part information; The music score generation unit sets a synchronization timing based on the detected beat timing and onset timing as a placement timing of the note object of the music game, and generates music score data using an onset timing that completely matches or does not completely match the beat timing among the plurality of onset timings as the synchronization timing. Music game data generation program.

7. A music game data generation method for generating a musical score for a music game from a music source, comprising: The computer A process of separating music parts based on the music sound source and acquiring music part information; A process of detecting beat information based on the music sound source; a process of detecting onset information including a plurality of onset timings for each of the musical piece parts based on the musical piece part information; a process of generating score data by setting a synchronization timing based on the detected beat timing and onset timing as a placement timing of a note object of the music game, and using an onset timing that perfectly matches or does not perfectly match the beat timing among the plurality of onset timings as the synchronization timing; A method for generating music game data.

Images