Music game system, music game data generation program, and music game data generation method

Abstract

The problem that this invention aims to solve is to provide a technology that can generate sheet music for music games that can reproduce songs through user operation. [Solution] To solve the above problems, the present invention provides a music game system for generating sheet music for a music game from a music sound source, wherein the music game system comprises a part separation unit, a beat detection unit, an onset detection unit, and a sheet music generation unit, wherein the part separation unit separates music parts based on the music sound source and acquires music part information, the beat detection unit detects beat information based on the music sound source, the onset detection unit detects onset information for each music part based on the music part information, and the sheet music generation unit generates sheet music data in which note objects for the music game are arranged based on the beat information and the onset information.

Description

Technical Field

[0001] The present invention relates to a music game system, a music game data generation program, and a music game data generation method.

Background Art

[0002] Conventionally, a technique for automatically generating musical scores from music sound sources by using voice analysis technology is known. An example of such an automatic score acquisition technique is proposed in, for example, Patent Document 1.

[0003] For example, Patent Document 1 describes receiving an audio signal, extracting frequency information from the audio signal, extracting amplitude information from the audio signal, extracting pitch information from the frequency information, generating a note start event from information on frequency and amplitude, determining the note length from the extracted frequency information and amplitude information, and the generated note start event, determining a rest, determining the tempo over a time window, determining the key, determining the instrument arrangement, and generating score display data using the note length, rest, tempo, key, and instrument arrangement.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in Patent Document 1, although it is possible to generate a score for playing a music piece from a music sound source, it is not possible to generate a score for a music game that allows a user to play the music piece alone.

[0006] In view of the above problems, an object of the present invention is to provide a technique capable of generating a score for a music game that can reproduce a music piece by a user's operation. [Means for solving the problem]

[0007] To solve the above problems, the present invention provides a music game system for generating sheet music for a music game from a music sound source, wherein the music game system comprises a part separation unit, a beat detection unit, an onset detection unit, and a sheet music generation unit, wherein the part separation unit separates music parts based on the music sound source and acquires music part information, the beat detection unit detects beat information based on the music sound source, the onset detection unit detects onset information for each music part based on the music part information, and the sheet music generation unit generates sheet music data in which note objects for the music game are arranged based on the beat information and the onset information.

[0008] This configuration allows for the generation of score data with note objects for a music game, using beat information and onset information obtained from a music sound source. This enables users to generate score data for their favorite music sound sources. Furthermore, in this invention, onset information is detected based on music part information obtained from the music sound source. If the music parts are separated after identifying the onsets, the timing of the onsets for each music part becomes inaccurate, impairing the gameplay. However, this invention achieves the remarkable effect of generating score data while maintaining gameplay by identifying the onsets after separating the music parts from the music sound source.

[0009] In a more preferred form, the music game system further comprises a game data generation unit, which generates music game data in which the sound of each song part is associated with the note object, based on the song part information and the score data.

[0010] This configuration allows for the output of sounds closer to the original audio source during music gameplay, enabling users to enjoy music games more.

[0011] In a more preferred embodiment, the music game system further comprises a pitch detection unit, the music game has a predetermined number of lanes, the pitch detection unit detects pitch information for each song part based on the song part information, and the score generation unit generates score data in which note objects that can distinguish the song parts are placed in the lanes based on the pitch information.

[0012] This configuration allows for the automatic generation of highly legible musical score data based on the type of song part and the pitch of each song part. This enables the placement of note objects that can identify song parts, allowing users to enjoy music games more.

[0013] In a more preferred form, the score generation unit determines the placement timing of the note objects based on the beat information and the onset information.

[0014] In a more preferred form, the score generation unit determines the synchronization timing based on the beat timing and the onset timing as the placement timing of the note objects.

[0015] This configuration allows the system to automatically determine the placement of note objects in accordance with the beat and onset derived from the music source. This enables users to play music games in sync with the rhythm and feel of the music source, making the music game more enjoyable.

[0016] In a more preferred form, the score generation unit further determines the placement timing of the note objects based on difficulty information.

[0017] This configuration allows the placement of note objects to be determined according to the difficulty level, enabling various users to enjoy the music game at a difficulty level that suits them.

[0018] In a more preferred form, the music game system further includes an output processing unit and a determination unit. The output processing unit receives the pressing of the note object and outputs a sound based on the music part information. The determination unit determines a pressing timing level based on the pressing timing of the note object, and the output processing unit adjusts the sound output timing and outputs a sound when the pressing timing level is equal to or higher than a predetermined level.

[0019] With such a configuration, a sound corresponding to the quality of the user's performance can be output, and the motivation for the user to play the music game can be increased.

Effect of the Invention

[0020] The present invention has an effect of providing a technique capable of generating a score for a music game in which a music piece can be reproduced by a user's operation.

Brief Description of the Drawings

[0021] [Figure 1] It is a block diagram showing the configuration of a system in an embodiment. [Figure 2] It is a hardware configuration diagram of a system in an embodiment. [Figure 3] It is a functional block diagram of a device in an embodiment. [Figure 4] It is a processing flowchart of a system in an embodiment. [Figure 5] It is an example of a method for specifying note timing in an embodiment. [Figure 6] It is an example of a game screen in an embodiment.

Mode for Carrying Out the Invention

[0022] Hereinafter, it will be described in more detail with reference to the accompanying drawings. Preferred embodiments are shown in the drawings. However, it can be implemented in many different forms and is not limited to the embodiments described in this specification.

[0023] For example, in this embodiment, the configuration and operation of a music game system will be described, but similar configurations, devices, computer programs, etc., can achieve similar effects. Furthermore, the program may be stored on a recording medium or provided downloadably from an external server. Using this recording medium, for example, the program can be installed on a computer, thereby configuring a music game data generation device and a music game system. Here, the recording medium storing the program may be a non-transient recording medium such as a CD-ROM.

[0024] This invention relates to a music game system for generating sheet music for music games. The invention accepts an uploaded music audio file and detects the beat of the music based on the music audio file. It also separates the music parts contained in the music audio file and detects the onset and pitch for each music part. Then, using the beat of the music, as well as the onset and pitch for each music part, it generates sheet music data in which music game note objects are placed.

[0025] In typical music games, the user selects one song from a database of multiple songs, and the game starts outputting sheet music data associated with that song, accepting button presses on note objects that move based on the sheet music data, and the overall score, calculated based on the timing of the button presses, is used to determine the winner.

[0026] In the music game of the present invention, when the user selects a song, the sound of a specific part of the song, separated from the sound source of the song, is output as background music (BGM), and the sound of a specific part is output when a note object is pressed. In this embodiment, the song part that is output as BGM is a rhythm part such as drums and bass, and the song part that is output when a note object is pressed is a melody or harmony part such as vocals, piano, and guitar, but is not limited to this. For example, if the song part of the sound source includes orchestral instruments or percussion instruments, the output can be changed as desired.

[0027] Furthermore, the music game in this invention has a predetermined number of lanes on the musical score, and accepts presses from the user for note objects that move along one lane to the other. In this embodiment, the predetermined number of lanes is 4, but there is no limit to the number.

[0028] <1. System Configuration> Figure 1 is a block diagram showing the configuration of a system in one embodiment. As shown in Figure 1, the music game system 0 comprises a music game data generation device 1 and a user terminal device 3. In this embodiment, the music game data generation device 1 and the user terminal device 3 are configured to communicate via a communication network NW. In this embodiment, the communication network NW is an IP (Internet Protocol) network, but there are no restrictions on the type of communication protocol, and furthermore, there are no restrictions on the type or size of the network.

[0029] In this embodiment, the music game data generation device 1 can be a general-purpose server computer or a personal computer. The user terminal device 3 can be a smartphone, tablet, personal computer, or wearable device. Furthermore, the music game data generation device 1 may be composed of multiple computers capable of sending and receiving information via a communication network NW or another network.

[0030] <1.1. Hardware Configuration of the Invention> Figure 2 is a hardware configuration diagram of the music game system 0. As shown in Figure 2(a), the server 10 (music game data generation device 1) comprises a processing unit 101, a storage unit 102, and a communication unit 103.

[0031] The processing unit 101 has one or more processors, such as a CPU, capable of executing instruction sets, and controls the entire operation process of the music game data generation device 1 by executing the music game data generation program, OS, and other applications according to the present invention. The memory unit 102 includes a volatile memory such as RAM capable of storing instruction sets, and a non-volatile recording medium such as an HDD or SSD capable of storing the OS and the music game data generation program according to the present invention. The communication unit 103 has a communication interface device with the communication network NW, and performs communication control with the communication network NW to input and output information.

[0032] As shown in Figure 2(b), the terminal device 9 (user terminal device 3) comprises a processing unit 91, a storage unit 92, a communication unit 93, an input unit 94, and an output unit 95.

[0033] The processing unit 91 has one or more processors, such as a CPU, capable of executing instruction sets, and controls the entire operation of the terminal device 9 by executing a music game program for playing music using the score generated by the music game data generation device 1, an OS, and other applications. The memory unit 92 includes volatile memory such as RAM that can store instruction sets, and non-volatile recording media such as HDDs or SSDs that can store the OS, etc. The communication unit 93 has a communication interface device for connecting to a network and performs communication control with the communication network NW to input and output information. The input unit 94 has an input device capable of input processing, such as a keyboard or a touch panel. The output unit 95 includes a display device capable of display processing, such as a display, and an audio device capable of outputting sound.

[0034] <1.2. System Functional Configuration> Figure 3 is a functional block diagram of the music game system 0 in this embodiment.

[0035] <1.2.1. Functional Configuration of Music Game Data Generation Device 1> As shown in Figure 3, the music game data generation device 1 comprises a sound source acquisition unit 11, a part separation unit 12, a beat detection unit 13, an onset detection unit 14, a pitch detection unit 15, a score generation unit 16, a data compression unit 17, a game data generation unit 18, a distribution unit 19, and a database 2. This is a music game data generation program stored in the storage unit 102 that is specifically implemented by the processing unit 101.

[0036] <1.2.2. Database 2> Database 2 stores the generated music game data. The music game data includes a music game data ID to uniquely identify the music game data, compressed part audio information, and sheet music data, and is registered in association with information that uniquely identifies the user who requested the generation of the music game data (user ID).

[0037] <1.2.2.1. Compressed Part Audio Information> Compressed part audio information is information about the audio of the music parts separated from the music source (music part information), with the data size reduced using well-known data compression techniques. Music part information includes background music part information and performance music part information. BGM track part information refers to information about the audio of the music part that is output as background music. For BGM track part information, the audio of the BGM track part is set for each BGM part ID, which is used to uniquely identify the BGM track part. The performance song part information refers to the audio of the song part that is output when a note object is pressed. Each performance song part is uniquely identified by a performance part ID, and a corresponding performance song part audio is assigned to that ID.

[0038] In this embodiment, compressed audio data is stored as the compressed audio part information, but uncompressed audio data may also be stored.

[0039] <1.2.2.2. Score Data> Score data is information about the sheet music of a generated song. The note placement information is set according to the difficulty level of the music game. Note placement information refers to information about note objects placed on the musical score. For each note object, the note placement information includes a performance part ID, an audio start time, and an audio end time.

[0040] <1.2.3. Sound source acquisition section 11> The sound source acquisition unit 11 acquires sound source information of a music sound source. The sound source acquisition unit 11 acquires sound source information by accepting an upload of sound source information from the user.

[0041] <1.2.4. Part Separation Section 12> The part separation unit 12 separates the music parts based on the sound source information and obtains the music part information. The part separation unit 12 separates the music parts from the obtained sound source information and obtains BGM music part information and performance music part information as music part information.

[0042] <1.2.5. Beat detection unit 13> The beat detection unit 13 detects the beat information of the song based on the sound source information. The beat information is information related to the beat of the song, and the combination of beat and beat timing (time) is detected. Here, a beat is a unit of rhythm that is marked at equal intervals.

[0043] <1.2.6. Onset detection unit 14> The onset detection unit 14 detects onset information based on the song part information. The onset detection unit 14 detects onset information based on the performance song part information from the acquired song part information. The onset information is information about the onset (the attack portion of the sound, that is, the moment when the sound suddenly occurs) of each performance song part, and the onset timing (speech time) for each performance song part is detected as onset information.

[0044] <1.2.7. Pitch detection unit 15> The pitch detection unit 15 detects pitch information based on the musical part information. The pitch detection unit 15 detects pitch information based on the performed musical part information from the acquired musical part information. The pitch information is information about the pitch (note height) of each performed musical part, and the combination of pitch and time for each performed musical part is detected as pitch information.

[0045] <1.2.8. Score Generation Section 16> The score generation unit 16 generates score data with note objects placed based on sound source information. The score generation unit 16 generates score data with note objects placed based on beat information and onset information.

[0046] <1.2.9. Data Compression Section 17> The data compression unit 17 compresses the audio data based on the song part information. The data compression unit 17 compresses the audio data of the song part information and generates compressed part audio information.

[0047] <1.2.10. Game Data Generation Unit 18> The game data generation unit 18 generates data to realize a music game in which the sound of a song part is output when the user presses a note object. Based on compressed part audio information and score data, the game data generation unit 18 generates music game data in which the sound of each song part is linked to a note object.

[0048] <1.2.11. Distribution Section 19> The distribution unit 19 distributes music game data stored in the database 2 to the user terminal device 3, and the data storage unit 31 stores the distributed music game data in the storage unit 92. The output processing unit 32 performs output processing based on the distributed music game data and transmits the output processing result to the output unit 95, so that various screens and sounds are output to the user terminal device 3.

[0049] <1.3. Functional Configuration of User Terminal Device 3> As shown in Figure 3, the user terminal device 3 comprises a data storage unit 31, an output processing unit 32, a determination unit 33, and an input processing unit 34. This is a concrete implementation of the music game program stored in the memory unit 92 by the processing unit 91.

[0050] <1.3.1. Output Processing Unit 32> The output processing unit 32 processes the output of the music game screen. The music game screen will be described later.

[0051] Furthermore, the output processing unit 32 receives a press of a note object and processes the output to produce sound. The output processing unit 32 receives a press of a note object from the input processing unit 34 and processes the output to produce sound based on the performance music part information associated with the note object.

[0052] <1.3.2. Judgment unit 33> The determination unit 33 determines the timing discrepancy of the note object's press. Based on the note placement information and the timing of the note object's press by the user, the determination unit 33 determines the press timing level. Specifically, the determination unit 33 detects the discrepancy between the timing at which the user presses the note object and the start time of the note object's audio, and determines a press timing level such as GREAT, GOOD, or BAD according to the magnitude of the discrepancy.

[0053] <2. Processing Flow> Next, with reference to Figures 4-6, a method for generating music game data using the music game system 0 according to the present invention will be described. Figure 4 is a flowchart showing the process from when the music game data generation device 1 acquires sound source information to when it generates music game data.

[0054] <2.1. Acquisition of sound source information> In step S1 (hereinafter, "step SX" will simply be referred to as "SX"), the sound source acquisition unit 11 acquires sound source information. In this embodiment, the sound source acquisition unit 11 accepts the upload of audio data such as WAV, MP3, and FLAC (Free Lossless Audio Codec) from the user as sound source information and acquires the sound source information.

[0055] <2.2. Obtaining song part information> In S2, the part separation unit 12 acquires song part information based on the sound source information. In this embodiment, the part separation unit 12 separates the song parts contained in the song sound source using well-known techniques (signal processing (frequency analysis and spectral analysis, etc.), machine learning, and sound source separation algorithms (non-negative matrix factorization and independent component analysis, etc.)) on the sound source information of the song acquired in S1. The part separation unit 12 then acquires BGM song part information, which includes the audio for each song part, and performance song part information as song part information.

[0056] <2.3. Obtaining Beat Information> In S3, the beat detection unit 13 detects beat information based on the sound source information. In this embodiment, the beat detection unit 13 uses well-known techniques (signal processing, machine learning, etc.) on the sound source information acquired in S1 to detect beat information, including a combination of the beat and beat timing of the sound source information. In this embodiment, the beat detection unit 13 detects beat information based on the sound source information, but it may also detect beat information based on BGM music part information.

[0057] <2.4. Detection of Onset Information> In S4, the onset detection unit 14 detects onset information based on the music part information. In this embodiment, the onset detection unit 14 detects onset information, including the onset timing for each music part, from the music part information acquired in S2, using a well-known method (onset detection algorithm (such as detection based on energy change, detection based on spectral difference, detection based on phase difference, and tempo-synchronized detection), machine learning, and combinations thereof).

[0058] <2.5. Pitch Information Detection> In S5, the pitch detection unit 15 detects pitch information based on the music part information. In this embodiment, the pitch detection unit 15 uses well-known methods (time-domain pitch detection (zero-crossing method, autocorrelation method, and AMDF method, etc.), frequency-domain pitch detection (Fourier transform, cepstrum method, and harmonic peak detection method, etc.), and machine learning, etc.) to detect pitch information, including combinations of pitch and time for each performed music part, from the music part information acquired in S2.

[0059] <2.6. Generating Score Data> In S6, the score generation unit 16 generates score data based on beat information, onset information, and pitch information. The score generation unit 16 identifies the placement timing of note objects based on at least the beat information and onset information. In this embodiment, the score generation unit 16 identifies the sound start time of the note object as the placement timing based on the beat information, onset information, and difficulty information. Specifically, the score generation unit 16 identifies the placement timing of note objects for each difficulty level based on the beat timing and onset timing. In this embodiment, the difficulty levels are, for example, hard, normal, and easy, but the number and notation of these levels are not limited to these.

[0060] Figure 5 is an example of an overview diagram of the process for identifying the placement timing of note objects based on beat information, onset information, and difficulty information. Figure 5(a) shows, from top to bottom, the sound source information acquired in S1, the song part information acquired in S2, and the onset information detected in S4.

[0061] Figure 5(b) shows, from top to bottom, the onset information detected in S4, the beat information detected in S3, and the note object placement timing. As shown in Figure 5(b), the score generation unit 16 identifies the synchronization timing based on the onset timing and beat timing as the note object placement timing. Specifically, the score generation unit 16 identifies the onset timing that perfectly or imperfectly matches the beat timing as the synchronization timing, and identifies this synchronization timing as the note object placement timing. Here, the onset timing that perfectly or imperfectly matches the beat timing refers to the timing where the onset timing and beat timing overlap, or the onset timing that is closest to the beat timing among multiple onset timings and beat timings.

[0062] Furthermore, if an onset appears after a predetermined period of time without an onset, the score generation unit 16 identifies the synchronization timing based on the timing of the onset's appearance. Specifically, the score generation unit 16 identifies the timing at which the onset appears and the beat timing are perfectly or imperfectly matched as the synchronization timing, and identifies this synchronization timing as the note object placement timing. For example, in Figure 5(b), an onset appears after a predetermined period of time without an onset (such as an intro or interlude), and the synchronization timing based on the timing of the first onset's appearance and the beat timing is identified as the placement timing.

[0063] Furthermore, the score generation unit 16 determines the placement timing so that there is one note object within a predetermined interval. Specifically, the score generation unit 16 determines the placement timing so that there is one note object within a predetermined interval according to the difficulty level. For example, in Figure 5(b), the placement timing is determined so that one note object is placed within a predetermined interval of two beats.

[0064] Furthermore, the score generation unit 16 identifies the placement lane for note objects based on pitch information. In this embodiment, the score generation unit 16 identifies the placement lane for note objects for each part of the musical performance so that note objects associated with the same pitch are placed in the same lane, and are arranged in order from highest to lowest pitch.

[0065] In a preferred embodiment of the present invention, for a note object in a musical performance part, the lane in which the note object is placed is determined based on the pitch associated with consecutively placed note objects and the number of lanes. Specifically, in this embodiment, since note objects are placed in order of increasing or decreasing pitch, depending on the pitch associated with consecutive note objects and the number of lanes, it may not be possible to place the note objects in order of increasing or decreasing pitch. In this case, the score generation unit 16 determines the lane in which the note object is placed based on the pitch associated with one or more note objects placed after the note object placed consecutively with the note object, where the pitch associated with the note object placed in each lane is not the same as the pitch associated with the note object placed in that lane.

[0066] Specifically, the score generation unit 16 identifies the placement lane for a note object by associating the pitches associated with consecutive note objects with the placement lanes in ascending or descending order from left to right, based on the pitches associated with the consecutive note objects, until the number of pitch types associated with consecutive note objects matches the number of lanes. If a pitch has been associated with all placement lanes, and there is a note object in the next sequence whose associated pitch does not match the pitch associated with the placed note object, the placement lane for that note object is identified by associating the pitches associated with consecutive note objects with the placement lanes in ascending or descending order from left to right, based on the pitches associated with the consecutive note objects, until the number of pitch types associated with the note objects after that note object matches the number of lanes, based on the pitches associated with the consecutive note objects.

[0067] For example, if there are four lanes and consecutive note objects are associated with pitches (international format) "A3", "C4", "F4", "A4", "C4", "G4", "E4", "B4", and "C5", then first, the pitch types are associated with the leftmost placement lane, in order from "A3", "C4", "F4", and "A4" (arrangement in order of increasing pitch from left to right). In this case, the placement lanes for the next consecutively placed note objects are determined based on the number of pitch types associated with the note objects after the one associated with a pitch (G4) that does not match an already placed pitch, and the corresponding pitches, so that the order from left to right is "E4", "G4", "B4", and "C5".

[0068] Furthermore, the score generation unit 16 arranges note objects that are distinguishable for each part of the music. In this embodiment, the score generation unit 16 arranges note objects that are distinguishable for each part of the music by associating them with a performance part ID, based on the performance part of the music associated with the note object. In a preferred embodiment of the present invention, note objects that are distinguishable for each part of the music are arranged in the same lane.

[0069] <2.7. Generation of compressed part audio information> In S7, the data compression unit 17 generates compressed part audio information based on the music part information. In this embodiment, the data compression unit 17 uses a well-known data compression technique on the music part information acquired in S2 to generate compressed part audio information by compressing the audio data of the background music part and the performance music part for each music part.

[0070] <2.8. Generating Music Game Data> In S8, the game data generation unit 18 generates music game data in which the sound of each song part is linked to a note object, based on the compressed part audio information and score data. In this embodiment, the game data generation unit 18 generates music game data for a music game in which the compressed performance song part audio is linked to a note object, based on the compressed part audio information generated in S7 and the score data generated in S6, and the compressed BGM song part information and performance song part information (compressed part audio information) for each song part, and the placement timing of the note placement information (audio start time). The game data generation unit 18 then registers the music game data in the database 2.

[0071] <3. Example of a music game screen> Figure 6 shows an example of a music game screen displayed on the user terminal device 3 based on music game data distributed by the distribution unit 19. In the illustrated example, instrument note objects INT and vocal note objects VNT are placed as note objects for each part of the song being played, and four lanes L1 to L4, a beat bar BB, a timing bar TB, and a button press receiver PI are displayed.

[0072] In this embodiment, the output processing unit 32 outputs note objects (instrument note object INT and vocal note object VNT arranged in four lanes L1 to L4 in the illustrated example) which have different configurations for each song part, so that they move from the top to the bottom of the screen. The press reception unit PI corresponding to each lane L1 to L4 receives a press from the user, thereby accepting the press of the note object in the corresponding lane. The closer the press is received when the center of the note object is near the timing bar TB, the higher the press timing level becomes, and a score corresponding to the press timing level is added.

[0073] Furthermore, the output processing unit 32 outputs BGM music part audio based on BGM music part information, and also receives a press on a note object, outputting the performance music part audio based on the press timing level and the performance music part information associated with the note object.

[0074] In this embodiment, the output processing unit 32, with respect to the first note object and the second note object associated with the same musical performance part, receives a press on the first note object and then outputs the sound of the musical performance part associated with the first note object based on the placement timing of the second note object and the press timing level of the second note object. The output processing unit 32 then outputs the sound of the musical performance part associated with the second note object based on the press timing level of the second note object.

[0075] <3.1. Example 1 of a case where the press timing level is less than GREAT> Specifically, the output processing unit 32, for first and second note objects associated with the same musical performance part, after receiving a press on the first note object, if it receives a press on the second note object more than a predetermined time before the placement timing of the second note object (a press timing where the press timing level is less than GREAT), outputs the sound of the musical performance part associated with the first note object up to the timing of the press. Then, at the timing of the press, the output processing unit 32 outputs the sound of the musical performance part associated with the second note object.

[0076] <3.2. Example 2 of cases where the press timing level is less than GREAT> Furthermore, for first and second note objects associated with the same musical performance part, the output processing unit 32, after receiving a press on the first note object, and if the press on the second note object is received more than a predetermined time after the placement timing of the second note object (a press timing where the press timing level is less than GREAT), outputs the sound of the first note object until the end time of the sound associated with the first note object. Then, at the timing of the press of the second note object, the output processing unit 32 outputs the sound of the musical performance part associated with the second note object.

[0077] With these configurations, if the timing of pressing a note object is significantly different from the timing of placing the note object, a sound will be output at a different timing than the sound source information, which can motivate the user to press the note object at the right time.

[0078] <3.3. Example 1 when the press timing level is GREAT> Furthermore, for the first and second note objects associated with the same musical performance part, the output processing unit 32, after receiving a press on the first note object, if it receives a press on the second note object within a predetermined time from before the placement timing of the second note object to the placement timing (a press timing where the press timing level is GREAT), outputs the sound of the musical performance part associated with the first note object until the placement timing of the second note object. Then, at the placement timing of the second note object, the output processing unit 32 outputs the sound of the musical performance part associated with the second note object.

[0079] <3.4. Example 2 when the press timing level is GREAT> Furthermore, with respect to the first and second note objects associated with the same musical performance part, the output processing unit 32, after receiving a press on the first note object, and then receiving a press on the second note object within a predetermined time period from the placement timing of the second note object to the press timing at which the press timing level becomes GREAT, outputs the sound of the musical performance part associated with the first note object until the placement timing of the second note object, then outputs the sound of the musical performance part associated with the second note object while attenuating it until the timing of the press, and then releases the attenuation process at the timing of the press of the second note object and outputs the sound of the musical performance part associated with the second note object thereafter. Specifically in this case, the output processing unit 32 outputs the sound of the musical performance part associated with the first note object from after receiving a press on the first note object until the placement timing of the second note object, and also outputs the sound of the musical performance part associated with the second note object while attenuating it from the placement timing of the second note object until the timing of the press of the second note object. Then, at the timing of the second note object's press, the attenuation is released, and the sound of the performance music part associated with the second note object is output until the end of the second note object's audio.

[0080] These configurations allow for seamless playback of sounds even if the timing of note object presses is slightly out of sync with the sound source information.

[0081] In this embodiment, the sound output timing is adjusted when the button press timing level is GREAT, but the sound output timing may also be adjusted arbitrarily at other button press timing levels. For example, the sound output timing may be adjusted when the button press timing level is GOOD or higher.

[0082] As described above, by executing processes S1 to S8, the user can generate music game data simply by uploading sound source information. Furthermore, in this invention, by accurately pressing note objects, it is possible to reproduce sounds close to the sound source, thereby increasing the user's motivation to play the music game.

[0083] In this embodiment, the music game system 0 installs a music game program on the user terminal device 3, and the music game program utilizes the music game data generation device 1. However, some or all of the functional components (parts) of the music game data generation device 1 may be provided on the user terminal device 3, which is the client. [Explanation of symbols]

[0084] 0: Music game system 1: Music game data generation device 2: Database 3: User terminal device 10: Server 101: Processing Unit 102: Storage section 103: Communications Department 9: Terminal device 91: Processing Unit 92: Storage section 93: Communications Department 94: Input section 95: Output section 11: Sound source acquisition section 12: Part separation section 13: Beat detection unit 14: Onset detection unit 15: Pitch detection unit 16: Score Generation Section 17: Data Compression Section 18: Game Data Generation Unit 19: Distribution Department 31: Data storage unit 32: Output Processing Unit 33: Judgment section 34: Input Processing Unit NW: Communication Network

Claims

1. A music game system that generates sheet music for music games from music audio sources, The aforementioned music game system comprises a part separation unit, a beat detection unit, an onset detection unit, and a score generation unit. The part separation unit separates the music parts based on the music sound source and obtains music part information. The beat detection unit detects beat information based on the music sound source, The onset detection unit detects onset information for each song part based on the song part information, The score generation unit generates score data in which note objects from the music game are placed based on the beat information and the onset information. Music game system.

2. The aforementioned music game system further comprises a game data generation unit, The game data generation unit generates music game data in which the sound of each song part is associated with the note object, based on the song part information and the score data. The music game system according to claim 1.

3. The aforementioned music game system further comprises a pitch detection unit, The aforementioned music game has a predetermined number of lanes, The pitch detection unit detects pitch information for each song part based on the song part information, The score generation unit generates the score data in which note objects that can distinguish the musical parts are placed in the lanes based on the pitch information. The music game system according to claim 1.

4. The score generation unit determines the placement timing of the note objects based on the beat information and the onset information. The music game system according to claim 1.

5. The score generation unit determines the synchronization timing based on the beat timing and the onset timing as the placement timing of the note objects. The music game system according to claim 4.

6. The aforementioned score generation unit further determines the placement timing of the note objects based on the difficulty information. The music game system according to claim 4 or 5.

7. The aforementioned music game system further comprises an output processing unit and a judgment unit, The output processing unit receives the press of the note object and outputs a sound based on the music part information. The determination unit determines the press timing level based on the press timing of the note object, The output processing unit adjusts the sound output timing and outputs sound when the press timing level is above a predetermined level. The music game system according to claim 1.

8. A music game data generation program that generates sheet music for music games from music audio sources, The aforementioned music game data generation program causes the computer to function as a part separation unit, a beat detection unit, an onset detection unit, and a score generation unit. The part separation unit separates the music parts based on the music sound source and obtains music part information. The beat detection unit detects beat information based on the music sound source, The onset detection unit detects onset information for each song part based on the song part information, The score generation unit generates score data in which note objects from the music game are placed based on the beat information and the onset information. A music game data generation program.

9. A method for generating music game data, which generates sheet music for music games from music audio sources, Computers A process to separate the song parts based on the aforementioned song audio source and obtain song part information, A process for detecting beat information based on the aforementioned music audio source, A process to detect onset information for each song part based on the aforementioned song part information, A process for generating score data in which note objects of the music game are placed based on the beat information and the onset information. A method for generating music game data to execute.

Images