System and method for generating MIDI-based multi-participant content

Abstract

The purpose of the present invention is to provide a system and method for generating MIDI-based multi-participant content, wherein, when a plurality of users communicate with a server via user terminals of the respective users, designate singing attributes of the respective users, with respect to a piece of music selected over a network, in different spaces at different times, and transmit voice and video data from the singing to the server, the server matches MIDI music with the voice and video data of each of the users according to the singing attributes designated and configured by each of the users, thereby creating and servicing high-quality choral music content appropriately including solo parts and choral parts of the respective users, and allows new users to participate in the content over the network and create other pieces of choral music content, thus making it possible for a large number of users to participate in the creation of high-quality choral music content.

Description

Multi-party participation MIDI-based content creation system and method

[0001] The present invention relates to a system and method for generating multi-participatory MIDI-based content, which allows a plurality of users to access a server through their user terminals and transmit a video of themselves singing a song based on the MIDI standard and the appearance of the singing, and the server to generate and provide MIDI-based content in which a plurality of users have participated using data transmitted from each user terminal.

[0002] Nowadays, people are not only listening to high-quality music such as MIDI standard music, but also showing off their singing skills by singing along to the latest popular songs.

[0003] In the past, people sang along to the accompaniment of a song using a karaoke machine installed in a karaoke room, but recently, with the advancement of technology for personal portable communication devices such as smartphones, high-quality video and audio recording are now possible with personal portable communication devices such as smartphones, and as the functions of social networks have expanded, people are using their smartphones to create various music-related contents that reflect their individuality and releasing them through social networks.

[0004] Through social networks, people share their daily lives through videos, or videos of themselves singing or dancing. Furthermore, friends who have become close through social networks gather together to create and share content related to songs, dances, etc.

[0005] There is a significant need for people to create content through social networks, where an unspecified number of users can participate in the creation of content together and build a sense of empathy among them. However, there is a significant lack of such programs and systematic support.

[0006] Especially in the case of singing, there is a strong need for people to form a sense of empathy among the participants by having anyone who wants to participate and sing together. However, in the past, such choral content was limited to simply combining the voices recorded by multiple users to create content as a sing-along rather than a choral song.

[0007] However, since the content created as a chorus like this loses the meaning of each user's individuality participating in the chorus, it is meaningful to create a high-quality multi-participation chorus content by appropriately including parts where each participant sings solo, parts where some users sing duet, and parts where all users sing together. However, in the past, there was no technology to service the production of such high-quality multi-participation chorus content.

[0008] [Prior Art Literature]

[0009] Patent Publication No. 10-2014-0056522

[0010] Patent Publication No. 10-2021-0024774

[0011] Patent Publication No. 10-2016-0028016

[0012] Patent Publication No. 10-2001-0082593

[0013] The present invention provides a multi-participatory MIDI-based content creation system and method, in which a plurality of users communicate with a server through their respective user terminals, and each of them selects a song through a network at different times and in different spaces, specifies singing properties for his or her own singing, and transmits voice and video data according to the singing to the server, and the server creates and services content of a high-quality choral song that appropriately includes solo parts and choral parts of each of the plurality of users by matching MIDI music with the voice data and video data of each of the plurality of users according to the singing properties designated and set by each of the plurality of users, and allows new users to participate on the network to create content of another choral song, thereby enabling a plurality of users to participate and create content of a high-quality choral song.

[0014] A method for generating multi-participatory MIDI-based content according to one embodiment of the present invention comprises: a step of dividing a song into a plurality of parts based on lyrics of the song and setting them; a step of allowing a user who has selected the song to designate singing properties for each of the divided parts of the song; a step of transmitting voice data and video data of a singing appearance captured by a camera module of the user terminal to the server as the user sings the song through his / her user terminal; a step of allowing the server to combine the MIDI data of the song and the transmitted data to generate seed content; a step of allowing a participant who has selected the seed content to designate singing properties for each of the divided parts of the song; a step of allowing voice data and video data of the participant's singing appearance to be transmitted to the server as the participant sings the song through his / her user terminal; And the server includes a step of generating MIDI-based choral content by matching voice data and video data for a part in which the participant participates according to the singing properties specified by the user and the singing properties specified by the participant in the seed content.

[0015] Also preferably, the singing property includes a solo property in which the user sings solo and a chorus property in which a plurality of participating users sing together, and the step of specifying the singing property for each of the divided parts includes a step of specifying one of the solo property and the chorus property for each of the divided parts.

[0016] Also preferably, the step of generating the choir content comprises: a step of aligning each corresponding part of voice data of a user or participant who has designated the solo attribute for each corresponding section among a plurality of parts of the song with MIDI data for the song, and generating a video of the user or participant corresponding to the voice data of each solo attribute as a single-screen video for each corresponding section; and a step of combining voice data of each of the participating users among the plurality of users with respect to a part with the choir attribute among a plurality of parts of the song, aligning the corresponding part part with MIDI data for the song, and generating a video of the participating users as a split-screen video, so that in the parts with the solo attribute designated for the song, the voice and video of each of the users who have designated the corresponding attribute are outputted, and in the parts with the choir attribute designated, the voice data of the users who have designated the corresponding attribute are combined, and the voices of the users who have designated the corresponding attribute and the video of the users singing together are outputted as a split-screen video.

[0017] Also preferably, the singing property includes a solo property in which the user sings solo, a solo lock property that locks the solo property so that only the designated user can sing the part solo, and a chorus property in which a plurality of participating users sing together, and the step of designating a singing property for each of the divided parts of the song by the user includes a step of selectively designating one or more of the solo property, the solo lock property, and the chorus property for each of the divided parts of the song by the user through his / her user terminal and storing the designation as the designated singing property information of the user on the server, and the step of designating a singing property for each of the divided parts of the song by the participant includes a step of selectively designating one or more of the solo property, the solo lock property, and the chorus property for each of the solo property and chorus property parts designated by the user through his / her user terminal and storing the designation as the designated singing property information of the participant on the server.

[0018] Also preferably, the step of generating the choir content is characterized by including a step of extracting and replacing voice data of the user corresponding to a section of a part designated as a solo attribute and a solo lock attribute by the participant among the solo attribute parts of the user in the seed content with voice data of the participant in the corresponding section, extracting the video data of the user in the corresponding section and replacing it with a single-screen video, extracting the voice data of the participant corresponding to a section of the part designated as the choir attribute and combining it with the voice data of the user in the corresponding section to generate a choir voice, and generating video data of each of the user and the participant corresponding to a section of the part designated as the choir attribute as a split-screen video, thereby generating the choir content.

[0019] In addition, preferably, the generated choral content is characterized in that a new participant can select the generated choral content, designate new singing properties, and match the voice data and video data of the new participant accordingly, thereby generating new choral content in which the new participant participates.

[0020] In addition, preferably, the method further comprises a step of designating and setting one or more singing attributes among a solo attribute, a solo lock attribute, and a choir attribute for a plurality of divided parts of the song by an nth user, a step of transmitting voice data according to the singing of the nth user and video data of the singing appearance of the nth user to the server according to MIDI music for the song, a step of extracting the voice data of the nth user corresponding to a section of a part designated as a solo attribute and a solo lock attribute by the nth user among solo attribute parts of previous users in the content of the choir song and replacing it with a portion of the voice data of the previous user, extracting the video data of the nth user for the corresponding section and extracting it as a single-screen video and replacing it with the video data of the previous user, extracting the voice data of the nth user corresponding to a section of a part designated as the choir attribute and combining it with the voice data of previous users for the corresponding section to generate a choir voice, and a step of generating the content of another choir song by generating video data of each of the previous users and the nth user corresponding to a section of a part designated as the choir attribute as a split-screen video.

[0021] Meanwhile, a multi-participation MIDI-based content generation method according to another embodiment of the present invention is a multi-participation MIDI-based content generation method provided by a server, wherein a plurality of users communicate with a server through their respective user terminals, the multi-participation MIDI-based content generation method comprising: a step of designating and setting a singing attribute of a form desired by the first user for each of a plurality of parts divided based on lyrics of a song selected from a user terminal of a first user; a step of transmitting voice data according to the singing of the first user and image data of the singing appearance of the first user acquired by a camera module of the user terminal of the first user to the server while MIDI music according to the MIDI data of the selected song is played from the user terminal of the first user; a step of the server generating first content, which is content in which the MIDI music for the selected song, the voice and image of the first user are combined or are combined with previously generated content, using data transmitted from the user terminal of the first user according to the singing attribute designated by the first user; A step of designating and setting a singing attribute in a form desired by the second user for each of the plurality of parts of the song of the first content in the user terminal of the second user; a step of transmitting voice data according to the singing of the second user and image data of the singing appearance of the second user acquired by a camera module of the user terminal of the second user to the server while MIDI music according to the MIDI data of the song of the first content is played in the user terminal of the second user; and a step of generating second content, which is content of a choral song through collaboration of multiple users including the first user and the second user, by combining the data transmitted by the user terminal of the second user by the server with the first content according to the singing attribute designated by the second user.

[0022] Also preferably, the singing property includes a solo property in which the user sings solo, a solo lock property that locks the solo property so that only the designated user can sing the part solo, and a chorus property in which a plurality of participating users sing together, and the step of generating the first content includes a step of extracting voice data of the first user corresponding to a section of a part designated as the solo property, the solo lock property, and the chorus property according to the singing property designated by the first user, matching it with the MIDI data, extracting video data of the first user of the corresponding section, and generating it as a single-screen video, thereby generating the first content, and the step of generating the second content includes a step of replacing the voice data of the first user corresponding to a section of a part designated as the solo property and the solo lock property by the second user among the solo property parts of the first user in the first content by extracting the voice data of the second user of the corresponding section, replacing the video data of the first user of the corresponding section with the video data of the second user, and extracting the voice data of the second user corresponding to a section of a part designated as the chorus property, thereby generating the first content. It is characterized by including a step of generating the second content by combining voice data to generate a choir voice and generating video data of each of the first user and the second user corresponding to a section of a part designated as the choir property as a split-screen video.

[0023] Also preferably, a step of designating and setting a property desired by the third user among a solo property in which the user sings solo for each of the plurality of parts of the song of the second content, a solo lock property in which the solo property is locked so that only the designated user can sing the corresponding part, and a chorus property in which a plurality of participating users sing together, and a step of transmitting voice data according to the singing of the third user and video data of the singing appearance of the third user to the server according to the MIDI music for the song, and extracting the voice data of the third user corresponding to a section of a part designated as a solo property and a solo lock property by the third user among the solo property parts of the previous user in the second content and replacing it with a portion of the voice data of the previous user, extracting the video data of the third user for the corresponding section and extracting it as a single-screen video and replacing it with the video data of the previous user, extracting the voice data of the third user corresponding to a section of the part designated as the chorus property and combining it with the voice data of the previous users for the corresponding section to generate a chorus voice, and generating a chorus voice by combining the voice data of the previous users corresponding to the section of the part designated as the chorus property and the It is characterized by further including a step of creating a third content, which is another choir, by creating a split-screen video of each of the third users' video data.

[0024] Meanwhile, a multi-participation MIDI-based content creation system according to one embodiment of the present invention is a multi-participation MIDI-based content creation system provided by a server, in which a plurality of users communicate with a server through their respective user terminals, wherein the user terminal is configured to play MIDI music according to MIDI data and collect voice data according to a user's singing, and divides a song selected from among a plurality of songs into a plurality of parts based on lyrics and displays the song so that the user can designate a singing attribute for each of the divided parts, and collects voice data according to the user's singing and image data of the user's singing appearance acquired by the camera module while the MIDI music of the song is played, and transmits the collected data to the server, and the server generates first content, which is content in which MIDI music for the song, the voice and image of the user are combined or combined with previously generated content, by using voice data and image data transmitted from the user terminal of the user of the user according to the singing attribute designated by the user for each of the plurality of divided parts of the song, and generates a plurality of first content for the song. For each of the divided parts, voice data and video data transmitted from the user terminal of the next user according to the singing properties specified by the next user are combined with the first content to generate second content, which is content of a choral song through collaboration between multiple users including the first user and the next user.

[0025] Also preferably, the singing property includes a solo property in which the user sings solo, a solo lock property that locks the solo property so that only the designated user can sing the corresponding part solo, and a chorus property in which a plurality of participating users sing together, and the server extracts voice data of the user corresponding to a section of a part designated as a solo property, a solo lock property, and a chorus property according to the singing property designated by the user, matches the MIDI data, extracts video data of the user of the corresponding section, and generates a single-screen video, thereby generating the first content, and replaces the voice data of the user corresponding to a section of a part designated as a solo property and a solo lock property by the next user among the solo property parts of the user in the first content by extracting the voice data of the next user of the corresponding section, and replaces the video data of the user of the corresponding section with the video data of the next user by extracting the video data of the next user, and extracts the voice data of the next user corresponding to a section of a part designated as the chorus property and combines it with the voice data of the user of the corresponding section to generate a chorus voice, and generates the chorus voice by combining the voice data of the user corresponding to a section of a part designated as the chorus property with the voice data of the user of the corresponding section. It is characterized in that the second content is generated by generating the video data of each user and the next user as a split-screen video.

[0026] The multi-participatory MIDI-based content creation system and method according to the present invention allows a plurality of users to communicate with a server through their respective user terminals, and each of them to specify singing properties for his or her own singing of a song selected through a network at different times and in different spaces, and to transmit voice and video data according to the singing to the server, and the server, in accordance with the singing properties specified and set by each of the plurality of users, matches MIDI music with the voice data and video data of each user to create and serve content of a high-quality choral song that appropriately includes solo parts and choral parts of each of the plurality of users, and allows a new user to participate in the network to create content of another choral song for the content, thereby enabling a large number of users to participate in creating content of a high-quality choral song.

[0027] FIG. 1 is a block diagram illustrating the configuration of a multi-participatory MIDI-based content creation system according to one embodiment of the present invention.

[0028] FIG. 2 is a flowchart illustrating a content creation method by a multi-participatory MIDI-based content creation system according to one embodiment of the present invention having a configuration as illustrated in FIG. 1.

[0029] FIGS. 3 to 7 are drawings for more specifically explaining the process from designation of a user's singing properties to creation of multi-participatory content according to a content creation method by a multi-participatory MIDI-based content creation system according to an embodiment of the present invention illustrated in FIG. 2.

[0030] FIGS. 8 to 11 are drawings for more specifically explaining the creation of seed content by a first user, the creation of choral content by a second user's participation in the seed content, and the creation of another new choral content by a third user's participation in the choral content, according to a multi-participatory MIDI-based content creation method according to one embodiment of the present invention.

[0031] The multi-participatory MIDI-based content creation system and method according to the present invention will be described in detail with reference to the drawings below.

[0032] First, a multi-participatory MIDI-based content creation system according to an embodiment of the present invention will be described with reference to FIG. 1. FIG. 1 is a block diagram illustrating the configuration of a multi-participatory MIDI-based content creation system according to an embodiment of the present invention.

[0033] As illustrated in FIG. 1, a multi-participatory MIDI-based content creation system according to one embodiment of the present invention is a system in which a plurality of user terminals (#1, #2...#n) and a server (200) communicate and form a network, and is basically configured such that the server (200) processes data collected from user terminals and transmitted to the server (200) to create predetermined content.

[0034] A multi-participatory MIDI-based content creation system according to one embodiment of the present invention is a system in which multiple users participate to create collaborative choral content. This system is completely different from a conventional method in which multiple users participating in choral content creation each transmit all of their voices and other related data to a server, and the server uses all of the related data to create a choral content.

[0035] That is, a multi-participatory MIDI-based content creation system according to one embodiment of the present invention is a system that creates multi-participatory choral content in such a way that the first user transmits voice and video data based on his / her singing to a server based on MIDI music for a specific song, the server uses this to create seed content, and then, each time a new user participates, data based on the singing of the new participant is added, thereby continuously creating new content.

[0036] At this time, the multi-participation MIDI-based content creation system according to one embodiment of the present invention is configured to create a high-quality choral content each time a user participates by dividing the original song required for content creation into a number of parts based on lyrics rather than simply combining the data of a new participant with the existing content, and then specifying a unique singing attribute for each of the divided parts so that the necessary part of the data of the new participant is matched to the existing content according to the singing attribute, thereby creating the content.

[0037] In a multi-participatory MIDI-based content creation system according to one embodiment of the present invention, a user terminal (110) possessed by each of a plurality of users may be configured to include a display (112) for displaying a song list, etc. on a screen, a camera module (113) for photographing the user's singing, a MIDI module (114) for implementing a software synthesizer and providing MIDI music by MIDI data based on a sound font, a recording module (115) for recording the user's voice, and a control unit (111) for controlling the above-mentioned components and transmitting the collected data to a server.

[0038] Although not shown in the drawing, the user terminal may basically include a speaker for outputting music and a communication module for communicating with the server.

[0039] The user terminal (110) is connected to a network, for example, the Internet, so as to be able to communicate with the server (200). When the user terminal (110) receives and displays information on a list of songs selectable for content creation from the server (200), the user can select a desired song from the list of songs, and the song selected by the user is divided into a plurality of parts based on the lyrics of the song and displayed, and the user can specify a singing attribute for each of the plurality of divided parts of the song and transmit the same to the server.

[0040] The user sings along with the MIDI music of the song implemented by the MIDI module (114) of the user terminal (110), and accordingly, the recording module (115) collects voice data according to the user's singing, and the camera module (113) photographs the user's singing to collect video data according to the singing and transmits it to the server.

[0041] The server (200) may be configured to include a user DB (210), a content creation processing unit (220), a voice data management unit (230), a video data management unit (240), a MIDI data DB (250), a lyric data DB (260), etc., as illustrated in FIG. 1.

[0042] The user DB (210) stores information about the user of the user terminal connected to the server (200), and when a user terminal of a user is connected to the server, the server receives information about the user from the user terminal and confirms it in the user DB (210), thereby recognizing the user of the user terminal.

[0043] The voice data management unit (230) stores the user's voice data transmitted from the user terminal and manages voice data for each section of a plurality of divided parts for the selected song.

[0044] The video data management unit (240) stores the video data of the user transmitted from the user terminal and manages the video data of each section of a plurality of divided parts for the selected song.

[0045] The MIDI data DB (250) stores and manages MIDI data for each song that can be selected for content creation.

[0046] The lyrics data DB (260) stores and manages lyrics information for each song that can be selected for content production and lyrics information corresponding to each of the multiple parts of each song.

[0047] When the content creation processing unit (220) transmits the relevant data for content creation from the user terminal to the server (200), the voice data of the corresponding user is managed by the voice data management unit (230) and the video data of the corresponding user is managed by the video data management unit (240). The content creation processing unit (220) checks the singing properties specified by the corresponding user, extracts voice data and video data corresponding to the section where the voice and video of the corresponding user are matched according to the singing properties, and uses this to create MIDI-based content.

[0048] A method for creating content using a multi-participatory MIDI-based content creation system according to one embodiment of the present invention having the configuration described above is described through the flowchart shown in FIG. 2.

[0049] FIG. 2 illustrates a process in which first content is created by a first user using a multi-participatory MIDI-based content creation system according to one embodiment of the present invention, and then data by a second user is added to the first content to create second content.

[0050] First, the server can provide information on a list of selectable songs and lyric data for each song to the user terminal of the first user (S310), and the first user can select one song from the list of selectable songs provided by the server through his / her user terminal (S110).

[0051] The user terminal of the first user can display multiple parts divided based on the lyrics of the selected song (S120), and the first user can specify his / her own singing properties for each of the multiple divided parts (S130).

[0052] In this regard, referring to FIG. 3, (a) of FIG. 3 shows a state in which a plurality of parts (105) divided based on the lyrics of a song selected by a user are displayed on the display screen (102) of a user terminal, and (b) of FIG. 3 shows a state in which a singing attribute is specified for each part in the state shown in (a) of FIG. 3.

[0053] As shown in (a) of Fig. 3, a plurality of divided parts (105) for a selected song may be set as one part, for example, a lyric portion corresponding to one verse in the entire lyrics for the song.

[0054] Dividing a song into multiple parts is preset on the server, and each divided part is divided into sections, for example, each part is distinguished in detail based on the sound data of a MIDI note, and the section information of each part, that is, information about the start and end positions of each part, is preset on the server.

[0055] For each of the multiple divided parts of the song, a singing attribute preset by the user can be designated. As shown in (a) of FIG. 3, the singing attribute can include a solo attribute (Ps) in which the user sings solo, a solo lock attribute (PLs) in which the solo attribute is locked so that only the designated user can sing solo, and a chorus attribute (Ph) in which multiple participating users sing together.

[0056] In (a) and (b) of FIG. 3, the solo property (Ps) is indicated as “I” to indicate that it is a property sung by the user himself, the solo lock property (PLs) is indicated as “Preserve my solo part”, and the chorus property (Ph) is indicated as “Together”.

[0057] Users can select desired singing properties (Ps, PLs, Ph) and designate parts for which the properties are specified. For example, as shown in (b) of Fig. 3, the first four parts can be designated as solo properties (Ps), the next five parts can be designated as solo lock properties (PLs), and the next three parts can be designated as chorus properties (Ph).

[0058] As described above, information on the specified singing properties is stored as singing property information of the user possessing the corresponding user terminal on the server and is used as a reference for editing data when producing content in the future.

[0059] In FIG. 3, the user-specified singing properties are described as having three properties: a solo property, a solo lock property, and a chorus property. However, the present invention is not limited thereto, and two properties can be specified, or four or more properties can be specified.

[0060] For example, the singing property can be set to only two properties, solo property and choir property, without the above-mentioned solo lock property, or can be set to four properties, solo property, solo lock property, choir property, and choir lock property (here, 'choir lock property' means fixing the choir property so that other users cannot change the property).

[0061] The content creation system and method according to the present invention basically have a configuration in which a plurality of users participate one by one and sequentially create new content, and rather than creating a simple chorus song in which the voices of the plurality of users are combined, the purpose is to create a high-quality chorus song that appropriately includes a solo part and a chorus part in which all users sing together so that each user can express his or her individuality. In this regard, the singing properties are designated as described above, and only the necessary parts of the voices and images of each participating user are appropriately extracted and matched according to the designated singing properties, and for that result, it is possible to determine whether the singing properties are set to two, three, or four as described above.

[0062] If the singing attribute is set to two types, 'solo attribute' and 'chorus attribute', then each time a new user is added after the seed content created by the first user, the previously specified singing attributes may be completely changed to suit the user. In severe cases, a greedy user may change all singing attributes to their own solo part and designate them. However, since each user can freely designate the singing attribute that suits them and create content, this may be the method that best reveals the user's individuality.

[0063] If the singing attribute is set to one of four types: 'solo attribute', 'solo lock attribute', 'choir attribute', and 'choir lock attribute', there may not be many parts left for the next user to specify the singing attribute for as more users are added after the first user. This may limit the participation of many users in the content, but it may be a method that can definitely guarantee the contribution of the users who participated.

[0064] What singing attributes are included in the designation may change depending on the nature or characteristics of the choral content that users will create.

[0065] Meanwhile, returning to FIG. 2, if a singing attribute is designated by the first user for each of the multiple parts of the song selected in the above manner, this can be transmitted to the server (S140) and stored in the server as the singing attribute of the first user (S320).

[0066] After specifying the singing properties as described above, the first user can start singing along to the MIDI music of the song by using the functions of the MIDI module, recording module, and camera module of the user terminal (S150), and the user terminal collects the voice data of the first user singing along to the MIDI music and the video data of the first user singing and transmits them to the server (S160), and the server can receive and store the voice data of the first user and the video data of the first user, respectively (S330).

[0067] And, the server can generate first content combining MIDI music for the song, the first user's voice, and a video according to the singing properties specified by the first user (S340).

[0068] If the first content mentioned above is the first content created by the first user, then the content only contains the voice and video of the first user, and such content is called 'seed content' in the sense that it serves as a seed for new content to be created by other users in the future.

[0069] Since the multi-participatory MIDI-based content generated by the system and method according to one embodiment of the present invention is a method of creating new multi-participatory content for each subsequent user based on the initial seed content, the first content may be a completed content in which the voice and video of the first user are combined with the seed content according to the singing properties designated by the user.

[0070] In Fig. 4, after a user specifies singing properties for each of a plurality of divided parts of a selected song (see Fig. 3), voice data and video data are collected as the user sings along with the MIDI music of the song, and the server receives the collected data to generate first content and execute it.

[0071] FIG. 4 (a) to (c) illustrate screens of a user terminal. FIG. 4 (a) illustrates a screen (106) in which a user designates singing properties for each of a plurality of divided parts of a song selected by the user, as in FIG. 3 (b). FIG. 4 (b) illustrates a user terminal collecting voice data and image data as the user sings along with the MIDI music of the selected song. FIG. 4 (c) illustrates a first content generated by a server being executed on a user terminal.

[0072] When a user sings a song selected through a user terminal, the user terminal operates a MIDI module to play MIDI music for the selected song, operates a camera module and a recording module to collect the user's voice data, and at the same time, the camera module captures the user singing to collect video data. As illustrated in (b) of FIG. 4, when the user sings, a video (104) of the user singing is displayed on the screen of the user terminal that the user is viewing, so that the user can check his or her appearance, and a song part display (107) indicating the part of the song in progress is also displayed on the screen, so that the user can check the part that he or she is currently singing.

[0073] Additionally, the song part display (107) can display which singing properties are assigned to each displayed part.

[0074] The voice data according to the user's singing collected in this way and the video data containing the user's singing appearance are transmitted to the server, and the server can create first content combining MIDI music, the user's voice, and the video for the song according to the singing properties specified by the user, list the first content in the song list, and provide it to the connected users.

[0075] The first content generated at this time can be executed on a user terminal, and as shown in (c) of Fig. 4, the video sung by the user can be displayed as a full-screen video (302) and the voice sung by the user can be output in time with MIDI music.

[0076] Meanwhile, returning to FIG. 2, the server can generate the first content using data regarding the first user's singing as described above, list it in a song list, and provide it to connected users. At this time, the second user can select the first content from the song list (S210).

[0077] The user terminal of the second user can display a plurality of parts divided based on the lyrics of the selected first content song (S220), and the second user can specify his / her own singing properties for each of the divided parts (S230).

[0078] At this time, the second user can newly designate singing properties for parts that can be changed among the parts that have singing properties previously designated by the first user, and the second user cannot newly designate singing properties for parts that the previous user, the first user, has set to 'lock', such as parts designated as 'solo lock properties'. This is because the parts previously designated as 'solo lock properties' by the first user are parts that are set to be fixed as parts that the first user sings solo, and therefore cannot be arbitrarily changed by subsequent users. Of course, the second user can also redesignate the solo lock properties for parts that can be designated as singing properties so that he or she can fix his or her solo part.

[0079] When a singing attribute is specified by a second user for each of a plurality of divided parts of a song selected in the above manner, this can be transmitted to the server (S240) and stored in the server as the singing attribute of the second user (S350).

[0080] After specifying the singing properties as described above, the second user can start singing along to the MIDI music of the song by using the functions of the MIDI module, recording module, and camera module of the user terminal (S250), and the user terminal collects the voice data of the second user singing along to the MIDI music and the video data of the second user singing and transmits them to the server (S260), and the server can receive and store the voice data of the second user and the video data of the second user, respectively (S360).

[0081] And, the server can combine the voice data and video data of the second user with the first content according to the second user-specified singing properties (by matching the voice data and video data for each section corresponding to the second user's solo part and choir part), and generate the second content, which is a choir content through collaboration of multiple users, by combining the first user's solo part singing, the second user's solo part singing, the first and second users' choir, and the first and second users' video data for each (S380).

[0082] Figure 2 shows the process of creating the second content as a result of the participation of the first user and the second user, but it is also possible for the third user, the fourth user, and the nth user to participate and create the third content, the fourth content, and the nth content in the same manner.

[0083] In this regard, referring to FIG. 5, (a) of FIG. 5 shows a state in which a plurality of parts are divided based on the lyrics of the song of the first content and a previous user has specified a singing attribute for each part.

[0084] Figure 5 (a) shows an example in which the first four parts were designated as solo properties (pPs) by the previous user, the next five parts were designated as solo lock properties (pPLs) by the previous user, and the next three parts were designated as chorus properties (pPh) by the previous user, which is the same as the singing properties designated by the previous user as shown in Figure 3 (b) and Figure 4 (a).

[0085] As shown in (a) of Fig. 5, the second user can re-assign his own singing properties for the remaining parts, except for the parts designated as locked properties, i.e., "solo lock properties," for which the previous user has designated singing properties. Parts for which the second user has newly designated singing properties are shown in (b) of Fig. 5.

[0086] In (b) of Fig. 5, the second user designated the first two parts (pPs) as his own solo properties and designated the next two parts (cPLs) as his own solo lock properties for the parts (pPs) designated by the previous user as his own solo parts, and the parts (pPLs) to which the previous user had assigned the solo lock properties were maintained as they were because the second user could not change them, and the parts (pPh) to which the previous user had assigned the choir properties were also designated (cPh) as choir properties by the second user.

[0087] In this way, after the singing attribute newly granted or changed by the current user, the second user, is completed from the singing attribute granted by the previous user, as shown in (c) of FIG. 5, when the user sings, a video (104) of the user singing is displayed on the screen of the user terminal that the user is viewing, so that the user can check his / her appearance, and a song part display (107) indicating the part of the song in progress is displayed on the screen together, so that the user can check the part that he / she is currently singing, and the song part display (107) can also display which singing attribute is assigned to each displayed part.

[0088] The voice data and video data of the user's singing collected in this manner are transmitted to the server, and the server can create a new choral piece, the second content, by matching the second user's voice and video to the previously created first content (seed content or choral piece content) according to the singing properties specified by the second user. Up to the nth content can be created in this manner.

[0089] The second content created at this time can be executed on a user terminal, and (a) to (c) of FIG. 6 show the progress according to the type of part of the choral content created by multiple users including the first user and the second user as described above.

[0090] Figure 6 (a) shows a scene in which a section corresponding to parts for which the first user has designated a solo attribute or a solo lock attribute in the second content is executed, in which the first user's solo part voice is output along with MIDI music, and at the same time, a video of the first user's singing is displayed as a full-screen video (302) and played.

[0091] (b) of Fig. 6 shows a scene in which a section corresponding to parts for which a second user has designated a solo attribute or a solo lock attribute in the second content is executed, in which the second user's solo part voice is output along with MIDI music, and at the same time, a video of the second user's singing is displayed as a full-screen video (303) and played.

[0092] (c) of Fig. 6 shows a scene in which a section corresponding to parts designated as chorus properties by the first user and the second user in the second content is executed, and the voices of the first user and the second user are output together with MIDI music, while at the same time, a video of the first user singing and a video of the second user singing are displayed and played as a split screen video (304) on one screen.

[0093] The choral content in which multiple users participate is characterized by the fact that during each user's solo part, a video of the user singing is displayed full screen, and during the part where the multiple users sing together, a video of each user singing is displayed as a split screen video, allowing the individuality of each user who participated in the creation of the choral content to be revealed while also directing the users to sing together, allowing for the creation of content with a high degree of completion.

[0094] As described above, the present invention is characterized in that not only 2 users, but also 3, 4, ... n users can participate and create new choir content each time, and an example of a split screen video that can be displayed in a choir part each time a new user participates is shown in FIG. 7.

[0095] FIG. 7 (a) shows an example of a split screen video containing the singing of each of the two users displayed in the choir part in content in which two users participated, FIG. 7 (b) shows an example of a split screen video containing the singing of each of the three users displayed in the choir part in content in which three users participated, and FIG. 7 (c) shows an example of a split screen video containing the singing of each of the five users displayed in the choir part in content in which five users participated.

[0096] Meanwhile, with reference to FIGS. 8 to 11, a mechanism for creating content according to a multi-participatory MIDI-based content creation method according to one embodiment of the present invention will be described.

[0097] In (a) of Fig. 8, it is shown that user A uses user terminal A to specify singing properties for each section (section 1 to section 9) of the divided parts of the entire Midi song lyrics, collects voice data (A Voice) according to singing in accordance with the entire Midi song accompaniment, and video data (A Video) containing the singing appearance and transmits them to the server, and the server uses this to create seed content, which is the first content.

[0098] When seed content is created in this way, other users can participate in the creation of new content by selecting a song from the song list. Figure 8 (b) shows a case where another user, user B, participated.

[0099] As shown in (b) of Fig. 8, a B user can use a B user terminal to specify singing properties for each section (section 1 to section 9) of the divided parts of the entire Midi song lyrics, and collect voice data (B Voice) according to singing and video data (B Video) containing the singing appearance in accordance with the accompaniment sound of the entire Midi song and transmit them to a server.

[0100] At this time, since the parts of section 1 and section 2 that the previous user, User A, designated as the 'solo lock attribute' among the many parts of the song are fixed to User A's voice, User B cannot change them to other singing attributes, and for the remaining parts, User B can designate the desired singing attribute among the solo attribute, solo lock attribute, and choir attribute. In the example shown in (b) of Fig. 8, this shows the case where User B designated the parts of section 3 and section 4 as the solo lock attribute (fixed to B's solo part), designated the parts of section 5 and section 6 as the solo attribute (this attribute can be arbitrarily changed by other users later), and designated sections 7 to 9 as the choir attribute.

[0101] In this way, data transmitted from user terminal B can be added to seed content to create new content, as shown in (c) of Fig. 8.

[0102] As illustrated in (c) of Fig. 8, the server can create a choral content having an integrated voice and video by aligning the voice data and video data of user A and the voice data and video data of user B into the necessary sections according to the singing properties specified by user A and the singing properties specified by user B.

[0103] The server stores the lyrics data for each song in the lyrics data DB for all songs stored in the MIDI data DB, and divides each song into multiple parts based on the lyrics, and finely distinguishes each part based on the section of each divided part, for example, the sound data of a MIDI note, and sets the section information of each part, that is, information about the start and end positions of each part, in advance.

[0104] In the example shown in Figure 8, sections 1 to 9 are sections that the server has divided into multiple parts based on the lyrics for the song, and each part corresponds to each section.

[0105] As shown in (c) of Fig. 8, since the singing properties of users A and B are specified for each section (section 1 to section 9) of each part of the entire song, the voice data of the corresponding user can be extracted for each section according to the specified singing properties and matched to the corresponding section to create an integrated voice, and accordingly, the video data of the corresponding user can be extracted for each section and matched to the corresponding section to create an integrated video.

[0106] In the example shown in (c) of FIG. 8, since sections 1 and 2 are parts that user A has designated as solo lock properties, user B's voice data and user B's video data for sections 1 and 2 can be removed so that only user A's voice and video are included in the content for sections 1 and 2. In addition, sections 3 and 4 are parts that user B has designated as solo lock properties, and section 5 is a part that user B has designated as solo properties, so that user A's voice data and user A's video data for sections 3 to 5 can be removed, and user B's voice data and video data can be extracted and matched to each corresponding section so that only user B's voice and video are included in the content for sections 3 to 5. In addition, since sections 6 to 9 are parts that user B has designated as choir properties, for those sections, the voice data of user A and the voice data of user B can be combined, and the video data of user A and the video data of user B can be combined to include the voice and video of the two users singing together.

[0107] The case where users A and B, as shown in (c) of Fig. 8, participated and created choral content is also shown in Fig. 9.

[0108] In Fig. 9, each part of the lyrics part corresponds to each section of sections 1 to 9, and the integrated choir content is divided into sections (sections 1 to 9) for each part based on the lyrics for the MIDI data for the song, and for sections 1 to 2, when the A voice data of user A is aligned, the video (A video) of user A is configured to appear as a full-screen video as a content screen, and for sections 3 to 5, when the B voice data of user B is aligned, the video (B video) of user B is configured to appear as a full-screen video as a content screen, and for sections 6 to 9, since the choir properties are designated, the A voice data and the B voice data are combined and aligned to each corresponding section, and the A video and the B video can be configured to appear as a split video on one screen as a content screen for the corresponding section.

[0109] In this way, a choral content in which users A and B participate can be created, but the present invention does not end here, and the choral content can be provided again to a song list to allow another user to participate in the choral content.

[0110] Figure 10 illustrates a case where a new choral content is created by another user, user C, participating in the choral content ((a) of Figure 10) in which users A and B participated.

[0111] As shown in (a) of Figure 10, the choral content in which users A and B participated is provided in a song list, and when user C selects it, C can collect various data according to his / her singing and transmit them to the server.

[0112] That is, as illustrated in (b) of FIG. 10, user C can designate singing properties according to his / her singing for each section (section 1 to section 9) of each part of the song through his / her user terminal C. At this time, among the singing properties of the parts designated by previous users, user A and user B, only the parts that can be changed can be newly designated as singing properties of user C.

[0113] As shown in (b) of Fig. 10, since sections 1 and 2 are designated as solo lock attributes of user A and sections 3 and 4 are designated as solo lock attributes of user B, user C can designate his / her singing attributes for the remaining sections, i.e., sections 5 to 9. Fig. 10 (b) shows an example in which user C designated sections 5 and 6 as solo lock attributes of user C and sections 7 to 9 as chorus attributes.

[0114] After the singing properties of the C user are specified in this way, the C user can collect the C user's voice data and video data for the entire song section using his / her C user terminal and transmit them to the server, or the C user can have the voice data and video data generated only for the necessary section (e.g., section 5 to section 9) according to his / her singing properties (the C user can record and film only that section).

[0115] In this regard, as illustrated in (c) of FIG. 10, the server can extract the voice data of the user for each section and match it to the corresponding section according to the singing properties specified by the C user and the singing properties specified by previous users, thereby generating an integrated voice, and accordingly, can extract the video data of the user for each section and match it to the corresponding section, thereby generating an integrated video.

[0116] In the example shown in (c) of FIG. 10, since sections 1 and 2 are parts that user A has designated as solo lock properties, user C's voice data and user C's video data for sections 1 and 2 can be removed so that only user A's voice and video are included in the content for sections 1 and 2. In addition, since sections 3 and 4 are parts that user B has designated as solo lock properties, user C's voice data and user C's video data for sections 3 and 4 can be removed so that only user B's voice and video are included in the content for sections 3 and 4.

[0117] Sections 5 and 6 are parts that user C has designated as his own solo lock property, so user C's voice data and video data for sections 5 and 6 can be extracted and aligned with each corresponding section so that only user C's voice and video are included in the content for sections 5 and 6.

[0118] In addition, since sections 7 to 9 are parts that user C has designated as choir properties, for those sections, the A voice data, B voice data, and C voice data can be combined, and the A user's video data, B user's video data, and C user's video data can be combined to include the voices and videos of the three users singing together.

[0119] The case where users A, B, and C, as shown in (c) of Fig. 10, participate and create another choral content is also shown in Fig. 11.

[0120] In Fig. 11, each part of the lyrics part corresponds to each section of sections 1 to 9, and the integrated choral content is divided into sections (sections 1 to 9) for each part based on the lyrics for the MIDI data for the song.

[0121] For sections 1 to 2, when user A's A voice data is matched, user A's video (video A) is configured to appear as a full-screen video as a content screen. For sections 3 to 4, when user B's B voice data is matched, user B's video (video B) is configured to appear as a full-screen video as a content screen. For sections 5 to 6, when user C's C voice data is matched, user C's video (video C) is configured to appear as a full-screen video as a content screen. For sections 7 to 9, since the choir properties are specified, the A voice data, the B voice data, and the C voice data are combined and matched to each corresponding section, and the A video, the B video, and the C video can be configured to appear as split videos on one screen as a content screen for the corresponding section.

[0122] In this way, new choral content can be created with the participation of users A, B, and C. This new choral content can then be provided to other users, allowing them to participate and create new choral content.

[0123] As described above, the multi-participatory MIDI-based content creation system and method according to the present invention has the special advantage that when a plurality of users communicate with a server through their respective user terminals and each of them selects a song through a network at different times and in different spaces and specifies singing properties for their own singing and transmits voice and video data according to the singing to the server, the server creates and services content of a high-quality choral song that appropriately includes solo parts and choral parts of each of the plurality of users by matching MIDI music with the voice data and video data of each user according to the singing properties designated and set by each of the plurality of users, and allows a new user to participate on the network to create content of another choral song for the content, thereby enabling a plurality of users to participate and create content of a high-quality choral song.

[0124] The multi-participatory MIDI-based content creation system and method according to the present invention can be used in a social network-based industry field, such as when multiple users participate through a social network to create and share MIDI-based content.

Claims

1. A method for creating multi-participatory MIDI-based content provided by a server, wherein multiple users communicate with the server through their respective user terminals. The above server divides the song into multiple parts based on the lyrics of the song and sets them, and a step in which a user who selects the song specifies singing properties for each of the divided parts of the song; A step in which voice data according to a user who has specified the above singing attribute sings the song through his / her user terminal and video data of the singing appearance acquired by the camera module of the user terminal are transmitted to the server, and the server combines the MIDI data of the song and the transmitted data to generate seed content; A step in which singing properties for each of the divided parts of the song are specified by the participant who selected the seed content; A step in which voice data and video data of the participant singing the song through his / her user terminal are transmitted to the server; and The server generates MIDI-based choral content by matching voice data and video data for a part in which the participant participates according to the singing properties specified by the user and the singing properties specified by the participant in the seed content; A method for creating multi-participatory MIDI-based content including:

2. In paragraph 1, The above singing properties include a solo property where the user sings solo and a chorus property where multiple participating users sing together. The step of specifying singing properties for each of the above divided parts is: A method for generating multi-participatory MIDI-based content, characterized in that it includes a step of assigning one of the solo properties and the chorus properties to each of the divided parts.

3. In the second paragraph, the step of generating the choral content is: A step of aligning each corresponding part of the voice data of a user or participant who has designated the solo attribute for each section among a plurality of parts of the song, and generating a video of the user or participant corresponding to the voice data of each solo attribute as a single-screen video for each section, Including a step of combining voice data of each of the participating users among the plurality of users for a part of the song to which the choir attribute is designated among the plurality of parts of the song, matching the corresponding part to the MIDI data for the song, and generating a video of the participating users as a split-screen video. A method for generating content based on multi-participation MIDI, characterized in that in parts for which the solo attribute is designated for the song, the voice and video of each user who designated the attribute are output separately, and in parts for which the choir attribute is designated, the voice data of the users who designated the attribute are combined to generate content in which the voices of the users singing together and the video of the users are output as a split-screen video.

4. In paragraph 1, The above singing properties include a solo property in which the user sings solo, a solo lock property that locks the solo property so that only the designated user can sing solo in the corresponding part, and a chorus property in which multiple participating users sing together. The step of specifying singing properties for each of the divided parts of the song by the above user is as follows: A step in which one or more of the solo properties, solo lock properties and chorus properties are selectively designated for each of the plurality of divided parts of the song by the user through his / her user terminal and stored in the server as the designated singing property information of the user, The step in which the singing properties for each of the divided parts of the song are specified by the above participants is: A method for creating multi-participatory MIDI-based content, characterized in that it includes a step of selectively designating one or more of a solo property, a solo lock property, and a chorus property for each of the solo property and chorus property parts designated by the user through the user terminal of the participant and storing the same in the server as the designated singing property information of the participant.

5. In the fourth paragraph, the step of generating the choral content is: In the seed content, the voice data of the user corresponding to the section of the part designated as the solo attribute and solo lock attribute by the participant among the solo attribute parts of the user is replaced by extracting the voice data of the participant in the corresponding section, and the video data of the user in the corresponding section is replaced by extracting the video data of the participant and a single-screen video. A method for generating multi-participatory MIDI-based content, characterized by comprising the steps of: extracting voice data of the participant corresponding to a section of the part designated as the choir attribute, combining the voice data of the one user of the section to generate a choir voice, and generating video data of each of the one user and the participant corresponding to the section of the part designated as the choir attribute as a split-screen video, thereby generating the choir content.

6. In paragraph 5, A multi-participatory MIDI-based content creation method characterized in that a new participant can select the above-generated choral content, designate new singing properties, and match the voice data and video data of the new participant accordingly to create new choral content in which the new participant participates.

7. In paragraph 6, A step in which one or more of the solo properties, solo lock properties and chorus properties are designated and set for a plurality of divided parts of the song by the nth user, A step in which voice data according to the singing of the nth user and video data of the singing appearance of the nth user are transmitted to the server according to the MIDI music for the above song, A method for generating content based on multi-participation MIDI, characterized in that it further comprises the steps of extracting voice data of the nth user corresponding to a section of a part designated as a solo attribute and a solo lock attribute by the nth user among solo attribute parts of previous users in the content of the choir and replacing it with a portion of the voice data of the previous user, extracting video data of the nth user for the corresponding section and extracting it as a single-screen video and replacing it with the video data of the previous user, extracting voice data of the nth user corresponding to a section of a part designated as a choir attribute and combining it with voice data of previous users for the corresponding section to generate a choir voice, and generating video data of each of the previous users and the nth user corresponding to a section of a part designated as a choir attribute as a split-screen video, thereby generating content of another choir.

8. A method for creating multi-participatory MIDI-based content provided by the server, wherein multiple users communicate with the server through their respective user terminals. A step of setting and specifying singing properties desired by the first user for each of a plurality of parts divided based on the lyrics of a song selected from the user terminal of the first user; A step in which MIDI music according to the MIDI data of the selected song is played on the user terminal of the first user, and voice data according to the singing of the first user and image data of the singing appearance of the first user acquired by the camera module of the user terminal of the first user are transmitted to the server; A step in which the server uses data transmitted from the user terminal of the first user according to the singing properties specified by the first user to generate first content, which is content that is a combination of MIDI music for the selected song, the voice and video of the first user, or content that is a combination of previously generated content; A step of setting and specifying singing properties in a form desired by the second user for each of the plurality of parts of the song of the first content in the user terminal of the second user; A step in which MIDI music according to MIDI data of a song of the first content is played on the user terminal of the second user, and voice data according to the singing of the second user and image data of the singing of the second user acquired by the camera module of the user terminal of the second user are transmitted to the server; and A step in which the server combines data transmitted from the user terminal of the second user with the first content according to the singing properties specified by the second user to generate second content, which is content of a choral song through collaboration between multiple users including the first user and the second user; A method for creating multi-participatory MIDI-based content including:

9. In paragraph 8, The above singing properties include a solo property in which the user sings solo, a solo lock property that locks the solo property so that only the designated user can sing solo in the corresponding part, and a chorus property in which multiple participating users sing together. The step of generating the above first content is: A step of extracting voice data of the first user corresponding to a section of a part designated as a solo attribute, a solo lock attribute, and a chorus attribute according to a singing attribute designated by the first user, matching it to the MIDI data, and extracting video data of the first user of the corresponding section and generating it as a single-screen video, thereby generating the first content, The step of generating the above second content is: A method for generating multi-participatory MIDI-based content, characterized in that it comprises the steps of extracting and replacing voice data of the first user corresponding to a section of a part designated as a solo property and a solo lock property by the second user among solo property parts of the first user in the first content with voice data of the second user in the corresponding section, extracting video data of the first user in the corresponding section and replacing it with a single-screen video, extracting voice data of the second user corresponding to a section of a part designated as a choir property and combining it with the voice data of the first user in the corresponding section to generate a choir voice, and generating video data of each of the first user and the second user corresponding to a section of a part designated as a choir property as a split-screen video, thereby generating the second content.

10. In paragraph 8, A step of setting a property desired by the third user among a solo property in which the user sings solo for each of the plurality of parts of the song of the second content on the user terminal of the third user, a solo lock property in which the solo property is locked so that only the designated user can sing solo for the part, and a chorus property in which multiple participating users sing together, A step in which voice data according to the third user's singing and video data of the third user's singing appearance are transmitted to the server according to the MIDI music for the song; A method for generating multi-participatory MIDI-based content, characterized in that it further comprises the steps of extracting voice data of the third user corresponding to a section of a part designated as a solo attribute and a solo lock attribute by the third user among solo attribute parts of the previous user in the second content and replacing it with the voice data of the previous user, extracting video data of the third user for the corresponding section and extracting it as a single-screen video and replacing it with the video data of the previous user, extracting voice data of the third user corresponding to a section of a part designated as a choir attribute and combining it with the voice data of the previous users for the corresponding section to generate a choir voice, and generating video data of each of the previous users and the third user corresponding to the section of the part designated as the choir attribute as a split-screen video, thereby generating third content which is another choir song.

11. A multi-participatory MIDI-based content creation system in which multiple users communicate with the server through their respective user terminals and are provided by the server. The user terminal is configured to play MIDI music according to MIDI data, collect voice data according to the user's singing, and display a song selected from among a plurality of songs by dividing the song into a plurality of parts based on lyrics, so that the user can specify a singing attribute for each of the divided parts, and collect voice data according to the user's singing and image data of the user's singing appearance acquired by the camera module while the MIDI music of the song is played, and transmit the collected data to the server. The server generates first content, which is content that is a combination of MIDI music for the song, the voice and video of the user, or content that is combined with previously generated content, by using voice data and video data transmitted from a user terminal of a user according to a singing attribute specified by the user for each of a plurality of divided parts of the song, and generates second content, which is content of a choral song through collaboration of multiple users including the user and the next user, by combining voice data and video data transmitted from a user terminal of a next user according to a singing attribute specified by the next user for each of a plurality of divided parts of the song. A multi-participatory MIDI-based content generation system.

12. In paragraph 11, The above singing properties include a solo property in which the user sings solo, a solo lock property that locks the solo property so that only the designated user can sing solo in the corresponding part, and a chorus property in which multiple participating users sing together. The above server, The first content is generated by extracting the voice data of the user corresponding to the section of the part designated as the solo attribute, solo lock attribute, and chorus attribute according to the singing attribute designated by the user, matching it to the MIDI data, and extracting the video data of the user of the corresponding section and generating it as a single-screen video. A multi-participatory MIDI-based content creation system characterized in that the second content is created by extracting and replacing voice data of the first user corresponding to a section of a part designated as a solo property and a solo lock property by the next user among the solo property parts of the first user, extracting and replacing the voice data of the next user in the corresponding section, extracting the video data of the first user in the corresponding section and replacing it with a single-screen video, extracting the voice data of the next user corresponding to a section of a part designated as a choir property and combining it with the voice data of the first user in the corresponding section to create a choir voice, and creating a split-screen video of video data of each of the first user and the next user corresponding to a section of a part designated as a choir property.

Images