Method and device for providing music sharing service

Abstract

An embodiment of the present invention relates to a method and apparatus for providing a sound source sharing service, and more specifically, to a method and apparatus for providing a sound source sharing service that allows users to share sound source playback using their respective sound source streaming services.

Description

Technology Field

[0001] The present invention relates to a method and apparatus for providing a sound source sharing service, and more specifically, to a method and apparatus for providing a sound source sharing service in which multiple users share sound sources using a sound source streaming service they use. Background Technology

[0002] Due to the recent growth of the entertainment industry and the advancement of IT technology, social network services and fan platforms for communication between celebrities, artists, and fans are on the rise. These social network services can serve as fan community hubs. For example, they can comprehensively provide features such as communication with celebrities and artists, live broadcasting, and fan community functions.

[0003] In addition, these fan platforms are expanding their functions beyond the functions of existing communities to include the sale and purchase of merchandise, ticketing for concerts and events, providing their own content, and the application of non-fungible tokens (NFTs).

[0004] As the functionality of fan platforms expands, the notifications and messages provided to users can also be diverse. For example, if a feature is provided that allows artists to directly write and post articles, messages, or videos to the community, users can receive immediate push notifications when artists write posts or comments, or receive notifications regarding various benefits for membership subscribers. Furthermore, notifications and messages with different content can be provided depending on the various features.

[0005] As such, not only are the functions of fan platforms expanding, but as the entertainment market becomes globalized, the number of artists, communities, and users of fan platforms is also rapidly increasing. Furthermore, as the user base expands globally due to the influence of the Korean Wave, various attempts are being made to increase communication between artists and a large number of fans.

[0006] The music industry has rapidly shifted from a download-based model to a subscription-based streaming model. As various streaming services such as Spotify®, Apple Music®, YouTube Music®, and Melon® have formed independent ecosystems, users are tied to the service they subscribe to to consume music. At the same time, there is a steadily increasing demand for "social listening," where artists and fans listen to the same song together in real time in a remote environment and interact through chat and voice calls.

[0007] To implement social listening, group playback features within a single platform have traditionally been provided. Spotify®’s Group Sessions, Apple SharePlay®, and others are structured so that all participants use the same streaming app and share the same song and playback timeline.

[0008] However, there are limitations in that users of different services cannot listen together, and song synchronization does not occur due to songs not available in certain countries or DRM restrictions.

[0009] The central relay method is a form in which the system audio of the host PC is transmitted as is in video conferencing and VoIP (Discord, Zoom), or a separate server re-encodes the audio source and transmits it to listeners in bulk.

[0010] The host-relay method had problems such as sound quality degradation, latency, the risk of copyright infringement (unauthorized retransmission), and high bandwidth burden, as well as difficulty in expanding into commercial services.

[0011] The link sharing and manual synchronization method involves the host sharing song URLs and playlists and manually coordinating them, such as "3, 2, 1 play!". However, there is a problem where the actual playback timing does not match due to network latency and differences in response speeds among users, which significantly reduces immersion.

[0012] As such, conventional technology entails limitations such as (i) mandatory subscription to the same service, (ii) copyright and server load issues, (iii) latency and sound quality degradation, and (iv) the inconvenience of manual synchronization. Furthermore, since the song identifier (ID) systems and playback APIs of various streaming services differ, there is no clearly established protocol for accurately matching and simultaneously playing the "same song" between different apps. The problem to be solved

[0013] The method and apparatus for providing a sound source sharing service according to an embodiment of the present invention are intended to share sound source playback between users using different streaming services.

[0014] The method and apparatus for providing a sound source sharing service according to an embodiment of the present invention are intended to additionally provide voice when sharing sound sources between users using different streaming services.

[0015] In addition, the method and apparatus for providing a sound source sharing service according to an embodiment of the present invention are for communication between an artist and a number of fans.

[0016] However, the technical problem that this embodiment aims to solve is not limited to the technical problem described above, and other technical problems may exist. means of solving the problem

[0017] As a technical means for achieving the technical problem described above, a method for sharing audio playback between a plurality of terminals according to an embodiment of the present invention comprises the steps of: a host terminal creating a shared audio playlist and transmitting the shared audio playlist data to an audio sharing management server; performing audio playback corresponding to the shared audio playlist using a first audio streaming application installed on a first listener terminal; performing audio playback corresponding to the shared audio playlist using a second audio streaming application installed on a second listener terminal; generating a host voice audio based on voice data stored in the host terminal; and the first listener terminal and the second listener terminal receiving and playing the host voice audio from the audio sharing management server, wherein the first audio streaming application refers to an audio streaming application used by the host terminal, and the second audio streaming application refers to an audio streaming application different from the first audio streaming application.

[0018] In addition, the step of the first listener terminal and the second listener terminal receiving and playing the host voice source from the sound source sharing management server according to an embodiment of the present invention includes the step of temporarily suspending the playback of the sound source using the first sound source streaming application and the second sound source streaming application, the step of the first listener terminal and the second listener terminal receiving and playing the host voice source from the sound source sharing management server, and the step of the first listener terminal and the second listener terminal resuming the playback of the sound source using the first sound source streaming application and the second sound source streaming application when the playback of the host voice source is completed.

[0019] Additionally, the step of generating a host voice source according to an embodiment of the present invention includes: adding the host voice source to a shared sound source playlist; temporarily suspending sound source playback using a first sound source streaming application and a second sound source streaming application when playback of a sound source prior to the host voice source in the shared sound source playlist is completed; receiving and playing the host voice source from a sound source sharing management server by a first listener terminal and a second listener terminal; and when playback of the host voice source is completed, resuming playback of a sound source following the host voice source in the shared sound source playlist using the first sound source streaming application and the second sound source streaming application.

[0020] Additionally, the step of performing audio playback corresponding to a shared audio playlist using a second audio streaming application according to an embodiment of the present invention includes the step of a audio sharing management server searching for the audio with the highest similarity among the audio available in the second audio streaming application based on audio information included in the shared audio playlist, and the step of matching the audio with the highest similarity to the audio included in the shared audio playlist.

[0021] In addition, the step of transmitting shared audio playlist data according to an embodiment of the present invention to an audio sharing management server includes the step of linking a first audio streaming application installed on a host terminal and generating a shared audio playlist based on audio information of the first audio streaming application.

[0022] Additionally, the step of searching for a sound source with high similarity according to an embodiment of the present invention includes: receiving first metadata for a sound source included in a shared sound source playlist from a first sound source streaming application management server; searching for one or more corresponding candidate sound sources in a first sound source streaming application based on the first metadata for a sound source included in the shared sound source playlist; receiving second metadata for a candidate sound source from a second sound source streaming application management server; deriving the similarity between the first metadata and the second metadata; and setting the sound source with the highest similarity among the candidate sound sources as a sound source that matches the sound source included in the shared sound source playlist.

[0023] Additionally, the step of deriving similarity between the first metadata and the second metadata according to an embodiment of the present invention includes the step of deriving similarity based on the similarity between one or more of the playback time, ISRC code, selected audio source name, artist name, and album name information included in the first metadata and the playback time, ISRC code, selected audio source name, artist name, and album name information included in the second metadata.

[0024] In addition, the step of setting the sound source with the highest similarity among the candidate sound sources according to an embodiment of the present invention as a sound source that matches the sound source included in the shared sound source playlist includes the step of setting matching management information for managing the matching between the sound source with the highest similarity among the candidate sound sources and the sound source included in the shared sound source playlist, and the step of providing sound source information corresponding to the first sound source streaming service or the second sound source streaming service to the first listener terminal or the second listener terminal based on the matching management information.

[0025] In addition, a method for sharing audio playback between a plurality of terminals according to an embodiment of the present invention further includes the steps of: a audio sharing management server receiving audio playback operation information of a host terminal and storing audio playback operation information of the host terminal; and synchronizing audio playback between the host terminal and the first listener terminal or the second listener terminal based on the audio playback operation information of the host terminal.

[0026] In addition, the step of synchronizing sound source playback according to an embodiment of the present invention includes the step of a first listener terminal or a second listener terminal performing a synchronization request at a preset period, the step of providing sound source playback operation information to the first listener terminal or the second listener terminal, the step of operating sound source playback of a first sound source streaming application using the first listener terminal based on the sound source playback operation information, and the step of operating sound source playback of a second sound source streaming application using the second listener terminal based on the sound source playback operation information.

[0027] Additionally, a sound source sharing service providing device for sharing sound source playback among a plurality of terminals according to an embodiment of the present invention comprises at least one memory and at least one processor that performs a sound source sharing service by executing at least one application stored in the memory, and the instructions of the processor include the steps of: a host terminal creating a shared sound source playlist and transmitting the shared sound source playlist data to a sound source sharing management server; a step of performing sound source playback corresponding to the shared sound source playlist using a first sound source streaming application installed on a first listener terminal; a step of performing sound source playback corresponding to the shared sound source playlist using a second sound source streaming application installed on a second listener terminal; a step of generating a host voice sound source based on voice data stored in the host terminal; and a step in which the first listener terminal and the second listener terminal receive and play the host voice sound source from the sound source sharing management server, wherein the first sound source streaming application refers to a sound source streaming application used by the host terminal, and the second sound source streaming application refers to a sound source streaming application different from the first sound source streaming application. Effects of the invention

[0028] A method and device for providing a sound source sharing service according to an embodiment of the present invention can share sound source playback between users using different streaming services.

[0029] The method and device for providing a sound source sharing service according to an embodiment of the present invention can additionally provide voice when sharing sound sources between users using different streaming services.

[0030] In addition, the method and device for providing a sound source sharing service according to an embodiment of the present invention can perform communication between an artist and a number of fans. Brief explanation of the drawing

[0031] FIG. 1 is an exemplary diagram showing a communication connection of a sound source sharing device using a heterogeneous streaming service according to an embodiment of the present invention. FIG. 2 is a configuration diagram of a sound source sharing management server according to an embodiment of the present invention. FIG. 3 is a configuration diagram of a terminal according to an embodiment of the present invention. FIG. 4 is a conceptual diagram illustrating the functions of a processor according to an embodiment of the present invention. FIG. 5 is a data flow diagram for sharing audio sources between heterogeneous streaming services according to an embodiment of the present invention. FIG. 6 is a data flow diagram for audio source playback synchronization according to an embodiment of the present invention. FIG. 7 is a data flow diagram for host voice playback according to an embodiment of the present invention. FIG. 8 is a data flow diagram for host voice playback according to an embodiment of the present invention. FIG. 9 is a flowchart of a method for sharing audio playback according to an embodiment of the present invention. FIG. 10 is a flowchart of a method for registering a shared sound source playlist according to an embodiment of the present invention. FIG. 11 is an example diagram of an interface for creating a sound source playback sharing chat room according to an embodiment of the present invention. FIG. 12 is an example diagram of a sound source playback sharing chat room creation reservation interface according to an embodiment of the present invention. FIG. 13 is an example diagram of a shared sound source playlist setting interface according to an embodiment of the present invention. FIG. 14 is an example diagram of a shared sound source search interface according to an embodiment of the present invention. FIG. 15 is an example diagram of a my playlist lookup interface according to an embodiment of the present invention. FIG. 16 is an example diagram of a shared sound source playlist setting completion interface according to an embodiment of the present invention. FIG. 17 is an example diagram of a notification interface including an age-certified sound source according to an embodiment of the present invention. FIGS. 18 to 23 are example diagrams of a sound source sharing chat room thumbnail according to an embodiment of the present invention. FIG. 24 is an example of a waiting screen for a sound source sharing chat room according to an embodiment of the present invention. FIG. 25 is an example diagram of a sound source sharing chat room interface according to an embodiment of the present invention. FIG. 26 is an example diagram of a shared music playlist viewing screen according to an embodiment of the present invention. FIG. 27 is an example diagram of a shared sound source playlist editing screen according to an embodiment of the present invention. FIG. 28 is a flowchart of a method for matching sound sources between streaming services according to an embodiment of the present invention. FIG. 29 is a flowchart of a sound source playback synchronization method according to an embodiment of the present invention. FIG. 30 is a flowchart of a chat processing method according to an embodiment of the present invention. FIGS. 31 and FIGS. 32 are flowcharts of a host voice playback method according to an embodiment of the present invention. Specific details for implementing the invention

[0032] Embodiments of the present invention are described below with reference to the attached drawings so that those skilled in the art can easily implement the invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the present invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification are denoted by similar reference numerals.

[0033] Throughout the specification, when a part is described as being "connected" to another part, this includes not only cases where they are "directly connected," but also cases where they are "electrically connected" with other components interposed between them. Furthermore, when a part is described as "including" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0034] In addition, the attached drawings are intended only to facilitate understanding of the embodiments disclosed in this specification, and the technical concept disclosed in this specification is not limited by the attached drawings; it should be understood that all modifications, equivalents, and substitutions included within the concept and technical scope of the present invention are included.

[0035] Terms including ordinal numbers, such as first, second, etc., may be used to describe various components, but said components are not limited by said terms. These terms are used solely for the purpose of distinguishing one component from another.

[0036] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. On the other hand, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

[0037] A singular expression includes a plural expression unless the context clearly indicates otherwise.

[0038] In this application, terms such as “comprising” or “having” are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0039] Hereinafter, a sound source sharing device using a heterogeneous streaming service according to an embodiment of the present invention will be described with reference to FIG. 1.

[0040] FIG. 1 is an exemplary diagram showing a communication connection between a sound source sharing management server (100) and a terminal (200) according to an embodiment of the present invention.

[0041] Referring to FIG. 1, the sound source sharing management server (100) is connected to a terminal (200) via a communication network. At this time, the sound source sharing management server (100) refers to a device for providing message transmission and reception between users using a fan platform, social network service, etc., and may correspond to a fan platform server, a social network service server, etc.

[0042] The terminal (200) includes a host terminal (200-1) that creates a shared sound source playlist for sharing sound sources and initiates sound source sharing, and one or more listener terminals (200-2) that participate in sharing sound sources.

[0043] At this time, the host terminal (200-1) may include a terminal used by an artist account such as a celebrity, influencer, or artist.

[0044] The sound source sharing management server (100) performs data processing to perform sound source sharing between a host terminal (200-1) and a plurality of listener terminals (200-2).

[0045] When sharing a sound source, the host terminal (200-1) and the listener terminal (200-2) can perform sound source playback sharing using a sound source streaming service (application) available at each terminal.

[0046] Additionally, the audio sharing management server (100) can perform data processing for exchanging chats while sharing audio playback between a host terminal (200-1) and a plurality of listener terminals (200-2) or between a plurality of fan terminals (200-2).

[0047] Additionally, the audio sharing management server (100) receives and stores voice from the host terminal (200-1) during audio playback sharing, and can additionally share the artist's voice during audio playback and provide it to the listener terminal (200-2).

[0048] The terminal (200) may refer to any type of handheld-based wireless communication device, such as a laptop, desktop, laptop, a wireless communication device with guaranteed portability and mobility, or a smartphone, tablet PC, etc., equipped with a web browser.

[0049] In addition, the communication network illustrated in FIG. 1 can be implemented as a wired network such as a Local Area Network (LAN), Wide Area Network (WAN), or Value Added Network (VAN), or as any type of wireless network such as a mobile radio communication network or a satellite communication network.

[0050] Hereinafter, the structure of a sound source sharing management server according to an embodiment of the present invention will be described with reference to FIG. 2.

[0051] FIG. 2 is a structural diagram illustrating the structure of a sound source sharing management server (100) according to an embodiment of the present invention.

[0052] Referring to FIG. 2, the audio source sharing management server (100) includes a communication module (110), memory (120), and a processor (140), and may further include a database (130). The communication module (110) performs information transmission and reception with a terminal (200). The communication module (110) may include a device that includes hardware and software necessary to transmit and receive signals, such as control signals or data signals, using a wired or wireless connection with another network device.

[0053] The memory (120) stores a music playback sharing management program (application). The name of the music playback sharing management program is set for convenience of explanation and does not limit the function of the program by the name itself. The memory (120) can store at least one of the following: information and data input to the communication module (110), information and data required for functions performed by the processor (140), and data generated by the execution of the processor (140).

[0054] The term "memory" (120) should be interpreted as a collective term for a non-volatile storage device that retains stored information even when power is not supplied, and a volatile storage device that requires power to retain stored information. Additionally, the memory (120) can perform the function of temporarily or permanently storing data processed by the processor (140). The memory (120) may include magnetic storage media or flash storage media in addition to a volatile storage device that requires power to retain stored information, but the scope of the present invention is not limited thereto.

[0055] The database (130) can store artist information, user information, subscription information, sound matching information, sound sharing chat room history information, etc. At this time, sound matching information refers to information for matching and managing sound sources between different types of sound streaming services (applications). In addition, sound sharing chat room history information refers to information related to a sound sharing chat room, such as creation information of a sound sharing chat room for performing sound sharing playback and chatting, progress information of a sound sharing chat room, and reservation creation information.

[0056] The database (130) may constitute a part of the memory (120), but is not necessarily located inside the sound source sharing management server (100), and may be connected outside the sound source sharing management server (100) to perform data transmission and reception using a communication connection.

[0057] Additionally, the database (130) can be configured to store information about sound sources and playlist data on an artist-by-artist and user-by-user basis.

[0058] The processor (140) is configured to execute a sound source playback sharing management program in memory (120). The processor (140) may include various types of devices for controlling and processing data. The processor (140) may refer to a data processing device embedded in hardware having a physically structured circuit to perform a function expressed by code or instructions included in the program. In one example, the processor (140) may be implemented in the form of a microprocessor, a central processing unit (CPU), a processor core, a multiprocessor, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), etc., but the scope of the invention is not limited thereto.

[0059] The processor (140) is configured to execute a sound source playback sharing management program and perform the following functions and procedures.

[0060] The processor (140) shares audio playback between the host terminal (200-1) and one or more listener terminals (200-2) and performs data processing for the host terminal (200-1) and the listener terminal (200-2) to exchange chats.

[0061] The host terminal (200-1) and the listener terminal (200-1) can run a sound source playback sharing management program and run a sound source streaming service (application) installed on each terminal to link with the sound source playback sharing management program.

[0062] The host terminal (200-1) can search for a sound source using a sound source playback sharing management program and add it to a shared playback playlist, or create a shared playback playlist using pre-generated sound source playlist information.

[0063] Additionally, the host terminal (200-1) can search for a sound source using the linked first sound source streaming service and add it to a shared playback playlist, or create a shared playback playlist by linking playlist information stored in the first sound source streaming service.

[0064] The processor (140) uses shared playback playlist information generated by the host terminal (200-1) to search for and match a matching sound source in a second sound source streaming service (application) linked with the listener terminal (200-2).

[0065] At this time, the first sound source streaming service refers to a sound source streaming service linked with the host terminal (200-1), and the second sound source streaming service refers to a sound source streaming service different from the first sound source streaming service.

[0066] The processor (140) can control the host terminal (200-1) and the listener terminal (200-2) to share sound source playback using the first sound source streaming service and the second sound source streaming service linked to each terminal by using sound source matching information.

[0067] The processor (140) receives first metadata regarding a sound source included in a shared playback playlist from a first sound source streaming service management server, and searches for a sound source with the highest similarity among the sound sources available in the second sound source streaming service based on the first metadata, and searches for a sound source that matches the sound source included in the shared playback playlist.

[0068] The processor (140) searches for one or more candidate audio sources in the second audio streaming service using information included in the first metadata. Then, the processor (140) receives second metadata regarding the candidate audio sources from the second audio streaming service management server.

[0069] The processor (140) compares the first metadata and the second metadata to derive the similarity between the candidate audio source and the audio source included in the shared playback playlist. Then, the processor (140) sets the audio source with the highest similarity among the candidate audio sources as the audio source that matches the audio source included in the shared playback playlist.

[0070] Additionally, the processor (140) can perform data processing for audio playback synchronization between the host terminal (200-1) and the listener terminal (200-2).

[0071] The processor (140) receives and stores operation information regarding audio playback, such as pause, play, skip section, and change song, when the host terminal (200-1) plays a shared playback playlist using the first audio streaming service.

[0072] And, the processor (140) transmits playback operation information of the host terminal (200-1) to the listener terminal (200-2) at a preset period or when requested by the listener terminal (200-2), and enables the playback of sound sources of the host terminal (200-1) and the listener terminal (200-2) to be synchronized.

[0073] Additionally, the processor (140) receives and displays chat or messages from the host terminal (200-1) and one or more listener terminals (200-2), thereby enabling the host terminal (200-1) and one or more listener terminals (200-2) to exchange chats.

[0074] Additionally, if the host terminal (200-1) is an artist terminal, the processor (140) can display the chat entered using the artist terminal (host terminal) by fixing it to the chat window. Accordingly, listeners (fans) can immediately check the artist's chat.

[0075] Additionally, when the processor (140) displays artist chat on the listener terminal (200-2), it can automatically translate and display the artist chat in the language set by each listener account.

[0076] Additionally, the processor (140) can filter messages entered by the listener terminal (200-2) or set a penalty or ban on chat input for a listener account that enters excessive chat for a preset period of time.

[0077] Additionally, if the host terminal (200-1) is an artist terminal, the processor (140) can share the voice input using the host terminal (200-1) with the listener terminal (200-2). The processor (140) generates a host voice source based on the voice data stored in the host terminal (200-1). Additionally, the processor (140) can receive and store the voice input using the host terminal (200-1) and generate a source for the host voice.

[0078] Then, the processor (140) temporarily suspends the shared playback of the audio source between the host terminal (200-1) and the listener terminal (200-2), and plays the host voice audio source on the host terminal (200-1) and the listener terminal (200-2). Accordingly, the host terminal (200-1) and the listener terminal (200-2) can share the artist's voice message during the shared playback of the audio source.

[0079] At this time, the processor (140) can play the host voice source after the currently playing audio source has ended, or play the host voice source in the middle of audio source playback. Additionally, after the playback of the host voice source has ended, the processor (140) can resume audio source playback sharing from where it left off.

[0080] In addition, information on shared playback playlists and matching information, information on music playback operation, chat information, and host voice information regarding music playback sharing chat rooms can be stored in a database (130) by classifying them by account or chat room. Accordingly, users can re-experience closed music playback sharing chat rooms using the stored information.

[0081] FIG. 3 is a block diagram illustrating the configuration of a terminal (200).

[0082] Referring to FIG. 3, the terminal (200) includes a memory (220), an input / output module (230), and a processor (240), and may further include a communication module (210).

[0083] The communication module (210) can perform information transmission and reception with an external database or an external device. Here, the external device may be the previously described audio sharing management server (100 in FIG. 1). The memory (220) stores an audio playback sharing management program. The name of the audio playback sharing management program is set for convenience of explanation and does not limit the function of the program by the name itself.

[0084] The input / output module (230) can receive information, data, etc. transmitted from the outside to the terminal (200), or output information, data, etc. held by the terminal (200) to the outside. For example, the input / output module (230) may include a display, a speaker, a keyboard, a touchpad, etc. The processor (240) executes a sound source playback sharing management program stored in the memory (220). Additional descriptions of the communication module (210), memory (220), and processor (240) will be replaced by the descriptions of the communication module (110 in FIG. 2), memory (120 in FIG. 2), and processor (140 in FIG. 2) described above with reference to FIG. 2.

[0085] The processor (240) is configured to execute a sound source playback sharing management program to perform the following functions and procedures.

[0086] The processor (240) can execute a sound source playback sharing management program to link and execute a sound source streaming service used by the listener user or installed on the listener terminal (200-2).

[0087] The processor (240) can perform audio playback and synchronization by executing an audio streaming service linked with the terminal (200).

[0088] Additionally, the processor (240) can transmit sound source, playlist information, chat information, voice information, etc. received from the terminal (200) to the sound source sharing management server (100), and can perform data transmission and reception with the sound source sharing management server (100) to perform sound source playback sharing.

[0089] Hereinafter, the operation of the processor (140) according to an embodiment of the present invention will be described in detail with reference to FIG. 4.

[0090] FIG. 4 is a conceptual diagram illustrating the functions of a sound source sharing management server according to an embodiment of the present invention.

[0091] Referring to FIG. 4, the processor (140) can perform the functions of a sound source sharing chat room management module (141), a sound source matching data management module (142), a sound source playback synchronization module (143), a chat management module (144), and a host voice sound source management module (145).

[0092] The audio sharing chat room management module (141) can perform the function of creating an audio sharing chat room for sharing audio playback and exchanging chats between a host terminal (200-1) and a listener terminal (200-2).

[0093] The music sharing chat room management module (141) provides an interface for creating a music sharing chat room to the host terminal (200-1). The host terminal (200-1) can input one or more of the following using the music sharing chat room creation interface: a chat room title, a thumbnail image, scheduled start information, and shared music playlist information.

[0094] The music sharing chat room management module (141) can select one of the multiple music streaming services and link it with the host account in order to input shared music playlist information.

[0095] The music sharing chat room management module (141) can create a shared music playlist based on music searched by the host account or pre-saved playlist information.

[0096] Additionally, the music sharing chat room management module (141) can search for music using the first music streaming service linked with the host terminal (200-1) or create a shared music playlist by linking playlist information.

[0097] When information is entered to create a music sharing chat room, a thumbnail for the music sharing chat room is generated and displayed to other users. Accordingly, other users can select the thumbnail to access the music sharing chat room as listeners.

[0098] At this time, when the host enters the reservation start information, a thumbnail is generated and displayed in advance, and the music sharing chat room is started at the scheduled date and time to perform music sharing playback.

[0099] The music sharing chat room management module (141) can send notifications and messages containing information about the creation of the music sharing chat room to user accounts following the artist when the artist creates a music sharing chat room, if the host is an artist account.

[0100] The sound source matching data management module (142) stores and manages sound source matching information between different sound source streaming services so that the host terminal (200-1) and the listener terminal (200-2) can play sound sources included in a shared sound source playlist using the sound source streaming service they each use.

[0101] The audio matching data management module (142) derives a similarity score for the audio included in the shared audio playlist created by the host terminal and selects the audio that matches each audio streaming service.

[0102] The sound source matching data management module (142) receives first metadata about a sound source included in a shared sound source playlist from a first sound source streaming service management server linked with a host terminal (200-1).

[0103] And, the sound source matching data management module (142) receives candidate sound source information corresponding to the track ID information of the sound source from the second sound source streaming service management server linked with the listener terminal (200-2) using the track ID information of the sound source included in the first metadata.

[0104] When a listener terminal (200-2) performs audio playback using a first audio streaming service linked with a host terminal (200-1), the audio matching data management module (142) transmits a track ID to the first audio streaming service linked with the listener terminal (200-2), and the first audio streaming service linked with the listener terminal (200-2) can search for and play audio included in a shared audio playlist using the track ID.

[0105] At this time, the sound source matching data management module (142) can search for candidate sound sources using not only the track ID of the sound source, but also the sound source name, artist name, album name, disc number, track number, etc.

[0106] The sound source matching data management module (142) receives second metadata for candidate sound sources from the second sound source streaming service management server. Then, the sound source matching data management module (142) derives the similarity between the first metadata and the second metadata and sets the sound source with the highest similarity among the candidate sound sources as the matching sound source.

[0107] The audio matching data management module (142) derives similarity based on the ISRC code, track name, album name, artist name, playback time, and obscenity status (whether playable upon adult verification) included in the audio metadata.

[0108] Specifically, the sound source matching data management module (142) excludes a candidate sound source from being matched if the difference in playback time between the sound source and the candidate sound source is 3 seconds or more, or if the degree of obscenity is different.

[0109] Then, the sound source matching data management module (142) sets the initial similarity score of the candidate sound source to 0 points, and then calculates the similarity-based score of the first metadata and the second metadata to derive the similarity score.

[0110] The sound source matching data management module (142) awards 10 bonus points to the candidate sound source if the ISRC codes of the first metadata and the second metadata are the same.

[0111] Additionally, the sound source matching data management module (142) assigns a score of 0 to 80 points based on the string similarity of each track name, album name, and artist name included in the first metadata and the second metadata.

[0112] Additionally, the sound source matching data management module (142) grants an additional 10 points if the album name included in the first metadata and the second metadata is 100% identical and the disc number and track number are also identical.

[0113] The sound source matching data management module (142) derives a final similarity score for each candidate sound source and sets the sound source with the highest similarity score among the candidate sound sources as the sound source that matches the sound source included in the shared sound source playlist.

[0114] Accordingly, the sound source matching data management module (142) sets a sound source of a second sound source streaming service that matches a sound source of a first sound source streaming service, and stores the sound source matching information in the database (130). The sound source matching information includes track ID information of the sound source for each sound source streaming service.

[0115] The sound source matching data management module (142) transmits sound source matching information to the listener terminal (200-2), and the listener terminal (200-2) performs playback of a shared sound source playlist in the second sound source streaming service based on the matching information.

[0116] The audio playback synchronization module (143) performs the function of synchronizing audio playback between the host terminal (200-1) and the listener terminal (200-2).

[0117] The audio playback synchronization module (143) stores audio playback operation execution information when performing an operation regarding audio playback using a first audio streaming service linked with a host terminal (200-1).

[0118] In this case, the audio playback operation information refers to all operation information related to audio playback, and includes operation information such as skipping sections, changing the song, pausing playback, and playing.

[0119] The audio playback synchronization module (143) stores information regarding the execution of audio playback operations by the host terminal (200-1) in the database (130). At this time, the audio playback synchronization module (143) stores information regarding the time when the audio playback operation action occurred, the operation of the host, the connection status of the host's first audio streaming service, the track ID of the audio currently being played, the track ID of the audio to be played next, the playback position (in seconds) of the audio currently being played, and the playback status (play, pause) of the audio currently being played.

[0120] The listener terminal (200-2) requests synchronization information from the audio sharing playback service management server (100) at preset intervals.

[0121] The audio playback synchronization module (143) transmits audio playback operation execution information stored in the database (130) to the listener terminal (200-2) in response to a synchronization request from the listener terminal (200-2). Then, the listener terminal (200-2) performs an audio playback operation in a first audio streaming service or a second audio streaming service linked with the listener terminal (200-2) based on the audio playback operation execution information received from the audio playback synchronization module (143).

[0122] Accordingly, the host terminal (200-1) and the listener terminal (200-2) can perform audio playback synchronization.

[0123] The chat management module (144) performs the function of exchanging chats between the host terminal (200-1) and the listener terminal (200-2).

[0124] The chat management module (144) displays chats or messages entered by the host terminal (200-1) and the listener terminal (200-2) in real time in the chat display area after the sound source sharing chat room is created.

[0125] At this time, the chat management module (144) can display the chat entered using the host terminal (200-1) so that it is visually distinguished from the chat entered by the listener terminal (200-2).

[0126] For example, the chat entered by the host terminal (200-1) can be fixed and displayed in the chat display area so that the listener can easily check the host's chat. In addition, the host's chat and the listener's chat can be distinguished by displaying different visual effects, such as displaying the host's chat with a different color or a different font size.

[0127] Additionally, the chat management module (144) can display the chat by automatically translating it into the language set by the listener account or the host account.

[0128] Additionally, the chat management module (144) can filter chats if they contain profanity, forbidden words, etc., or impose disciplinary measures such as banning chat input and banning the creation of a music sharing chat room on accounts that excessively input chats in the chat room.

[0129] The host voice source management module (145) performs the function of sharing the host's voice stored in the host terminal (200-1) with the listener terminal (200-2).

[0130] The host voice source management module (145) generates a host voice source based on voice data stored in the host terminal (200-1). Additionally, the host voice input using the host terminal (200-1) can be stored in the database (130) and a host voice source can be generated. Then, when a host voice source is generated, the sharing of sound source playback using the listener terminal (200-2) is temporarily suspended.

[0131] The host voice source management module (145) temporarily suspends the sharing of sound source playback using the listener terminal (200-2) and then performs streaming (playback) of the host voice source using the listener terminal (200-2).

[0132] When the host voice source management module (145) is finished playing the host voice source, it resumes sharing the sound source playback using the listener terminal (200-2).

[0133] Additionally, the host voice source management module (145) can set the playback time of the host voice source by adding the host voice source between the sources of the shared sound source playlist.

[0134] The host can set the location where the host voice source is to be played using the host terminal (200-1). At this time, the host can add the host voice source to the shared sound source playlist along with other sound sources.

[0135] Accordingly, when the previous audio source of the host voice source management module (145) is completed, the host voice source management module (145) temporarily suspends audio source playback sharing using the listener terminal (200-2) and then performs streaming (playback) of the host voice source using the listener terminal (200-2).

[0136] And, when the host voice source management module (145) is finished playing the host voice source, it can resume sharing the sound source playback using the listener terminal (200-2) and perform the playback of the sound source that is next in order to the host voice source.

[0137] Additionally, when a music sharing chat room is terminated, information related to the music sharing chat room is stored in a database (130). The music sharing playback service management server (100) provides a replay function, and when a user executes the replay function, the user can experience the music sharing chat room again by receiving one or more of the shared music playlist information, the host's music playback operation information, chat between the host account and the listener account, and the host's voice music information for the terminated music sharing chat room stored in the database (130) from the music sharing playback service management server (100).

[0138] Hereinafter, the specific data processing process between the music sharing playback service management server, the terminal, and the streaming service management server will be explained in detail with reference to Figures 5 to 8 described later.

[0139] FIG. 5 is a data flow diagram for sharing audio sources between heterogeneous streaming services according to an embodiment of the present invention.

[0140] Referring to FIG. 5, the host terminal (200-1) registers a shared music playlist and transmits the registered shared music playlist information to the music sharing playback service management server (100).

[0141] As described above, the host terminal (200-1) can create a shared music playlist based on music searched using the music sharing playback service management server (100) or pre-stored playlist information.

[0142] Additionally, the host terminal (200-1) can search for a sound source using the first sound source streaming service linked with the host terminal (200-1) or create a shared sound source playlist by linking playlist information.

[0143] The music sharing playback service management server (100) retrieves music included in the shared music playlist from the first music streaming service management server (400) linked with the host terminal (200-1).

[0144] And, the first audio streaming service management server (400) returns the first metadata of the retrieved audio to the audio sharing playback service management server (100).

[0145] The music sharing and playback service management server (100) searches for candidate music in the second music streaming service based on the first metadata. To do this, the music sharing and playback service management server (100) transmits the track ID information included in the first metadata to the second music streaming service management server (500) to search for candidate music.

[0146] At this time, the audio sharing playback service management server (100) can search for candidate audio sources using not only the track ID of the audio source, but also the audio name, artist name, album name, disc number, track number, etc.

[0147] The second audio streaming service management server (500) returns second metadata for candidate audio sources to the audio sharing playback service management server (100).

[0148] The music sharing playback service management server (100) derives a similarity score between a music track included in a shared music playlist and a candidate music track based on the first metadata and the second metadata. Then, the music sharing playback service management server (100) derives a music track that matches a music track included in a shared music playlist based on the similarity score of the candidate music track.

[0149] The audio sharing and playback service management server (100) excludes a candidate audio source from being matched if the difference in playback time between the audio source and the candidate audio source is 3 seconds or more, or if the degree of obscenity is different. Additionally, the audio sharing and playback service management server (100) grants 10 bonus points to the candidate audio source if the ISRC codes of the first metadata and the second metadata are identical.

[0150] Additionally, the music sharing playback service management server (100) assigns a score of 0 to 80 points based on the string similarity of each track name, album name, and artist name included in the first metadata and the second metadata.

[0151] In addition, the music sharing playback service management server (100) grants an additional 10 points if the album name included in the first metadata and the second metadata is 100% identical and the disc number and track number are also identical.

[0152] The audio sharing playback service management server (100) calculates a similarity score for each candidate audio source in this manner, and based on the similarity score, sets the candidate audio source with the highest similarity score as the audio source that matches the audio source included in the shared audio playlist.

[0153] The audio sharing and playback service management server (100) stores matching information of audio sources in the database (130). The matching information of audio sources includes track ID information of audio sources for each audio streaming service.

[0154] The sound source sharing playback service management server (100) transmits sound source matching information to the listener terminal (200-2).

[0155] Then, the listener terminal (200-2) transmits sound source matching information to the first sound source streaming service management server (400) or the second sound source streaming service management server (500) to register a playlist corresponding to the shared sound source playlist or to play the sound source.

[0156] When the listener terminal (200-2) performs audio playback using the first audio streaming service, the listener terminal (200-2) transmits the track ID included in the audio matching information to the first audio streaming service management server (400) linked with the listener terminal (200-2).

[0157] Accordingly, the first audio streaming service management server (400) can search for an audio source corresponding to a track ID in the received first audio streaming service and play the audio source using the first audio streaming service linked with the listener terminal (200-2).

[0158] When the listener terminal (200-2) performs audio playback using the second audio streaming service, the listener terminal (200-2) transmits the track ID in the second audio streaming service included in the audio matching information to the second audio streaming service management server (500). The second audio streaming service management server (500) searches for an audio source that matches in the second audio streaming service based on the track ID in the second audio streaming service, and can play the audio source using the second audio streaming service linked with the listener terminal (200-2).

[0159] FIG. 6 is a data flow diagram for audio source playback synchronization according to an embodiment of the present invention.

[0160] Referring to FIG. 6, when an operation regarding sound source playback is performed using a host terminal (200-1), the host terminal (200-1) transmits sound source playback operation information to a sound source sharing playback service management server (100), and the sound source sharing playback service management server (100) stores the sound source playback operation information performed using the host terminal (200-1).

[0161] The audio playback operation information may include cases where an operation regarding audio playback is performed using a first audio streaming service linked with a host terminal (200-1).

[0162] At this time, the audio sharing playback service management server (100) stores information regarding the time when the audio playback operation action occurred, the operation of the host, the host's connection status to the first audio streaming service, the track ID of the audio currently being played, the track ID of the audio to be played next, the playback position (in seconds) of the audio currently being played, and the playback status (play, pause) of the audio currently being played in the database (130).

[0163] The listener terminal (200-2) requests synchronization information from the audio sharing playback service management server (100) at preset intervals.

[0164] The audio sharing playback service management server (100) transmits audio playback operation execution information stored in the database (130) to the listener terminal (200-2) in response to a synchronization request from the listener terminal (200-2).

[0165] Then, the listener terminal (200-2) performs sound playback synchronization by transmitting the sound playback operation execution information received from the sound playback synchronization module (143) to the first sound streaming service management server (400) or the second sound streaming service management server (500) linked to the listener terminal (200-2) to perform the sound playback operation.

[0166] FIG. 7 is a data flow diagram for host voice playback according to an embodiment of the present invention.

[0167] The host selects one or more voice data to be shared from among the voice data (host voice data) stored in the host terminal (200-1) and transmits them to the audio source sharing playback service management server (100).

[0168] Additionally, when the host terminal (200-1) inputs host voice data to be shared with a listener, the host voice data is transmitted to the audio source sharing playback service management server (100).

[0169] The audio sharing playback service management server (100) stores the received host voice in the database (130) and generates a host voice source.

[0170] And, when a host voice source is generated, the sound source sharing playback service management server (100) requests the listener terminal (200-2) to temporarily suspend sound source playback sharing.

[0171] Accordingly, the listener terminal (200-2) requests a temporary suspension of sound playback to the first sound source streaming service management server (400) or the second sound source streaming service management server (500) linked with the listener terminal (200-2).

[0172] The audio sharing playback service management server (100) temporarily suspends audio playback sharing using the listener terminal (200-2) and then performs streaming (playback) of the host voice audio source using the listener terminal (200-2).

[0173] When the playback of the host voice source is completed, the listener terminal (200-2) transmits a host voice source playback completion signal to the voice source sharing playback service management server (100).

[0174] When the audio sharing playback service management server (100) receives a signal that the host voice audio playback is complete from the listener terminal (200-2), it sends a request to resume audio playback sharing to the listener terminal (200-2).

[0175] When the listener terminal (200-2) receives a request to resume audio playback sharing, it transmits the request to resume audio playback to the first audio streaming service management server (400) or the second audio streaming service management server (500) linked with the listener terminal (200-2), and accordingly, audio playback can be resumed from a point in time prior to playing the host voice audio.

[0176] Figure 8 is a data flow diagram for playing a host voice by adding a host voice between the audio tracks of the shared audio playlist.

[0177] The host can set the location where the host voice source is to be played using the host terminal (200-1). At this time, the host can add the host voice source to the shared sound source playlist along with other sound sources.

[0178] The host selects one or more voice data to be shared from among the voice data (host voice data) stored in the host terminal (200-1) and transmits them to the audio source sharing playback service management server (100).

[0179] Additionally, when the host terminal (200-1) inputs host voice data to be shared with a listener, the host voice data is transmitted to the audio source sharing playback service management server (100).

[0180] The audio sharing playback service management server (100) generates a host voice audio source.

[0181] Then, the host uses the host terminal (200-1) to add the host voice source to the desired position within the shared audio playlist.

[0182] The audio sharing playback service management server (100) receives shared audio playlist information with added host voice audio from the host terminal (200-1).

[0183] And, when the playback of the previous audio source of the host voice audio source in the shared audio playlist is completed, the audio sharing service management server (100) requests the listener terminal (200-2) to temporarily suspend audio playback sharing.

[0184] Accordingly, the listener terminal (200-2) requests a temporary suspension of sound playback to the first sound source streaming service management server (400) or the second sound source streaming service management server (500) linked with the listener terminal (200-2).

[0185] The audio sharing playback service management server (100) temporarily suspends audio playback sharing using the listener terminal (200-2) and then performs streaming (playback) of the host voice audio source using the listener terminal (200-2).

[0186] When the playback of the host voice source is completed, the listener terminal (200-2) transmits a host voice source playback completion signal to the voice source sharing playback service management server (100).

[0187] When the audio sharing playback service management server (100) receives a signal that the host voice audio playback is complete from the listener terminal (200-2), it sends a request to resume audio playback to the listener terminal (200-2).

[0188] When the listener terminal (200-2) receives a request to resume audio playback sharing, it can transmit the request to resume audio playback to the first audio streaming service management server (400) or the second audio streaming service management server (500) linked with the listener terminal (200-2) and perform playback of the audio that is next in sequence to the host voice audio.

[0189] Hereinafter, a method for sharing audio playback will be described in detail with reference to FIGS. 9 to 32.

[0190] FIG. 9 is a flowchart of a method for sharing audio playback according to an embodiment of the present invention.

[0191] A method for sharing audio playback according to an embodiment of the present invention (hereinafter referred to as the 'audio playback sharing method') includes a step of opening an audio sharing chat room (S100), a step of participating in a listener account (S200), a step of deriving a matching audio source for each streaming service (S300), and a step of sharing audio playback (S400).

[0192] In the step of creating a music sharing chat room (S100), the music sharing playback service management server (100) provides an interface to the host terminal (200-1) for creating a music sharing chat room.

[0193] And, the music sharing playback service management server (100) receives information for creating a music sharing chat room from the host terminal (200-1) and creates a music sharing chat room based on the received information.

[0194] In the step of creating a music sharing chat room (S100), the host terminal can register a shared music playlist to share with the listener terminal.

[0195] In the step of creating a music sharing chat room (S100), when a music sharing chat room is created, the music sharing playback service management server (100) creates a thumbnail for the music sharing chat room and provides it to another user terminal.

[0196] In the listener account participation step (S200), other users participate in the music sharing chat room as listener accounts using a thumbnail of the music sharing chat room.

[0197] In the step of deriving matching audio for each streaming service (S300), the audio sharing and playback service management server (100) searches for audio that matches for different streaming services for the audio included in the shared audio playlist.

[0198] In the audio playback sharing step (S400), the audio sharing playback service management server (100) transmits matching audio information to the listener terminal (200-2). Based on the matching audio information received from the audio sharing playback service management server (100), the listener terminal (200-2) can play an audio corresponding to the shared audio playlist using a streaming service linked to each listener terminal (200-2).

[0199] Hereinafter, a method for creating a sound source sharing chat room and an interface for a sound source sharing chat room will be described in detail with reference to FIGS. 10 to 27.

[0200] FIG. 10 is a flowchart of a method for creating a sound source sharing chat room according to an embodiment of the present invention.

[0201] Referring to FIG. 10, the method for creating a music sharing chat room includes a step of providing a music sharing chat room creation interface (S110), a step of linking with a first streaming service (S120), a step of searching for music and playlists (S130), and a step of registering a music sharing playlist (S140).

[0202] In the step of providing a music sharing chat room creation interface (S110), the music sharing playback service management server (100) provides a music sharing chat room creation interface (1000) that allows a host terminal (200-1) that wants to create a music sharing chat room to input information for creating a music sharing chat room.

[0203] In the first streaming service linkage step (S120), the host terminal (200-1) can select and link to a first music streaming service to play music when using a music sharing chat room.

[0204] In the first streaming service linkage step (S120), when the host terminal (200-1) uses the first music streaming service to be linked, the first music streaming service is executed, and the linkage of the first music streaming service is completed only when the host terminal (200-1) performs authentication of the usage credentials for the first music streaming service through login, etc.

[0205] In the sound source and playlist search step (S130), the host terminal (200-1) views the sound source or playlist it has saved, or enters a search term to search for a sound source.

[0206] The audio sharing playback service management server (100) searches for audio sources corresponding to a search term received from the host terminal (200-1) in the database (130) or the first audio streaming service management server and provides the search results to the host terminal (200-1).

[0207] In the shared music playlist registration step (S140), the host terminal (200-1) selects a music and a playlist to be added to the shared music playlist and registers the shared music playlist to be applied to the music sharing chat room. Accordingly, the music sharing playback service management server (100) performs music playback sharing between multiple listener terminals (200-2) and the host terminal (200-1) based on the registered shared music playlist.

[0208] FIG. 11 is an example of a sound source sharing chat room creation interface (1000).

[0209] As illustrated in FIG. 11, the audio sharing chat room creation interface (1000) includes an image input area (1100) for creating a thumbnail, an audio sharing chat room title input area (1200), an audio sharing chat room content input area (1300), a shared audio playlist setting area (1400), and a reservation setting area (1500).

[0210] When the shared music playlist setting area (1400) is selected, the music streaming service selection area (1401) is activated as shown on the right side of Fig. 11.

[0211] The host terminal (200-2) selects a first music streaming service that it can use among various types of music streaming services. Then, when the first music streaming service is selected, the first music streaming service is executed, and the integration of the first music streaming service is completed only when the host terminal (200-1) performs authentication of the usage credentials for the first music streaming service through login, etc.

[0212] In the reservation setting area (1500), information for creating a reservation for a music sharing chat room can be entered.

[0213] When the host terminal (200-1) enables reservation, the date and time to start the music sharing chat room can be set. Additionally, depending on whether the exposure state is enabled, it can choose whether to display a thumbnail of the music sharing chat room to other users before the reservation starts.

[0214] FIG. 12 is an example of an interface (1510) for setting the date and time in a reservation setting area (1500).

[0215] As illustrated in FIG. 12, when the reservation function is activated in the reservation setting area (1500) and a date is selected, a calendar-style interface is displayed so that the reservation date can be selected and entered. Additionally, when the time is selected, the time can be selected and entered in a dial format.

[0216] An interface for setting up a shared audio playlist will be described in detail with reference to FIGS. 13 to 20.

[0217] FIG. 13 is an example of a shared music playlist creation interface (1410) that is displayed when a music streaming service is selected in the music streaming service selection area (1401) and the linkage is completed.

[0218] As illustrated in FIG. 13, the shared sound source playlist setting interface (1410) includes a sound source addition selection area (1411), a my library selection area (1412), a track count display area (1413), and a track display area (1414).

[0219] When the host terminal (200-1) selects the sound source addition selection area (1411), the sound source search interface (1600) is activated, and the host terminal (200-1) can use the sound source search interface (1600) to search for the sound source it wants and add it to the shared sound source playlist.

[0220] When selecting the My Library selection area (1412), the My Playlist selection interface (1700) is activated, and one or more playlists are selected from the playlists I have saved in advance or the playlists I have liked, and the tracks (audio) included in the selected playlists can be added to the shared audio playlist.

[0221] A playlist I liked refers to a playlist created by another user that I have liked.

[0222] In the track count display area (1413), the number of tracks added to the shared music playlist changes and is displayed in real time.

[0223] The track display area (1414) displays the audio added to the shared audio playlist. If there is no audio added to the shared audio playlist, a message saying "Try creating a playlist by adding audio or importing my library" is displayed in the track display area (1414).

[0224] When one or more sound sources are added, the track display area (1414) displays an image, sound source title, singer, and playback time information for the added sound sources.

[0225] When multiple audio tracks are added, the order of the audio tracks can be changed in the track display area (1414). Additionally, when an audio track is slid to the left, a trash can icon is activated, and when the trash can icon is selected, the corresponding audio track can be deleted from the shared audio playlist.

[0226] Additionally, when one or more sound sources are added to the shared sound source playlist, the completion icon (1415) is activated. When the completion icon (1415) is selected, a shared sound source playlist is created to run a sound source sharing chat room using the sound sources added so far.

[0227] FIG. 14 is an example of a sound source search interface (1600) that is activated when the sound source additional selection area (1411) is selected.

[0228] As illustrated in FIG. 14, the sound source search interface (1600) includes a search term input area (1610) and a search result display area (1620).

[0229] A string can be entered into the search input area (1610). When a string is entered into the search input area (1610), the audio sharing and playback service management server (100) searches for an audio source corresponding to the entered string and displays the audio source corresponding to the search result in the search result display area (1620).

[0230] In the search result display area (1620), the image, title, and singer information of the sound source corresponding to the search result are displayed, and you can select the sound source you want to add to the shared sound source playlist by selecting the '+' icon of the sound source.

[0231] Additionally, if the '+' icon is selected, it changes to a check icon. This allows for distinguishing between audio sources that can be added and those that have already been selected.

[0232] When one or more of the sound sources displayed in the search result display area (1620) are selected, the playlist addition selection area (1630) is activated.

[0233] In the playlist addition selection area (1630), a 'Cancel' icon and a 'playlist addition' icon are displayed. If the 'Cancel' icon is selected, the selection of the selected sound source is reset.

[0234] When the 'Add to Playlist' icon is selected, the selected audio track is added to the shared audio playlist. Accordingly, the added audio track is displayed in the track display area (1414).

[0235] FIG. 15 is an example of a my playlist selection interface (1700) that is activated when the my library selection area (1412) is selected.

[0236] Referring to FIG. 15, my playlist selection interface (1700) includes a my playlist display area (1710) and a playlist addition area (1720).

[0237] My Playlist display area (1710) displays information about playlists saved by the host account or playlists liked by the host account.

[0238] In the My Playlist display area (1710), an image of the playlist, the playlist title, the number of tracks included in the playlist, and information on the playlist playback time may be displayed.

[0239] The host account can select one or more playlists from the playlists displayed in the My Playlist display area (1710) and add the selected playlists to the shared music playlist.

[0240] You can select a playlist to add to the shared music playlist by selecting the '+' icon for the playlist displayed in the My Playlist display area (1710).

[0241] Additionally, if the '+' icon is selected, it changes to a check icon. This allows you to distinguish between playlists that can be added and those that have already been selected.

[0242] When one or more playlists are selected from the playlists displayed in the My Playlist display area (1710), the playlist addition selection area (1720) is activated.

[0243] In the playlist addition selection area (1720), a 'Cancel' icon and an 'Add Playlist' icon are displayed. If the 'Cancel' icon is selected, the selection of the selected playlist is reset.

[0244] When the 'Add to Playlist' icon is selected, the audio source corresponding to the selected playlist is added to the shared audio playlist. Accordingly, the added audio source is displayed in the track display area (1414).

[0245] Additionally, if the number of songs added to the shared song playlist exceeds the preset maximum number of songs, a notification message (1730) indicating that the maximum number of songs has been exceeded can be displayed as shown on the right side of Fig. 15.

[0246] FIG. 16 is an example of a shared music playlist creation interface (1410) in which a music track is added to a shared music playlist.

[0247] As shown in Fig. 16, when the complete icon is selected after setting the sound source to be added to the shared sound source playlist and the playback order of the sound source, the shared sound source playlist setting is completed and the system returns to the sound source sharing chat room creation interface (1000).

[0248] Additionally, if the shared music playlist includes music with age restrictions, a notification message may be displayed. As illustrated in FIG. 17, when the complete icon is selected while the shared music playlist includes music with age restrictions, a notification message (1416) indicating the inclusion of age-restricted music may be displayed as a pop-up window, as shown in FIG. 17.

[0249] Additionally, if the shared music playlist contains age-restricted music, the music sharing playback service management server (100) may restrict music playback synchronization based on whether the listener account is verified to be an adult.

[0250] The audio sharing playback service management server (100) may stop audio synchronization playback in the audio playback order with an age restriction set for a listener terminal (200-2) that has not completed adult verification.

[0251] Then, the audio sharing playback service management server (100) resumes audio synchronization playback for a listener terminal (200-2) that has not completed adult verification from the time when playback of an audio with an age restriction is completed.

[0252] Accordingly, the audio sharing playback service management server (100) can play audio with age restrictions set only for listener terminals that have completed adult verification, and can perform audio playback synchronization.

[0253] FIGS. 18 to 23 are example diagrams of a thumbnail display interface for a music sharing chat room.

[0254] Figure 18 is an example of a thumbnail displayed when an artist account creates a music sharing chat room.

[0255] When an artist account creates a music sharing chat room and the chat room is started, as illustrated in FIG. 18, a thumbnail (2100) of the music sharing chat room created by the artist may be displayed in the home interface (2000) of the fan platform service or in the area displaying the thumbnail of the broadcast currently being live-streamed by the artist.

[0256] The thumbnail for a music sharing chat room may display a thumbnail image, the title of the music sharing chat room, and the host nickname information.

[0257] Additionally, as illustrated in FIG. 19, if there is an artist-specific page (3000) in the fan platform service, a thumbnail (2100) of a music sharing chat room currently in progress for the artist account can be displayed on the artist page.

[0258] Additionally, as shown in FIG. 20, a tab (3100) for displaying music sharing chat rooms can be created and displayed on the artist page (3000). When the tab (3100) for displaying music sharing chat rooms is selected, thumbnails of music sharing chat rooms created by the artist, music sharing chat rooms created by the artist, music sharing chat rooms created by fan accounts following the artist, and music sharing chat rooms created by fan accounts following the artist are displayed.

[0259] When a tab (3100) for displaying a music sharing chat room is selected, a selection area (3200) is displayed that can distinguish between a currently active music sharing chat room and a music sharing chat room that was created and closed in the past.

[0260] When selecting the display of currently ongoing music sharing chat rooms, thumbnails for music sharing chat rooms scheduled for progress, thumbnails for music sharing chat rooms currently being conducted by artist accounts, and thumbnails for music sharing chat rooms currently being conducted by fan accounts are displayed separately as shown in FIG. 21.

[0261] When selecting to display music sharing chat rooms that were created and closed in the past, thumbnails of closed music sharing chat rooms are displayed as shown in FIG. 22, and thumbnails of music sharing chat rooms created and closed by artist accounts and thumbnails of music sharing chat rooms created and closed by fan accounts are displayed separately.

[0262] When selecting a thumbnail of a closed music sharing chat room, the user can experience the music sharing chat room again by receiving one or more of the shared music playlist information for the closed music sharing chat room, the host's music playback operation information, and chat information between the host account and the listener account from the music sharing playback service management server (100) stored in the database (130).

[0263] Figure 23 is an example of an interface for viewing a collection of music sharing chat rooms.

[0264] The music sharing chat room collection interface (3030) displays thumbnails for music sharing chat rooms that are scheduled to be created and run by the artist, music sharing chat rooms that are currently running, and music sharing chat rooms that have ended, on an artist-by-artist basis.

[0265] In addition, the music sharing chat room collection interface (3030) can display thumbnails of music sharing chat rooms that are scheduled to be created and run by fan accounts following the artist, music sharing chat rooms that are currently running, and music sharing chat rooms that have ended.

[0266] FIG. 24 is an example of a sound source sharing chat room start waiting interface (4000).

[0267] When a sound sharing chat room is created in the sound sharing chat room creation interface (1000), a sound sharing chat room start waiting interface (4000) as shown in FIG. 24 is activated.

[0268] Therefore, the audio sharing chat room start waiting interface (4000) is activated only for the host account.

[0269] The audio sharing chat room start waiting interface (4000) is created to include a shared audio information display area (4100) and a chat start / end selection area (4200).

[0270] The shared audio information display area (4100) displays audio information that is being played using the first audio streaming service linked with the host terminal (200-1). At this time, the image, title, and playback time of the audio being played may be displayed.

[0271] The chat start / end selection area (4200) displays a standby mode notification message and an icon that allows selecting to start or end the audio sharing chat room.

[0272] When you select the start icon, a music sharing chat room is opened, and listener accounts can participate.

[0273] If the host account does not play a track according to the shared music playlist, the track is not played and the music sharing chat room opens.

[0274] On the other hand, if the host account is playing a song according to the shared song playlist, a song sharing chat room is opened so that the song playback is shared at the time of the host account's playback.

[0275] FIG. 25 is an example of a sound sharing chat room interface (5000) displayed to the host and listener when a sound sharing chat room is opened.

[0276] The audio sharing chat room interface (5000) includes a chat room information display area (5100), an audio playback button display area (5200), an audio information display area currently playing (5300), a chat display area (5400), and a chat input area (5500).

[0277] The chat room information display area (5100) displays information regarding the number of real-time users, the number of users linked to the streaming service, the number of cumulative listeners, the number of cumulative hearts (likes), and the number of cumulative streams.

[0278] The number of real-time active users is displayed, reflecting the number of users participating in the music sharing chat room in real time since the chat room opened. The number of users linked to streaming services is displayed, reflecting the number of users who have linked their streaming services among those participating in the music sharing chat room in real time.

[0279] The cumulative listener count displays the total number of users who have joined the music sharing chat room since its opening. The cumulative heart (like) count is displayed by reflecting in real-time the number of users who have selected a heart (like) among those participating in the music sharing chat room.

[0280] The cumulative streaming count displays the number of streams played in real time by participating in the music sharing chat room and sharing the music.

[0281] The audio playback button display area (5200) displays a playlist icon and a play / pause icon. When the host account selects the playlist icon, the shared audio playlist settings interface (1410) is activated, and the shared audio playlist can be viewed or edited. When the listener account (a user account participating in the audio sharing chat room) selects the playlist icon, the shared audio playlist set by the host account can be viewed.

[0282] The play / pause icon is linked to a music streaming service linked to the host account or listener account, allowing you to pause or play music.

[0283] The currently playing audio information display area (5300) displays information regarding the audio currently being shared and played. The currently playing audio information display area (5300) displays the image, title, singer information, and playback time of the audio currently being played.

[0284] In the chat display area (5400), chats entered by the host account and the listener account are displayed in real time. At this time, messages entered by the host account are not displayed in real time, and a separate area for displaying host messages is set in the chat display area, and host messages can be fixedly displayed in that area.

[0285] The chat input area (5500) includes a chat input field where a host account or a listener account enters a chat and a send icon for sending the chat to another user.

[0286] When the chat input field is selected, a text input interface is activated, and the user can input chat using the text input interface.

[0287] FIGS. 26 and 27 are examples of a shared music playlist setting interface (1410) that is activated when a playlist icon is selected in the music playback button display area (5200).

[0288] As illustrated in FIG. 26, when a host account selects a playlist icon in the audio playback button display area (5200), the shared audio playlist setting interface (1410) is activated.

[0289] On the other hand, when a listener account selects a playlist icon in the music playback button display area (5200), only the shared music playlist information is displayed, and the shared music playlist settings interface (1410) is not activated.

[0290] As illustrated in FIG. 27, the host account can add new sounds, change the order of sounds, or delete sounds using the shared sound playlist setting interface (1410).

[0291] At this time, the shared audio playlist setting interface (1410) can display audio by displaying audio that has already finished playing, audio that is currently playing, and audio that is scheduled to be played differently.

[0292] For example, audio sources that have already finished playing may be displayed by shading them with a first color, and audio sources currently playing may be displayed by shading them with a second color. Additionally, audio sources scheduled for playback may be displayed without separate shading or by shading them with a third color.

[0293] Hereinafter, a sound source matching method according to an embodiment of the present invention will be described in detail with reference to FIG. 28.

[0294] Referring to FIG. 28, a sound source matching method according to an embodiment of the present invention includes a first metadata lookup step (S310), a candidate sound source search step (S320), a second metadata lookup step (S330), a similarity score derivation step (S340), and a sound source matching data storage step (S350).

[0295] In the first metadata lookup step (S310), the audio sharing playback service management server (100) receives first metadata for an audio file included in a shared audio playlist from the first audio streaming service management server linked with the stream terminal (200-1).

[0296] In the candidate sound source search step (S320), the sound source sharing playback service management server (100) searches for a candidate sound source corresponding to the track ID information of the sound source from the second sound source streaming service management server linked with the listener terminal (200-2) using the track ID information of the sound source included in the first metadata.

[0297] At this time, the audio sharing playback service management server (100) can search for candidate audio sources using not only the track ID of the audio source, but also the audio name, artist name, album name, disc number, track number, etc.

[0298] In the second metadata lookup step (S330), the audio sharing playback service management server (100) receives second metadata for candidate audio from the second audio streaming service management server.

[0299] In the similarity score derivation step (S340), the audio sharing playback service management server (100) derives the similarity between the first metadata and the second metadata and sets the audio with the highest similarity among the candidate audio as the matching audio.

[0300] In the similarity score derivation step (S340), the audio sharing playback service management server (100) derives a similarity based on the ISRC code, track name, album name, artist name, playback time, and obscenity status (whether playback is possible upon adult verification) included in the metadata of the audio.

[0301] In the similarity score derivation step (S340), the audio sharing playback service management server (100) excludes a candidate audio from the matching candidate audio if the difference in playback time between the audio included in the shared audio playlist and the candidate audio is 3 seconds or more, or if the degree of obscenity is different.

[0302] In the similarity score derivation step (S340), the audio sharing playback service management server (100) sets the initial similarity score of the candidate audio to 0 points and then calculates a similarity-based score of the first metadata and the second metadata to derive a similarity score.

[0303] In the similarity score derivation step (S340), the audio sharing playback service management server (100) grants 10 bonus points to the candidate audio source if the ISRC codes of the first metadata and the second metadata are identical.

[0304] In the similarity score derivation step (S340), the audio sharing playback service management server (100) assigns a score of 0 to 80 points based on the string similarity for each of the track name, album name, and artist name included in the first metadata and the second metadata.

[0305] In the similarity score derivation step (S340), the sound source sharing playback service management server (100) and the sound source matching data management module (142) grant an additional 10 points if the album name included in the first metadata and the second metadata is 100% identical and the disc number and track number are also identical.

[0306] In the similarity score derivation step (S340), the audio sharing playback service management server (100) derives a final similarity score for each candidate audio and sets the audio with the highest similarity score among the candidate audio as the audio that matches the audio included in the shared audio playlist.

[0307] In the sound source matching data storage step (S350), the sound source sharing playback service management server (100) sets a sound source of a second sound source streaming service that matches a sound source of a first sound source streaming service, and stores the sound source matching information in the database (130). The sound source matching information includes track ID information of the sound source for each sound source streaming service.

[0308] Accordingly, the music sharing playback service management server (100) transmits music track ID information corresponding to the linked music streaming service for each listener terminal (200-2) so that shared playback of the music is performed even if different music streaming services are used.

[0309] Hereinafter, a method for synchronizing audio playback according to an embodiment of the present invention will be described in detail with reference to FIG. 29.

[0310] Referring to FIG. 29, a sound source playback synchronization method according to an embodiment of the present invention includes a sound source playback operation data storage step (S410), a sound source playback operation information transmission step (S420), and a sound source playback synchronization execution step (S430).

[0311] In the audio playback operation data storage step (S410), the audio sharing playback service management server (100) stores audio playback operation execution information when performing an operation regarding audio playback using a first audio streaming service linked with a host terminal (200-1).

[0312] At this time, the audio sharing playback service management server (100) stores information regarding the time when the audio playback operation action occurred, the operation of the host, the host's connection status to the first audio streaming service, the track ID of the audio currently being played, the track ID of the audio to be played next, the playback position (in seconds) of the audio currently being played, and the playback status (play, pause) of the audio currently being played in the database (130).

[0313] In the audio playback operation information transmission step (S420), the listener terminal (200-2) requests synchronization information from the audio sharing playback service management server (100) at preset intervals.

[0314] The audio sharing playback service management server (100) transmits audio playback operation execution information stored in the database (130) to the listener terminal (200-2) in response to a synchronization information request from the listener terminal (200-2).

[0315] In the audio playback synchronization execution step (S430), the listener terminal (200-2) performs an audio playback operation in the first audio streaming service or the second audio streaming service linked with the listener terminal (200-2) based on audio playback operation execution information received from the audio sharing playback service management server (100).

[0316] Accordingly, the host terminal (200-1) and the listener terminal (200-2) can perform audio playback synchronization.

[0317] Hereinafter, a chat transmission and reception method according to an embodiment of the present invention will be described in detail with reference to FIG. 30.

[0318] Referring to FIG. 30, a chat transmission and reception method according to an embodiment of the present invention includes a chat transmission and reception step (S510), a host input chat fixed display step (S520), a forbidden word filtering step (S530), and a chat window spamming account disciplinary step (S540).

[0319] In the chat transmission and reception step (S510), the chat information entered and transmitted by the host terminal (200-1) and the listener terminal (200-2) using the chat input area (5500) is transmitted to the audio sharing playback service management server (100).

[0320] In the chat transmission and reception step (S510), the audio sharing playback service management server (100) separates the received chat information by audio sharing chat room and stores it in the database (130).

[0321] In the chat transmission and reception step (S510), the audio sharing playback service management server (100) displays the user's chat in the chat display area (5400) based on the received chat information, the user's nickname information, transmission time information, and chat information.

[0322] In the fixed display step (S520) for host input chat, the audio sharing playback service management server (100) can fix and display the chat entered by the host terminal (200-1) in the chat display area. Additionally, the host's chat and the listener's chat can be distinguished by setting different visual effects, such as displaying the host's chat with a different color or a different font size.

[0323] Additionally, in the host input chat fixed display step (S520), the audio sharing playback service management server (100) can automatically translate and display the chat entered by the host in the language set by the listener account.

[0324] In the forbidden word filtering step (S530), the audio sharing playback service management server (100) performs filtering so that the chat sent by the host terminal (200-1) and the listener terminal (200-2) does not contain a pre-set forbidden word, and the chat is not displayed in the chat display area (5400).

[0325] In the forbidden word filtering step (S530), the music sharing playback service management server (100) can set different forbidden words for each artist and fan.

[0326] Additionally, the music sharing and playback service management server (100) can convert fan message and artist message data into image data. Then, the music sharing and playback service management server (100) derives text corresponding to the converted image data.

[0327] The sound source sharing playback service management server (100) can identify words included in text derived from an image and determine whether the identified words correspond to forbidden words, thereby performing forbidden word filtering of artist messages and fan messages.

[0328] The audio sharing playback service management server (100) converts characters included in the chat into image data using a CNN (Convolutional Neural Network) learning model.

[0329] Specifically, when deriving text corresponding to image data, the sound source sharing and playback service management server (100) can convert the text into Korean consonants and vowels, respectively. For example, if a chat contains content such as "んЙ刀│○ㅑ" and '人ㅣ凹卜', the sound source sharing and playback service management server (100) converts the corresponding message into an image, derives consonants "ㅅ, ㄲ, ㅇ" and "ㅅ, ㅂ", and derives vowels "ㅐ, ㅣ, ㅑ" and "ㅣ, ㅏ". Then, "새끼야" and "시바", which are combinations of consonants and vowels, are derived as text.

[0330] In addition, CNN can be composed of a part that extracts image features and a part that classifies images in a manner that effectively recognizes and emphasizes features of adjacent images while maintaining spatial information of the image. The feature extraction area is composed of a Convolution layer that finds features of an image while minimizing the number of shared parameters using a filter, and a Pooling layer that enhances and collects features. Since it is possible to retrain and use CNN for a new recognition task based on an existing network, the CNN learning model can filter new malicious messages that are creatively modified to avoid forbidden words.

[0331] In addition, the sound source sharing and playback service management server (100) can filter whether the derived text contains forbidden words using a Bidirectional Encoder Representation of Transformer (BERT) learning model. The BERT learning model can be fine-tuned using the accuracy of the judgment result.

[0332] BERT is a pre-trained model used by Google to label and train on vast amounts of text on the web using semi-supervised learning. BERT is used in the embedding process when performing tasks such as named entity recognition and text classification.

[0333] As a method for the sound source sharing and playback service management server (100) to determine whether a forbidden word is included, a method using an algorithm that labels messages containing vocabulary pre-set by the user and a method of determining based on pre-set criteria may be used.

[0334] The pre-set criteria can be classified into expressions containing general profanity, vulgar and low-class expressions that offend others, sexually provocative expressions, expressions regarding physical threats, discriminatory expressions based on region / race / nationality / religion, etc., and derogatory expressions that cause humiliation and shame to the other party.

[0335] The music sharing playback service management server (100) can filter messages by outputting '0' as a value for messages containing profanity or slander, and outputting '1' as a value for messages not containing profanity or slander.

[0336] Additionally, the audio sharing playback service management server (100) identifies the context of the derived text and filters whether the fan message corresponds to a malicious message based on the identified context.

[0337] The audio sharing and playback service management server (100) classifies the context of a chat by performing encoding using bidirectional context analysis of the chat. More specifically, the audio sharing and playback service management server (100) can filter whether a fan message is a malicious message by inputting the fan message into a BERT (Bidirectional Encoder Representation of Transformer) learning model. The BERT learning model used in the audio sharing and playback service management server (100) receives a single embedding that combines the token embedding, segment embedding, and position embedding of the fan message to determine the context of the fan message.

[0338] In addition, the audio sharing playback service management server (100) prevents out of vocabulary of words constituting the chat by using word piece embeddings that segment words into sub-word units as token embeddings.

[0339] Specifically, the audio sharing playback service management server (100) can process token embedding using a word piece embedding method. In the case of word piece embedding, embedding can be performed in units smaller than words, and the longest sub-word can be set as a single unit. That is, frequently occurring sub-words can be used as units for embedding themselves, and rare words can be separated into sub-words. While existing word embedding methods include the problem of vocabulary loss, making it difficult to learn or translate rare words, names, numbers, or words that do not exist in the vocabulary list, the word piece embedding method can be applied to all languages ​​and is effective in solving the problem of vocabulary loss and increasing accuracy by segmenting words into sub-word units.

[0340] Additionally, the audio sharing and playback service management server (100) can receive two sentences along with a sentence separator ([SEP]). At this time, due to the input length limit, the total length of the two sentences may be limited to 512 subwords or less. That is, since the learning time increases exponentially as the length of the input increases, the length of the input must be set appropriately. Since Korean consists of an average of 20 subwords and 99% does not exceed 60 subwords, the input length may be limited to 128 when combining the two sentences. However, since long sentences may exist, a method may be used in which the input length is first limited to 128 for learning, and for inputs longer than 128, additional learning is performed in the final stage.

[0341] Additionally, the music sharing and playback service management server (100) may use position encoding. For this purpose, a Transformer model may be applied, and the Transformer model may use a Self-Attention model instead of a CNN or RNN model. In this case, since the Self-Attention model does not reflect the position of the input, additional position information regarding the input token must be input. Therefore, in the Transformer model, positional encoding using a Sinusoid function was used, and the music sharing and playback service management server (100) may use position encoding by modifying this. In position encoding, encoding can be performed simply in the order of tokens, such as 0, 1, 2, ...

[0342] Additionally, the music sharing and playback service management server (100) can combine token embeddings, segment embeddings, and position embeddings to generate a single embedding value (i.e., a single embedding). At this time, the single embedding can be generated in the form of a feature map and can be used after being transformed into a suitable form according to the subsequent operation process. For example, the music sharing and playback service management server (100) can use the result of applying layer normalization and dropout to the single embedding as input.

[0343] Additionally, the audio sharing and playback service management server (100) can increase the accuracy of malicious message judgment by providing the output malicious message encoding value to a fine tuning network. That is, the audio sharing and playback service management server (100) can receive two sentences regarding chat as input along with a sentence separator ([SEP]), and can be built through prior learning to generate a malicious message encoding value regarding the possibility of a malicious message as output through an internal inference step.

[0344] The music sharing and playback service management server (100) can basically perform malicious comment filtering operations by utilizing the BERT learning model as is, but can improve the accuracy of malicious message judgment through additional learning thereafter. To this end, the music sharing and playback service management server (100) can define a model architecture that combines the BERT learning model and a precision tuning network and utilize it in the malicious message judgment process.

[0345] The audio sharing playback service management server (100) can generate a feature map by providing the malicious message encoding value to a 1D Convolution Layer or by performing an ensemble operation based on a variable kernel size, and can determine the possibility of a malicious message for a fan message by providing the feature map to a BiLSTM (Bidirectional Long Short-Term Memory) network.

[0346] In addition, the audio sharing and playback service management server (100) can perform binary classification of the possibility of a malicious message by passing the result output through the BiLSTM network to the Affine and Softmax layers. Here, BiLSTM (Bidirectional Long Short-Term Memory) and bidirectional LSTM may correspond to a structure that uses two independent LSTM architectures together. First, the bidirectional LSTM can receive sentences sequentially.

[0347] In other words, it can receive sentences sequentially from left to right, just like a human. Additionally, the bidirectional LSTM can be used in conjunction with a reverse LSTM, which reads the sentence from right to left, to consider the context following the sentence. When making predictions in the output layer, the bidirectional LSTM can combine the outputs of the forward and reverse LSTMs to utilize both types of information.

[0348] These bidirectional LSTMs enable end-to-end learning by simultaneously learning all parameters while minimizing loss on output values, and can improve performance by internalizing similarity between words and phrases into input vectors. In addition, bidirectional LSTMs have the advantage of not degrading performance even when the data length is long, as a result of introducing the attention mechanism to the basic performance of LSTMs.

[0349] Additionally, the music sharing and playback service management server (100) can implement a malicious message filter by combining a BERT learning model and a CNN (Convolutional Neural Network) learning model. More specifically, the music sharing and playback service management server (100) can input a malicious message encoding value into a 1D Convolution Layer, input the output of the 1D Convolution Layer into a GeLU layer, input the output of the GeLU layer into a MaxPooling layer, and generate a binary classification result based on the output of the MaxPooling layer. At this time, a linear classification method may be applied to the binary classification.

[0350] For example, linear classification methods may include linear regression or linear classification models.

[0351] Additionally, the audio sharing and playback service management server (100) can implement a malicious message filter by combining a BERT learning model and an Ensemble CNN model. More specifically, the audio sharing and playback service management server (100) can perform a first step of performing an ensemble operation based on a variable kernel size based on a malicious message encoding value, a second step of independently inputting the results of the ensemble operation into a GeLU layer, a third step of independently inputting the outputs of the GeLU layer into a MaxPooling layer, a fourth step of sequentially connecting the outputs of the MaxPooling layer to generate an intermediate value, a fifth step of inputting the intermediate value into an Affine layer, a sixth step of inputting the output of the Affine layer into a softmax layer, and a seventh step of generating a binary classification result based on the result of the softmax layer.

[0352] Here, the variable kernel size-based ensemble operation may correspond to multiple convolution operations performed by applying different kernel sizes to the same malicious message encoding value. Additionally, the convolution operations may include 1D convolution operations. That is, the music sharing and playback service management server (100) can perform multiple convolution operations by independently applying kernels of different sizes to a feature map corresponding to the malicious comment encoding value. Subsequently, the music sharing and playback service management server (100) can perform a classification operation regarding the possibility of a fan message being a malicious message by going through a step of integrating the results of each convolution operation into one (i.e., the fourth step).

[0353] In addition, the audio sharing playback service management server (100) can implement a malicious message filter by combining a BERT learning model, an Ensemble CNN model, and a BiLSTM. More specifically, the audio sharing playback service management server (100) can perform a first step of performing an ensemble operation based on a variable kernel size based on an audio encoding value, a second step of independently inputting the results of the ensemble operation into a GeLU layer, a third step of independently inputting the outputs of the GeLU layer into a MaxPooling layer, a fourth step of sequentially connecting the outputs of the MaxPooling layer to generate an intermediate value, a fifth step of applying the intermediate value to a BiLSTM (Bidirectional LSTM) (Long Short-Term Memory) model, a sixth step of inputting the result of applying the BiLSTM model into an Affine layer, a seventh step of inputting the output of the Affine layer into a softmax layer, and an eighth step of generating a binary classification result based on the result of the softmax layer.

[0354] That is, the audio sharing playback service management server (100) can implement a malicious message filter by adding a step of applying an intermediate value to a BiLSTM (Bidirectional LSTM) (Long Short-Term Memory) model between the fourth and fifth steps of the method of implementing a malicious message filter by combining a BERT learning model and an Ensemble CNN model.

[0355] In the chat window spamming account disciplinary step (S540), the music sharing playback service management server (100) can perform disciplinary measures against an account that excessively inputs chat messages, such as prohibiting chat input and prohibiting the opening of music sharing chat rooms.

[0356] An account that excessively types in chat refers to an account that types in chat more than a preset number of times within a preset time, or repeatedly types in chat with the same content more than a preset number of times.

[0357] Hereinafter, a host voice sharing method according to an embodiment of the present invention will be described in detail with reference to FIGS. 31 and FIGS. 32.

[0358] A host voice sharing method according to an embodiment of the present invention includes a voice data selection step (S610), a host voice source generation step (S620), a listener terminal sound source playback pause step (S630), a host voice source streaming provision step (S640), and a listener terminal sound source playback resumption step (S650).

[0359] In the voice data selection step (S610), the audio source sharing playback service management server (100) receives voice data stored in the host terminal (200-1).

[0360] In the voice data selection step (S610), the host selects one or more voice data to be shared from among the voice data (host voice data) stored in the host terminal (200-1) and transmits them to the audio source sharing playback service management server (100).

[0361] Additionally, when the host inputs host voice to be shared with the listener in real time at the voice data selection step (S610) to the host terminal (200-1), the host terminal (200-1) transmits the input host voice data to the audio source sharing playback service management server (100).

[0362] In the host voice source generation step (S620), the voice source sharing playback service management server (100) stores the voice received from the host terminal (200-1) in the database (130) and generates a host voice source.

[0363] In the listener terminal audio playback pause step (S630), the audio sharing playback service management server (100) requests the listener terminal (200-2) to pause audio playback. Accordingly, the listener terminal (200-2) pauses the audio playback of the linked audio streaming service.

[0364] In the host voice source streaming provision step (S640), the sound source sharing playback service management server (100) performs streaming (playback) of the host voice source using a listener terminal (200-2).

[0365] In the listener terminal audio playback resumption step (S650), when the playback of the host audio source is completed, the listener terminal (200-2) transmits a host audio source playback completion signal to the audio source sharing playback service management server (100).

[0366] In the listener terminal audio playback resumption step (S650), when the audio sharing playback service management server (100) receives a host voice audio playback completion signal from the listener terminal (200-2), it sends a request to resume audio playback sharing to the listener terminal (200-2).

[0367] In the listener terminal audio playback resumption step (S650), the listener terminal (200-2) performs a synchronization request to the audio sharing playback service management server (100), and accordingly, can perform synchronized playback with the host terminal (200-1).

[0368] In addition, the host can share host voices by setting the order in which host voices are played.

[0369] Referring to FIG. 32, the host voice sharing method according to an embodiment of the present invention may further include a step (S621) of adding a host voice source to a shared sound source list.

[0370] In the step of adding a host voice source to a shared audio list (S621), the host terminal (200-1) can set a location where the host voice is to be played. At this time, the host can add the host voice source to a set location within the shared audio playlist.

[0371] In the listener terminal audio playback pause step (S630), the audio sharing playback service management server (100) requests the listener terminal (200-2) to pause audio playback sharing when the playback of the previous audio of the host voice audio in the shared audio playlist is completed.

[0372] In the host voice source streaming provision step (S640), the sound source sharing playback service management server (100) performs streaming (playback) of the host voice source using a listener terminal (200-2).

[0373] In the listener terminal audio playback resumption step (S650), when the playback of the host audio source is completed, the listener terminal (200-2) transmits a host audio source playback completion signal to the audio source sharing playback service management server (100).

[0374] In the listener terminal audio playback resumption step (S650), when the audio sharing playback service management server (100) receives a signal indicating completion of host voice audio playback from the listener terminal (200-2), it sends a request to resume audio playback sharing to the listener terminal (200-2). Accordingly, the listener terminal (200-2) can perform playback of the audio that is next in sequence to the host voice audio.

[0375] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will understand that other specific forms can be easily modified without altering the technical spirit or essential features of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form.

[0376] The scope of the present invention is defined by the claims set forth below rather than by the detailed description above, and all modifications or variations derived from the meaning and scope of the claims and equivalent concepts thereof should be interpreted as being included within the scope of the present invention. Explanation of the symbols

[0377] 100: Music sharing management server 200: Terminal 110: Communication module 120: Memory 130: Database 140: Processor 210: Communication module 220: Memory 230: Input / Output Module 240: Processor

Claims

Claim 1 A method for sharing audio playback among multiple terminals comprises: a step in which a host terminal creates a shared audio playlist and transmits the shared audio playlist data to an audio sharing management server; a step in which audio playback corresponding to the shared audio playlist is performed using a first audio streaming application installed on a first listener terminal; a step in which audio playback corresponding to the shared audio playlist is performed using a second audio streaming application installed on a second listener terminal; a step in which a host voice audio source is generated based on voice data stored in the host terminal; and a step in which the first listener terminal and the second listener terminal receive and play the host voice audio source from the audio sharing management server, wherein the first audio streaming application refers to an audio streaming application used by the host terminal, and the second audio streaming application refers to an audio streaming application different from the first audio streaming application; a step in which the audio sharing management server receives audio playback operation information of the host terminal and stores audio playback operation information of the host terminal; and, based on the audio playback operation information of the host terminal, the host terminal and the A method for sharing audio playback, further comprising the step of synchronizing audio playback of a first listener terminal or the second listener terminal, wherein the step of synchronizing audio playback includes: the first listener terminal or the second listener terminal performing a synchronization request at a preset period; providing audio playback operation information to the first listener terminal or the second listener terminal; operating audio playback of a first audio streaming application using the first listener terminal based on the audio playback operation information; and operating audio playback of a second audio streaming application using the second listener terminal based on the audio playback operation information. Claim 2 A method for sharing audio playback according to claim 1, wherein the step of the first listener terminal and the second listener terminal receiving and playing the host voice audio source from the audio source sharing management server comprises: the step of temporarily suspending audio playback using the first audio streaming application and the second audio streaming application; the step of the first listener terminal and the second listener terminal receiving and playing the host voice audio source from the audio source sharing management server; and the step of the first listener terminal and the second listener terminal resuming audio playback using the first audio streaming application and the second audio streaming application when the playback of the host voice audio source is completed. Claim 3 A method for sharing audio playback according to claim 2, wherein the step of generating the host voice source comprises: adding the host voice source to the shared audio playlist; temporarily suspending audio playback using the first audio streaming application and the second audio streaming application when the playback of the audio source preceding the host voice source in the shared audio playlist is completed; receiving and playing the host voice source from the audio sharing management server by the first listener terminal and the second listener terminal; and when the playback of the host voice source is completed, resuming the playback of the audio source following the host voice source in the shared audio playlist by the first listener terminal and the second listener terminal using the first audio streaming application and the second audio streaming application. Claim 4 In paragraph 3, the step of performing sound playback corresponding to the shared sound playlist using the second sound streaming application comprises: a step in which the sound sharing management server searches for the sound with the highest similarity among the sound available in the second sound streaming application based on sound information included in the shared sound playlist; and a step of matching the sound with the highest similarity to the sound included in the shared sound playlist. Claim 5 A method for sharing music playback according to claim 4, wherein the step of transmitting the shared music playlist data to a music sharing management server includes the step of linking a first music streaming application installed on the host terminal and generating the shared music playlist based on the music information of the first music streaming application. Claim 6 A method for sharing audio playback according to claim 5, wherein the step of searching for the audio source with the highest similarity comprises: receiving first metadata for an audio source included in the shared audio playlist from the first audio streaming application management server; searching for one or more corresponding candidate audio sources in the first audio streaming application based on the first metadata for an audio source included in the shared audio playlist; receiving second metadata for the candidate audio source from the second audio streaming application management server; deriving the similarity between the first metadata and the second metadata; and setting the audio source with the highest similarity among the candidate audio sources as an audio source that matches the audio source included in the shared audio playlist. Claim 7 A method for sharing audio playback according to claim 6, wherein the step of deriving similarity between the first metadata and the second metadata comprises the step of deriving similarity based on the similarity between one or more of the playback time, ISRC code, selected audio name, artist name, and album name information included in the first metadata and the playback time, ISRC code, selected audio name, artist name, and album name information included in the second metadata. Claim 8 A method for sharing audio playback according to claim 7, wherein the step of setting the audio with the highest similarity among the candidate audio sources as the audio source that matches the audio source included in the shared audio playlist comprises: the step of setting matching management information for managing the matching between the audio source with the highest similarity among the candidate audio sources and the audio source included in the shared audio playlist; and the step of providing audio information corresponding to the first audio streaming service or the second audio streaming service to the first listener terminal or the second listener terminal based on the matching management information. Claim 9 delete Claim 10 delete Claim 11 A device providing a music sharing service for sharing music playback among multiple terminals comprises at least one memory and at least one processor that executes at least one application stored in the memory to perform a music sharing service, wherein the instructions of the processor include: a step in which a host terminal creates a shared music playlist and transmits the shared music playlist data to a music sharing management server; a step in which music playback corresponding to the shared music playlist is performed using a first music streaming application installed on a first listener terminal; a step in which music playback corresponding to the shared music playlist is performed using a second music streaming application installed on a second listener terminal; a step in which a host voice sound source is generated based on voice data stored in the host terminal; and a step in which the first listener terminal and the second listener terminal receive and play the host voice sound source from the music sharing management server, wherein the first music streaming application refers to a music streaming application used by the host terminal, and the second music streaming application refers to a music streaming application different from the first music streaming application, and the instructions of the processor include the host A step of receiving audio playback operation information of a terminal and storing audio playback operation information of the host terminal; a step of synchronizing audio playback of the host terminal and the first listener terminal or the second listener terminal based on the audio playback operation information of the host terminal; a step of the first listener terminal or the second listener terminal performing a synchronization request at a preset period; a step of providing the audio playback operation information to the first listener terminal or the second listener terminal; and a step of operating audio playback of a first audio streaming application using the first listener terminal based on the audio playback operation information.A computing device further comprising the step of operating the playback of a second audio streaming application using the second listener terminal based on the above audio playback operation information. Claim 12 A computing device according to claim 11, wherein the instructions of the processor further include the step of temporarily suspending audio playback using the first audio streaming application and the second audio streaming application; the step of the first listener terminal and the second listener terminal receiving and playing the host voice audio source from the audio sharing management server; and the step of, when the playback of the host voice audio source is completed, the first listener terminal and the second listener terminal resuming audio playback using the first audio streaming application and the second audio streaming application. Claim 13 A computing device according to claim 12, wherein the instructions of the processor further include: adding the host voice source to the shared sound source playlist; temporarily suspending sound source playback using the first sound source streaming application and the second sound source streaming application when the playback of a sound source prior to the host voice source in the shared sound source playlist is completed; the first listener terminal and the second listener terminal receiving and playing the host voice source from the sound source sharing management server; and when the playback of the host voice source is completed, the first listener terminal and the second listener terminal resuming the playback of a sound source following the host voice source in the shared sound source playlist using the first sound source streaming application and the second sound source streaming application. Claim 14 A computing device according to claim 13, wherein the instructions of the processor further include the step of searching for the sound source with the highest similarity among the sound sources available in the second sound source streaming application based on sound source information included in the shared sound source playlist, and the step of matching the sound source with the highest similarity to the sound source included in the shared sound source playlist. Claim 15 A computing device according to claim 14, wherein the instructions of the processor further include the step of generating the shared sound source playlist based on the sound source information of the first sound source streaming application by linking with the first sound source streaming application installed on the host terminal. Claim 16 A computing device according to claim 15, wherein the instructions of the processor further include: receiving first metadata for a sound source included in the shared sound source playlist from the first sound source streaming application management server; searching for one or more corresponding candidate sound sources in the first sound source streaming application based on the first metadata for a sound source included in the shared sound source playlist; receiving second metadata for the candidate sound source from the second sound source streaming application management server; deriving a similarity between the first metadata and the second metadata; and setting the sound source with the highest similarity among the candidate sound sources as the sound source that matches the sound source included in the shared sound source playlist. Claim 17 A computing device according to claim 16, wherein the instruction of the processor further comprises the step of deriving a similarity based on the similarity of one or more of the playback time, ISRC code, selected sound source name, artist name, and album name information included in the first metadata and the playback time, ISRC code, selected sound source name, artist name, and album name information included in the second metadata. Claim 18 A computing device according to claim 17, wherein the instructions of the processor further include the step of setting matching management information for managing the matching of the sound source with the highest similarity among the candidate sound sources and the sound source included in the shared sound source playlist, and the step of providing sound source information corresponding to the first sound source streaming service or the second sound source streaming service to the first listener terminal or the second listener terminal based on the matching management information. Claim 19 delete Claim 20 delete

Images