Multimedia scoring system and method

Abstract

The present invention discloses a multimedia scoring system and method, wherein the multimedia scoring system comprises a database, a karaoke main system, a mixing module and an auxiliary scoring system. The karaoke main system obtains designated song data and designated song scoring timing data from the database according to a song designation command, and calculates a synchronization starting point according to the synchronization timing and the designated song scoring timing data, so as to generate a starting point indication signal correspondingly, and transmit the starting point indication signal and the designated song scoring timing data. The auxiliary scoring system samples the vocal data, and executes a song scoring program when the synchronization starting point is reached according to the synchronization starting point indicated by the starting point indication signal.

Description

Technical Field

[0001] The present invention relates to a multimedia scoring system and method, and in particular to a multimedia scoring system and method capable of greatly improving the applicable platform range of karaoke scoring technology and functions. Background Art

[0002] Although most of the existing karaoke systems have been commercialized, most of them do not have professional intelligent scoring functions. Many electronic devices such as smart TVs or TV boxes are unable or difficult to record for scoring when running karaoke systems, making it impossible to obtain recording files for real-time scoring analysis.

[0003] On the other hand, the existing karaoke scoring technology mostly uses the karaoke application running on the karaoke system to score after the singing voice is fully recorded. However, when the system running the karaoke software is a smart TV, a TV set-top box or a karaoke machine, many systems do not fully consider supporting the recording function, so that the scoring software cannot obtain the singing voice for analysis and scoring, and therefore the karaoke scoring function cannot be realized. Alternatively, when the device for recording the singing voice and the karaoke system operate independently, it is difficult to achieve the goal of real-time scoring due to the delay in timing.

[0004] Therefore, there is an urgent need for a multimedia scoring system and method that can greatly improve the applicable platform range of karaoke scoring technology and functions, and can accurately achieve timing synchronization for real-time scoring. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a multimedia scoring system and method that can greatly improve the applicable platform range of karaoke scoring technology and functions in view of the shortcomings of the prior art.

[0006] In order to solve the above-mentioned technical problems, one of the technical solutions adopted by the present invention is to provide a multimedia scoring system, which includes: a database, storing multiple song data and corresponding multiple song scoring timing data; a first timing unit, used to generate a synchronization timing signal for indicating a synchronization timing; a karaoke main system, electrically connected to the database and including: a first communication module, configured to transmit a synchronization request signal and obtain the synchronization timing signal; and a first processing unit, configured to obtain a specified song data and a specified song scoring timing data from the database according to a song specifying command, and calculate a synchronization starting point according to the synchronization timing and the specified song scoring timing data, so as to generate a starting point indication signal for indicating the synchronization starting point; a mixing module, configured to collect sound and generate human voice data; and an auxiliary scoring system, electrically connected to the karaoke main system. The system and the mixing module include: a second communication module, configured to obtain the synchronization timing signal from the first communication module, and receive the starting point indication signal and the designated song scoring timing data; a sampling unit, configured to sample the vocal data; and a second processing unit, configured to obtain the designated song scoring timing data, and execute a song scoring program when the synchronization starting point is reached according to the synchronization starting point indicated by the starting point indication signal, wherein the song scoring program includes: controlling the sampling unit to continuously sample the part after the starting point in the vocal data to generate a sampled audio data; comparing the sampled audio data with the designated song scoring timing data to obtain a comparison result; scoring according to the comparison result and correspondingly generating a scoring result data; and transmitting the scoring result data to the K song main system through the second communication module.

[0007] Preferably, the karaoke main system also includes a user interface for a user to input the song designation command and play the designated song data according to the song designation command.

[0008] Preferably, the first communication module is configured to transmit the synchronization request signal multiple times to obtain the synchronization timing signal multiple times.

[0009] Preferably, the second communication module is configured to transmit the synchronization timing signal to the first communication module.

[0010] Preferably, the first communication module and the second communication module are configured to be connected to the first timing unit via a network to obtain the synchronous timing signal, and the synchronous timing signal is a network clock timing signal.

[0011] Preferably, the karaoke main system also includes an audio playback module, and the first processing unit is configured to control the audio playback module to play the designated song data.

[0012] Preferably, the mixing module also includes a third communication module, which is configured to transmit the vocal data to the K song main system, and in response to receiving the vocal data, the first processing unit is also configured to mix the designated song data with the vocal data to generate a mixed data, and control the audio playback module to play the mixed data.

[0013] Preferably, the K song master system also includes a second timing unit, which is configured to generate another timing synchronization signal, and the first communication module and the second communication module are also configured to allocate the K song master system and the auxiliary scoring system as a main synchronization device and a slave synchronization device according to the first communication protocol to determine whether to synchronize between the K song master system and the auxiliary scoring system with the timing synchronization signal or the other timing synchronization signal.

[0014] Preferably, the database is set in a network server, and the first processing unit is configured to control the first communication module to obtain the designated song data and the designated song score timing data from the database through the network.

[0015] Preferably, the auxiliary scoring system and the sound mixing module are arranged in a sound mixing attachment device, and the sound mixing attachment device is further electrically connected to a sound receiving unit.

[0016] Preferably, the auxiliary scoring system includes a fifth communication module, the mixing module includes a third communication module and a fourth communication module, the third communication module is connected to the karaoke main system, and the fourth communication module is configured to be connected to the fifth communication module through a second network interface, wherein the second communication module is connected to the first communication module through a first network interface.

[0017] Preferably, the mixing module includes a third communication module, a fourth communication module and a sixth communication module, the auxiliary scoring system also includes a fifth communication module, the third communication module is configured to be connected to the K song main system through a first network interface, the second communication module is connected to the first communication module through the network, and the sixth communication module is configured to be connected to the fifth communication module through a second network interface.

[0018] Preferably, the K song master system also includes a second timing unit, which is configured to generate another timing synchronization signal, and the first communication module and the second communication module are also configured to allocate the K song master system and the auxiliary scoring system as a main synchronization device and a slave synchronization device through the fourth communication module, the fifth communication module and the sixth communication module according to the first communication protocol to determine whether to synchronize between the K song master system and the auxiliary scoring system with the timing synchronization signal or the other timing synchronization signal.

[0019] In order to solve the above-mentioned technical problems, another technical solution adopted by the present invention is to provide a multimedia scoring method, which includes the following steps: configuring a first processing unit of a karaoke main system to obtain a specified song data and a specified song scoring timing data from the database according to a song specifying command; configuring a first timing unit to generate a synchronization timing signal for indicating a synchronization timing; configuring a first communication module of the karaoke main system to be connected with a second communication module of the auxiliary scoring system through a first communication protocol, and configuring the first communication module and the second communication module to obtain the synchronization timing signal; configuring the first processing unit to calculate a synchronization starting point according to the synchronization timing and the specified song scoring timing data, so as to generate a starting point indication signal for indicating the synchronization starting point; configuring the first processing unit to calculate a synchronization starting point according to the synchronization timing and the specified song scoring timing data, so as to generate a starting point indication signal for indicating the synchronization starting point; configuring the first processing unit to calculate a synchronization starting point according to the synchronization timing and the specified song scoring timing data, so as to generate a starting point indication signal for indicating the synchronization starting point; configuring the first processing unit to calculate a synchronization starting point according to the synchronization timing and the specified song scoring timing data, and configuring the first timing unit to generate a synchronization timing ... A communication module transmits the starting point indication signal and the designated song scoring timing data to the auxiliary scoring system; a mixing module is configured to collect sound and generate vocal data; and a second processing unit of the auxiliary scoring system is configured to obtain the designated song scoring timing data, and according to the synchronization starting point indicated by the starting point indication signal, a song scoring program is executed when the synchronization starting point is reached, wherein the song scoring program includes the following steps: controlling the sampling unit to continuously sample the part after the starting point in the vocal data to generate a sampled audio data; comparing the sampled audio data with the designated song scoring timing data to obtain a comparison result; scoring according to the comparison result and generating a corresponding scoring result data; and transmitting the scoring result data to the karaoke main system through the second communication module.

[0020] Preferably, the multimedia scoring method further includes configuring a user interface for a user to input the song specifying command, and configuring the user interface to play the specified song data according to the song specifying command.

[0021] Preferably, the multimedia scoring method further comprises configuring the first communication module to transmit the synchronization request signal multiple times to obtain the synchronization timing signal multiple times.

[0022] Preferably, the multimedia scoring method further comprises configuring the second communication module to transmit the synchronization timing signal to the first communication module.

[0023] Preferably, the multimedia scoring method further comprises configuring the first communication module and the second communication module to be connected to the first timing unit via a network, and obtaining the synchronous timing signal via the network, wherein the synchronous timing signal is a network clock timing signal.

[0024] Preferably, the multimedia scoring method further includes configuring the first processing unit to control an audio playback module of the karaoke main system to play the designated song data.

[0025] Preferably, the multimedia scoring method also includes: configuring a third communication module of the mixing module to transmit the vocal data to the K song main system; and in response to receiving the vocal data, configuring the first processing unit to mix the designated song data with the vocal data to generate a mixed data, and controlling the audio playback module to play the mixed data.

[0026] Preferably, the multimedia scoring method also includes: configuring a second timing unit of the K song main system to generate another timing synchronization signal; configuring the first communication module and the second communication module to allocate the K song main system and the auxiliary scoring system as a main synchronization device and a slave synchronization device according to the first communication protocol to determine whether to use the timing synchronization signal or the other timing synchronization signal to synchronize between the K song main system and the auxiliary scoring system.

[0027] Preferably, the database is set in a network server, and the multimedia scoring method also includes configuring the first processing unit to control the first communication module to obtain the designated song data and the designated song scoring timing data from the database through the network.

[0028] Preferably, the auxiliary scoring system and the sound mixing module are arranged in a sound mixing attachment device, and the sound mixing attachment device is further electrically connected to a sound receiving unit.

[0029] Preferably, the auxiliary scoring system includes a fifth communication module, the mixing module includes a third communication module and a fourth communication module, and the multimedia scoring method also includes: configuring the second communication module to be connected to the first communication module through a first network interface; configuring the third communication module to be connected to the karaoke main system; and configuring the fourth communication module to be connected to the fifth communication module through a second network interface.

[0030] Preferably, the mixing module includes a third communication module, a fourth communication module and a sixth communication module, the auxiliary scoring system also includes a fifth communication module, and the multimedia scoring method also includes: configuring the third communication module to be connected to the K song main system through a first network interface; configuring the second communication module to be connected to the first communication module through the network; configuring the sixth communication module to be connected to the fifth communication module through a second network interface.

[0031] Preferably, the multimedia scoring method also includes: configuring a second timing unit of the K song main system to generate another timing synchronization signal; and configuring the first communication module and the second communication module to allocate the K song main system and the auxiliary scoring system as a main synchronization device and a slave synchronization device according to the first communication protocol through the fourth communication module, the fifth communication module and the sixth communication module, so as to decide to synchronize between the K song main system and the auxiliary scoring system using the timing synchronization signal or the other timing synchronization signal.

[0032] One of the beneficial effects of the present invention is that the multimedia scoring system and method provided by the present invention does not require the karaoke main system to support the recording function. It only needs data communication and timing synchronization between the karaoke main system and the auxiliary scoring system, and can transmit specific data through the encoding and decoding of the played audio. Therefore, the applicable platform range of karaoke scoring technology and functions can be greatly increased, including smart TVs or TV set-top boxes, karaoke accompaniment machines, etc.

[0033] To further understand the features and technical contents of the present invention, please refer to the following detailed description and drawings of the present invention. However, the drawings provided are only for reference and description and are not intended to limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 FIG. 4 is a system block diagram of a multimedia rating system according to a first embodiment of the present invention.

[0035] Figure 2 4 is a flowchart of a multimedia scoring method according to an embodiment of the present invention.

[0036] Figure 3 FIG. 4 is a schematic diagram of the architecture of a multimedia rating system according to another embodiment of the present invention.

[0037] Figure 4 FIG. 4 is a system block diagram of a multimedia rating system according to another embodiment of the present invention.

[0038] Figure 5 FIG. 4 is a schematic diagram of the architecture of a multimedia rating system according to another embodiment of the present invention.

[0039] Fig. 6A and Figure 6B FIG. 4 is a flowchart of a multimedia scoring method according to another embodiment of the present invention.

[0040] Figure 7 FIG. 4 is a system block diagram of a multimedia rating system according to another embodiment of the present invention. DETAILED DESCRIPTION

[0041] The following is an explanation of the implementation methods of the "Multimedia Scoring System and Method" disclosed in the present invention through specific embodiments. Those skilled in the art can understand the advantages and effects of the present invention from the contents disclosed in this specification. The present invention can be implemented or applied through other different specific embodiments, and the details in this specification can also be modified and changed in various ways based on different viewpoints and applications without departing from the concept of the present invention. In addition, it is stated in advance that the drawings of the present invention are only simple schematic illustrations and are not depicted according to actual sizes. The following implementation methods will further explain the relevant technical contents of the present invention in detail, but the disclosed contents are not intended to limit the scope of protection of the present invention.

[0042] It should be understood that, although the terms "first", "second", "third", etc. may be used herein to describe various components or signals, these components or signals should not be limited by these terms. These terms are mainly used to distinguish one component from another component, or one signal from another signal. In addition, the term "or" used herein may include any one or more combinations of the associated listed items depending on the actual situation.

[0043] See also Figure 1 As shown, an embodiment of the present invention provides a multimedia scoring system 1, which includes a database 10, a karaoke main system 12, a mixing module 14 and an auxiliary scoring system 16.

[0044] The database 10 stores a plurality of song data 100 and a plurality of corresponding song scoring time series data 102. In an embodiment of the present invention, a plurality of song data 100 to be used in the karaoke main system 12 may be stored in the database 10, and the song data 100 may include, for example, song data sung by a specific singer, unaccompanied vocal data, lyrics data, and music video data. On the other hand, the song scoring time series data 102 may include original singing audio data corresponding to the plurality of song data 100 and subsequently used to score the singing content of the user, and edited with time tags corresponding to each line of lyrics, as well as original singing audio data, pitch, beat, length, volume data, and music score data corresponding to the time tags.

[0045] The karaoke main system 12 of the present invention is further described. The device implementing the karaoke main system 12 may include hardware, firmware and / or software, and may be implemented in any of a variety of ways. Typical examples of this approach include laptops, smart phones, small personal computers, personal digital assistants, smart TVs, cloud set-top boxes, and the like. The functions described herein may also be implemented in peripheral devices or built-in cards. By further example, this function may also be implemented on different chips or circuit boards with different programs executed on a single device. For example, the karaoke main system 12 may be used to execute karaoke software in the above-mentioned devices, or to play backing tracks.

[0046] like Figure 1 As shown, the karaoke main system 12 is electrically connected to the database 10, and includes a user interface 120, a first communication module 122, and a first processing unit 124. The karaoke main system 12 may include at least one controllable display device for displaying the user interface 120, and may be, for example, an interface for user control, such as a touch interface, and the user may input various instructions through the user interface 120, such as karaoke software execution commands, play commands, stop commands, and song designation commands. The user interface 120 may further provide a window corresponding to the song data 100 for the user to select a designated song to sing.

[0047] The first communication module 122 can be used to electrically connect the karaoke main system 12 to the database 10. Alternatively, the database 10 can be built into a device for executing the karaoke main system 12, directly electrically connected to the karaoke main system 12, or connected via a wired network or a wireless network communication, but is not limited to the above methods. The first communication module 122 can be, for example, a USB communication module, an Ethernet communication module, a wireless network communication module, or a wireless Bluetooth communication module. Another purpose of the first communication module 122 is to transmit a synchronization request signal S1 to obtain a synchronization timing signal S2 for indicating a synchronization timing. This synchronization timing signal S2 will be used for subsequent time synchronization of song playback and scoring, and will be described below.

[0048] In other words, the first processing unit 124 may be configured to obtain the designated song data 20 and the designated song score time sequence data 22 from the database 10 according to the song designation command received from the user interface 120, and control the user interface 120 to play the designated song data 20 according to the song designation command. In detail, the designated song data 20 and the designated song score time sequence data 22 are one of the song data 100 selected from the plurality of song data 100 and its corresponding song score time sequence data 102 specified by the user.

[0049] On the other hand, the functions of the first processing unit 124 can be implemented by using one or more processing units. The first processing unit 124 can be a programmable unit, such as a microprocessor, a microcontroller, a digital signal processor (DSP) chip, a field programmable gate array (FPGA), etc. The functions of the first processing unit 124 can also be implemented by one or more electronic devices or ICs. In other words, the functions performed by the first processing unit 124 can be implemented in the hardware domain or the software domain or a combination of the hardware domain and the software domain.

[0050] Further, the mixing module 14 of the present invention is first described. The mixing module 14 is electrically connected to the sound receiving unit 18, and can be configured to receive sound through the sound receiving unit 18 and generate vocal data 24. The mixing module 14 can integrate sounds from multiple sources into a stereo track (Stereo) or a mono track (Mono), and can adjust the frequency, dynamics, sound quality, positioning, reverberation and sound field of each individual original signal according to needs, so that each track is optimized, and then superimposed with the vocal data 24. The mixing module 14 may include a synthesizer (Synthesizer), a sound processor (Signal Processor) and a mixing console (Mixing Console), or software mixing can be used to achieve the effects of the above devices.

[0051] In addition, the sound receiving unit 18 may be, for example, a microphone, and may be electrically connected to the sound mixing module 14 via a wired or wireless method. The sound mixing module 14 may be electrically connected to the karaoke main system 12 and the auxiliary scoring system 16 via a wired or wireless method.

[0052] In a specific embodiment, the mixing module 14 may simply transmit the unmixed vocal data 24 to the karaoke main system 12. After the karaoke main system 12 mixes the vocal data 24 to generate mixed data, the mixed data is broadcast through the audio playback unit (e.g., speaker) of the karaoke main system 12.

[0053] In an alternative embodiment, the mixing module 14 itself can optimize the mixing of the vocal data 24 and the designated song data 20 to generate the mixed data, and transmit it to the karaoke main system 12 to broadcast the mixed data through the audio playback unit (e.g., speaker) of the karaoke main system 12. Therefore, after being broadcast through the audio playback unit, the singer can listen to his own singing. At the same time, in the above embodiment, the mixing module 14 also transmits the unmixed vocal data 24 to the auxiliary scoring system 16 for analysis, comparison and scoring when the singing is subsequently scored.

[0054] Before describing the details of song scoring, the auxiliary scoring system 16 of the present invention is first described. In order to solve the problem that the existing karaoke main system has no professional intelligent scoring function, the multimedia scoring system 1 of the present invention further provides an auxiliary scoring system 16, which is electrically connected to the karaoke main system 12 and the mixing module 14, and includes a second communication module 160, a first timing unit 162, a sampling unit 164 and a second processing unit 166. The device implementing the auxiliary scoring system 16 may include hardware, firmware and / or software, and may be implemented in any of a variety of ways. Typical examples of this approach include laptops, smart phones, small personal computers, personal digital assistants, or plug-and-play USB devices, etc. The functions described herein may also be implemented in peripheral devices or built-in cards. By further example, this function may also be implemented in circuit boards of different programs executed on different chips or on a single device. In this embodiment, although the first timing unit 162 is included in the auxiliary scoring system 16 and is used to provide a synchronous timing signal S2, the present invention is not limited thereto, and another timing unit may also be provided in the karaoke main system 12, and will be described in subsequent embodiments.

[0055] The second communication module 160 is used to connect with the first communication module 122 through a first communication protocol, for example, a USB communication protocol, and the second communication module 160 can be used to obtain the synchronization timing signal S2 from the first timing unit 162. Similarly, the second communication module 160 can be, for example, a USB communication module, an Ethernet communication module, a wireless network communication module, or a wireless Bluetooth communication module.

[0056] Generally speaking, when performing real-time song scoring between the karaoke main system 12 and the auxiliary scoring system 16, the difficulty lies in the different timings of the two, and it is difficult to achieve the goal of real-time feedback of song scoring. Usually, the entire song must be recorded, and the singing scoring result can only be known at the end of the song. Therefore, in order to synchronize the signal streams of the two, in an embodiment of the present invention, when the first processing unit 124 controls the user interface 120 to play the specified song data 20 according to the song designation command, the first communication module 122 is first controlled to send a synchronization request signal S1 to the second communication module 160, and in response to this, the second communication module 160 transmits the obtained synchronization timing signal S2 to the first communication module 122. After obtaining the synchronization timing, the first processing unit 124 simultaneously calculates the synchronization starting point according to the synchronization timing and the specified song scoring timing data 22, so as to generate a starting point indication signal S3 for indicating the synchronization starting point, and transmits the starting point indication signal S3 and the specified song scoring timing signal S4 with the specified song scoring timing data 22 to the auxiliary scoring system 16 through the first communication module 122. For example, the first processing unit 124 may obtain the time point at which the first line of lyrics in the designated song rating timing data 22 begins as the synchronization starting point to generate the corresponding starting point indication signal S3.

[0057] Next, the second communication module 160 further receives the start point indication signal S3 and the designated song score timing signal S4. The sampling unit 164 can receive the vocal data 24 from the mixing module 14 and sample the vocal data 24. Sampling can include converting the vocal data 24, which is originally an analog signal, into a digital signal that can be recognized or processed by the second processing unit 166 for use in subsequent song scoring.

[0058] Similarly, the functions of the second processing unit 166 can be implemented by using one or more processing units. The second processing unit 166 can be a programmable unit, such as a microprocessor, a microcontroller, a digital signal processor (DSP) chip, a field programmable gate array (FPGA), etc. The functions of the second processing unit 166 can also be implemented by one or more electronic devices or ICs. In other words, the functions performed by the second processing unit 166 can be implemented in the hardware domain or the software domain or a combination of the hardware domain and the software domain.

[0059] Furthermore, the designated song scoring timing signal S4 may have a specific signal format, and therefore the designated song scoring timing signal S4 must first be decoded by the second processing unit 166 to obtain the designated song scoring timing data 22, which includes a time tag corresponding to each line of lyrics of the designated song data 20, and the original audio data, pitch, beat, length, volume data and music score data corresponding to the time tags, and executes the song scoring program when the synchronization starting point is reached according to the synchronization starting point indicated by the starting point indication signal S3.

[0060] The song scoring procedure will be described together with the embodiment of the multimedia scoring method provided by the present invention. Figure 2 , which is a flow chart of a multimedia scoring method according to an embodiment of the present invention. It should be noted that the multimedia scoring method of this embodiment can be applied to the multimedia scoring system of the above-mentioned embodiment, but is not limited thereto.

[0061] like Figure 2 As shown, for the convenience of explanation, the process is divided according to the configuration of the karaoke main system and the auxiliary scoring system, and repeated descriptions in some steps will be omitted. The multimedia scoring method of the embodiment of the present invention includes the following steps:

[0062] Step S100: configuring the user interface 120 to receive a song specifying command input by a user.

[0063] Step S101: configuring the first processing unit 124 to obtain the designated song data 20 and the designated song score time series data 22 from the database 10 according to the song designation command.

[0064] Step S102: configure the first timing unit 162 to generate a synchronization timing signal S2 for indicating a synchronization timing.

[0065] Step S103: configure the first communication module 122 to connect with the second communication module 160 via the first communication protocol to obtain the synchronization timing signal S2.

[0066] Step S104 : configuring the first processing unit 124 to control the user interface 120 to play the designated song data 20 .

[0067] Step S105: configuring the first processing unit 124 to calculate the synchronization start point according to the synchronization timing and the designated song rating timing data 22, so as to correspondingly generate a start point indication signal S3 for indicating the synchronization start point.

[0068] Step S106 : transmitting the starting point indication signal S3 and the designated song rating timing signal S4 to the auxiliary rating system 16 via the first communication module 122 .

[0069] Step S107: configure the second processing unit 166 to decode the designated song score timing signal S4. Here, the designated song score timing signal S4 has the encoded designated song score timing data 22, so the designated song score timing data 22 can be obtained by decoding it.

[0070] Step S108: Configure the second processing unit 166 to execute the song scoring program when the synchronization starting point is reached according to the synchronization starting point indicated by the starting point indication signal S3. For example, the second processing unit 166 can obtain the time point at which the first line of lyrics starts in the designated song scoring timing data 22 from the karaoke main system 12 as the synchronization starting point, and the singer usually starts singing at the time point at which the first line of lyrics starts. The song scoring program starts scoring the vocal data after the synchronization starting point, and includes the following steps:

[0071] Step S109: Control the sampling unit 164 to continuously sample the portion after the start point of the vocal data 24 to generate sampled audio data.

[0072] Step S110: Compare the sampled audio data with the designated song score time series data 22 to obtain a comparison result. In this step, the sampled audio data is data obtained by sampling the singer's voice. This data is compared with the original singer's audio data, pitch, beat, length, volume data and music score data corresponding to the time tags of each line of lyrics in the designated song score time series data 22 to obtain the above comparison result.

[0073] Step S111: According to the comparison result, score is scored and score result data S5 is generated accordingly. In this step, in order to generate score result data in real time, data is usually collected and analyzed in units of one line of lyrics. The second processing unit 166 can determine whether the currently obtained sampled audio data is sufficient based on the time tag of each line of lyrics, for example, whether the sampled audio data of a complete line of lyrics has been obtained, and then decide to output the comparison result.

[0074] Step S112: The scoring result data S5 is transmitted to the karaoke host system 12 via the second communication module 160. In some embodiments, in response to receiving the scoring result data S5, the karaoke host system 12 may display the scoring result data S5 on the user interface 120.

[0075] It should be noted that although the above process scores a complete line of lyrics, the present invention is not limited to this. In actual operation, the score result data S5 of each line of lyrics can be stored in advance, and a more accurate score will be presented until a certain number is accumulated (for example, when the song has been played 1 / 3 or 1 / 2), so that the quality of the song can be more realistically presented.

[0076] Through the above steps, it can be ensured that the karaoke main system will synchronize with the auxiliary scoring system in real time after starting to play the specified song, and the auxiliary scoring system can sample, compare and score the vocal data in real time, and transmit the scoring results to the karaoke main system for its use in real time. In addition, the karaoke main system itself does not need to support the recording function, as long as the karaoke main system and the auxiliary scoring system can communicate data and synchronize in time, and specific data can be transmitted through the encoding and decoding of the playing audio. Therefore, the applicable platform range of karaoke scoring technology and functions can be greatly improved, including smart TVs or TV set-top boxes, karaoke accompaniment machines, etc.

[0077] Please refer to Figure 3 , which is a schematic diagram of the architecture of a multimedia scoring system according to another embodiment of the present invention. As shown in the figure, the auxiliary scoring system 16 and the mixing module 14 are arranged in a mixing additional device 30, which is connected to the karaoke main system 12 via a USB communication protocol, and the mixing additional device 30 is further electrically connected to a sound receiving unit 18, which may be, for example, a microphone.

[0078] In this embodiment, the mixing attachment 30 can be connected to the karaoke main system 12 as a peripheral device, and the synchronization request signal S1 mentioned in the above embodiment can be transmitted multiple times through the first communication module to obtain the synchronization timing signal S2 multiple times. The synchronization request signal S1 can be, for example, a request signal for transmitting a Start Of Frame (SOF) type packet, and the SOF type packet includes a frame number (Frame Number) with a length of 11 bits. In the USB communication protocol, a SOF packet is sent once every predetermined time interval, so it can be used as a reference for the synchronization timing during the synchronization process between the karaoke main system 12 and the auxiliary scoring system 16.

[0079] Please refer to Figure 4 , which is a system block diagram of a multimedia scoring system according to another embodiment of the present invention. As shown in the figure, in this embodiment, the first communication module 122 and the second communication module 160 are configured to be connected to the first timing unit 162 through the network 34, and obtain the synchronization timing signal S2 generated by the first timing unit 162 through the network 34, and the synchronization timing signal S2 can be, for example, a network clock timing signal 26. Therefore, when the karaoke main system 12 and the auxiliary scoring system 16 need to be synchronized to perform song scoring, the karaoke main system 12 and the auxiliary scoring system 16 can be synchronized with the first timing unit 162 (for example, a third-party network clock), and the first processing unit 124 of the karaoke main system 12 calculates the synchronization starting point according to the synchronization timing provided by the third-party network clock, and then transmits the starting point indication signal S3 to the auxiliary scoring system 16.

[0080] On the other hand, in this embodiment, the database 10 is set in the network server 32, and the first processing unit 124 can control the first communication module 122 to obtain the designated song data 20 and the designated song score time series data 22 from the database 10 of the network server 32 through the network.

[0081] The karaoke main system 12 may include an audio playback module 126 (e.g., a speaker), and the first processing unit 124 may control the audio playback module 126 to play the specified song data 20. At the same time, the mixing module 14 also includes a third communication module 142 and a fourth communication module 144, and the third communication module 142 may be used to transmit the vocal data 24 to the karaoke main system, and the karaoke main system 12 may, in response to receiving the vocal data 24, mix the specified song data 20 with the vocal data 24 to generate mixed data, and control the audio playback module 126 to play the mixed data.

[0082] The first communication module 122 and the second communication module 160 can be connected to each other through the first network interface 36, and the first network interface 36 can be, for example, a wireless network interface, and the first communication module 122 and the second communication module 160 can be connected to the network 34 through the first network interface 36. In some embodiments, the first network interface 36 can be, for example, a router or a sharing device having an indoor network architecture, the first communication module 122 can be an Ethernet module, connected to the router through a wired network, and the second communication module 160 can be a wireless network module, connected to the router through a WIFI network interface.

[0083] In addition, the auxiliary scoring system 16 may include a fifth communication module 168, and connected to the fourth communication module 144 through the second network interface 38. The fourth communication module 144 and the fifth communication module 168 may be Bluetooth communication modules, and the second network interface 38 may be a Bluetooth transmission interface.

[0084] Under this architecture, different signal paths are provided for the synchronization request signal S1, the synchronization timing signal S2, the starting point indication signal S3, the designated song scoring timing signal S4, and the scoring result data S5. For example, unlike the previous embodiment, the synchronization request signal S1 and the synchronization timing signal S2 can be transmitted through the network 34, and the designated song scoring timing signal S4 can be transmitted between the karaoke main system 12 and the auxiliary scoring system 16 through the first network interface 36.

[0085] In a specific embodiment, the designated song data 20 and the designated song score timing data 22 can be further transmitted to the mixing module 14 through the third communication module 142. The mixing module 14 can adjust the frequency, dynamics, sound quality, positioning, reverberation and sound field of each individual original signal according to the designated song data 20 and the designated song score timing data 22 as required to optimize each track, and then superimpose and mix with the human voice data 24 to generate mixed data, and then send the mixed data back to the audio playback module 126 through the third communication module 142 for broadcasting, and at the same time transmit the unmixed human voice data 24 and the designated song score timing data 22 to the auxiliary scoring system 16 through the second network interface 38. In this way, the multimedia scoring system and method provided by the present invention can be applied to different network architectures and meet different needs.

[0086] about Figure 4 An example of a multimedia rating system can be found in Figure 5 , which is a schematic diagram of the architecture of a multimedia scoring system according to another embodiment of the present invention. As shown in the figure, the auxiliary scoring system 16 is arranged in the mobile device 33, or the mobile device 33 executes an application program including the functions of the auxiliary scoring system 16, and the display interface 330 of the mobile device 33 displays the scoring results analyzed by the auxiliary scoring system 16. On the other hand, the mixing module 14 can be arranged in the mixing attachment 30, which can be connected to the cloud (Over the top, OTT) set-top box 35 via USB, and the karaoke main system 12 can be arranged in the cloud set-top box 35, or the cloud set-top box 35 executes an application program including the functions of the karaoke main system 12, and the present invention is not limited thereto.

[0087] Furthermore, the fifth communication module 168 can be a Bluetooth wireless transmission module built into the mobile device 33, and the fourth communication module 144 can also be a Bluetooth wireless transmission module, so that the mixing attachment device 30 can perform wireless transmission with the mobile device 33 through the Bluetooth transmission interface. In addition, the first communication module 122 and the second communication module 160 can be wireless communication modules, which can be connected through a wireless network interface.

[0088] Please refer to further Fig. 6A and Figure 6B , which is a flowchart of a multimedia scoring method according to another embodiment of the present invention. Figure 4 , Figure 5 The multimedia scoring system provided by the present invention and the multimedia scoring method of the present invention may further include the following steps:

[0089] Step S200 : configure the first communication module 122 and the second communication module 160 to connect to the first timing unit 162 through the network 34 , and obtain the synchronization timing signal S2 generated by the first timing unit 162 through the network 34 . The synchronization timing signal S2 may be, for example, the network clock timing signal 26 .

[0090] Step S201 : synchronizing the karaoke main system 12 and the auxiliary scoring system 16 with the first timing unit 162 .

[0091] Step S202 : After configuring the first processing unit 124 to calculate the synchronization start point according to the synchronization timing provided by the first timing unit 162 , the start point indication signal S3 is transmitted to the auxiliary scoring system 16 .

[0092] On the other hand, the multimedia scoring method of the present invention may further include the following steps:

[0093] Step S203 : configuring the first processing unit 124 to transmit the designated song data 20 and the designated song rating time series data 22 to the mixing module 14 via the third communication module 142 .

[0094] Step S204: configure the mixing module 14 to mix the vocal data 24 according to the designated song data 20 to generate mixed data.

[0095] Step S205 : configure the third communication module 142 to transmit the mixed audio data back to the audio playback module 126 for broadcasting, and transmit the vocal data 24 and the designated song rating timing data 22 to the auxiliary rating system 16 through the second network interface 38 .

[0096] In addition, in the above embodiment, the mixing module 14 can be further connected to the sound receiving unit 18 via wired or wireless means. For example, the mixing module 14 can be connected to the sound receiving unit 18 (for example, a microphone) via an ultra high frequency (UHF), or the mixing attachment 30 can be attached to the sound receiving unit 18 as an adapter (dongle), so that the sound receiving unit 18 has a microphone capable of communicating with the karaoke main system 12 and the auxiliary scoring system 16, thereby enhancing the flexibility of the application.

[0097] Please refer to Figure 7, which is a system block diagram of a multimedia scoring system according to another embodiment of the present invention. As shown in the figure, the karaoke main system 12 also includes a second timing unit 128, which is configured to generate another timing synchronization signal. For example, the first communication module 122 and the second communication module 160 can be based on a first communication protocol, such as IEEE-1588 Precision Time Protocol (PTP), such as IEEE-1588v2 PTP, to allocate the karaoke main system 12 and the auxiliary scoring system 16 as a main synchronization device (such as PTP Master) and a slave synchronization device (PTP Slave), so as to determine whether to synchronize the karaoke main system 12 and the auxiliary scoring system 16 with the timing synchronization signal S2 generated by the first timing unit 126 or another timing synchronization signal generated by the second timing unit 128. For example, when the karaoke main system 12 is used as the main synchronization device and the auxiliary scoring system 16 is used as the slave synchronization device, the timing synchronization signal of the second timing unit 128 is used as the synchronization reference to calculate the synchronization starting point, and vice versa.

[0098] On the other hand, in the present embodiment, the karaoke main system 12 and the auxiliary scoring system 16 are in different network environments. For example, the karaoke main system 12 is connected to the network 34 through the first communication module 122, such as an Ethernet module, and the auxiliary scoring system 16 is connected to the network 34 through the second communication module 160, such as a 3G / 4G / 5G network communication module, and the mixing module 14 may further include a sixth communication module 146, such as a wireless network module, connected to the karaoke main system 12 through a first network interface (such as a WIFI network interface). At this time, the karaoke main system 12 and the mixing module 14 are in the internal network, and the auxiliary scoring system 16 is in the external network. The auxiliary scoring system 16 is further connected to the fourth communication module 144 of the mixing module 14 through the second network interface 38 by the fifth communication module 168. Among them, the fourth communication module 144 and the fifth communication module 168 can be Bluetooth communication modules, and the second network interface 38 can be a Bluetooth transmission interface.

[0099] In this embodiment, in order to solve the timing synchronization problem of the internal and external networks, the aforementioned first communication protocol, such as IEEE-1588 Precision Time Protocol (PTP), can still be used to achieve synchronization between the karaoke main system 12 and the auxiliary scoring system 16. To this end, the first communication module 122 and the second communication module 160 are also configured to allocate the karaoke main system 12 and the auxiliary scoring system 16 as a main synchronization device (such as PTP Master) and a slave synchronization device (PTP Slave) through the fourth communication module 144, the fifth communication module 168 and the sixth communication module 146.

[0100] In other words, after the main synchronization device (such as PTP Master) and the slave synchronization device (PTP Slave) are determined through the above signal path, the karaoke main system 12 can obtain the corresponding synchronization timing signal S2 to calculate the synchronization starting point and generate the starting point indication signal S3. It can be imagined that the starting point indication signal S3, the designated song scoring timing signal S4 and the scoring result data S5 can be directly transmitted through the network 34, or transmitted through the signal path of the fourth communication module 144, the fifth communication module 168 and the sixth communication module 146. The karaoke main system 12 and the auxiliary scoring system 16 can select a faster transmission path to transmit the starting point indication signal S3, the designated song scoring timing signal S4 and the scoring result data S5 according to the current network transmission status.

[0101] Advantageous Effects of Embodiments

[0102] One of the beneficial effects of the present invention is that the multimedia scoring system and method provided by the present invention does not require the karaoke main system to support the recording function. It only needs data communication and timing synchronization between the karaoke main system and the auxiliary scoring system, and can transmit specific data through the encoding and decoding of the played audio. Therefore, the applicable platform range of karaoke scoring technology and functions can be greatly increased, including smart TVs or TV set-top boxes, karaoke accompaniment machines, etc.

[0103] Furthermore, the multimedia scoring system and method provided by the present invention can ensure that the karaoke main system is synchronized with the auxiliary scoring system in real time after starting to play the specified song, and the auxiliary scoring system can sample, compare and score the vocal data in real time, and transmit the scoring results to the karaoke main system in real time for its use.

[0104] The contents disclosed above are only preferred feasible embodiments of the present invention, and are not intended to limit the protection scope of the claims of the present invention. Therefore, all equivalent technical changes made using the contents of the present invention description and drawings are included in the protection scope of the claims of the present invention.

Claims

1. A multimedia scoring system, characterized in that: The multimedia scoring system comprises: A database storing a plurality of song data and a plurality of corresponding song rating time series data; A first timing unit, used for generating a synchronization timing signal for indicating a synchronization timing; A karaoke main system is electrically connected to the database and includes: a first communication module configured to transmit a synchronization request signal and obtain the synchronization timing signal; and a first processing unit configured to obtain a designated song data and a designated song score timing data from the database according to a song designation command, and calculate a synchronization start point according to the synchronization timing and the designated song score timing data, so as to correspondingly generate a start point indication signal for indicating the synchronization start point; a mixing module configured to collect sound and generate human voice data; and An auxiliary scoring system is electrically connected to the karaoke main system and the mixing module and includes: a second communication module, configured to obtain the synchronization timing signal from the first communication module, and receive the starting point indication signal and the designated song score timing data; a sampling unit configured to sample the vocal data; and A second processing unit is configured to obtain the designated song scoring timing data and execute a song scoring program when the synchronization starting point is reached according to the synchronization starting point indicated by the starting point indication signal, wherein the song scoring program includes: Controlling the sampling unit to continuously sample the portion of the vocal data after the starting point to generate a sampled audio data; Comparing the sampled audio data with the designated song score time series data to obtain a comparison result; According to the comparison result, scoring is performed and scoring result data is generated accordingly; and The scoring result data is transmitted to the karaoke main system through the second communication module.

2. The multimedia rating system according to claim 1, characterized in that: The karaoke main system also includes a user interface, which is used by a user to input the song designation command and play the designated song data according to the song designation command.

3. The multimedia scoring system according to claim 1, characterized in that: The first communication module is configured to transmit the synchronization request signal multiple times to obtain the synchronization timing signal multiple times.

4. The multimedia scoring system according to claim 1, characterized in that: The second communication module is configured to transmit the synchronization timing signal to the first communication module.

5. The multimedia scoring system according to claim 1, characterized in that: The first communication module and the second communication module are configured to be connected to the first timing unit through a network to obtain the synchronous timing signal, and the synchronous timing signal is a network clock timing signal.

6. The multimedia scoring system according to claim 1, characterized in that: The karaoke main system also includes an audio playback module, and the first processing unit is configured to control the audio playback module to play the specified song data.

7. The multimedia scoring system according to claim 6, characterized in that: The mixing module also includes a third communication module, which is configured to transmit the vocal data to the karaoke main system. In response to receiving the vocal data, the first processing unit is also configured to mix the designated song data with the vocal data to generate a mixed data, and control the audio playback module to play the mixed data.

8. The multimedia rating system according to claim 1, characterized in that: The karaoke main system also includes a second timing unit, which is configured to generate another timing synchronization signal, and the first communication module and the second communication module are also configured to allocate the karaoke main system and the auxiliary scoring system as a main synchronization device and a slave synchronization device according to a first communication protocol to determine whether to synchronize between the karaoke main system and the auxiliary scoring system using the timing synchronization signal or the other timing synchronization signal.

9. The multimedia rating system according to claim 1, characterized in that: The database is set in a network server, and the first processing unit is configured to control the first communication module to obtain the designated song data and the designated song score timing data from the database through the network.

10. The multimedia scoring system according to claim 1, characterized in that: The auxiliary scoring system and the sound mixing module are arranged in a sound mixing attachment device, and the sound mixing attachment device is further electrically connected to a sound receiving unit.

11. The multimedia scoring system according to claim 1, characterized in that: The auxiliary scoring system includes a fifth communication module, the mixing module includes a third communication module and a fourth communication module, the third communication module is connected to the karaoke main system, and the fourth communication module is configured to be connected to the fifth communication module through a second network interface, The second communication module is connected to the first communication module via a first network interface.

12. The multimedia rating system according to claim 1, characterized in that: The mixing module includes a third communication module, a fourth communication module and a sixth communication module, and the auxiliary scoring system also includes a fifth communication module. The third communication module is configured to be connected to the karaoke main system through a first network interface, the second communication module is connected to the first communication module through the network, and the sixth communication module is configured to be connected to the fifth communication module through a second network interface.

13. The multimedia rating system according to claim 12, characterized in that: The karaoke main system also includes a second timing unit, which is configured to generate another timing synchronization signal, and the first communication module and the second communication module are also configured to allocate the karaoke main system and the auxiliary scoring system as a main synchronization device and a slave synchronization device according to a first communication protocol through the fourth communication module, the fifth communication module and the sixth communication module to determine whether to synchronize between the karaoke main system and the auxiliary scoring system using the timing synchronization signal or the other timing synchronization signal.

14. A multimedia scoring method using the multimedia scoring system according to any one of claims 1 to 13, characterized in that: The multimedia scoring method comprises the following steps: A first processing unit of a karaoke main system is configured to obtain a designated song data and a designated song score time series data from the database according to a song designation command; A first timing unit is configured to generate a synchronization timing signal for indicating a synchronization timing; A first communication module of the karaoke main system is configured to be connected with a second communication module of the auxiliary scoring system through a first communication protocol, and the first communication module and the second communication module are configured to obtain the synchronization timing signal; The first processing unit is configured to calculate a synchronization starting point according to the synchronization timing and the designated song score timing data, so as to correspondingly generate a starting point indication signal for indicating the synchronization starting point; Transmitting the starting point indication signal and the designated song scoring timing data to the auxiliary scoring system through the first communication module; A mixing module is configured to collect sound and generate human voice data; as well as A second processing unit configured in the auxiliary scoring system obtains the designated song scoring timing data, and executes a song scoring program when the synchronization starting point is reached according to the synchronization starting point indicated by the starting point indication signal, wherein the song scoring program includes the following steps: Controlling the sampling unit to continuously sample the portion of the vocal data after the starting point to generate a sampled audio data; Comparing the sampled audio data with the designated song score time series data to obtain a comparison result; According to the comparison result, scoring is performed and scoring result data is generated accordingly; and The scoring result data is transmitted to the karaoke main system through the second communication module.

15. The multimedia scoring method according to claim 14, characterized in that: The multimedia scoring method further includes configuring a user interface for a user to input the song specifying command, and configuring the user interface to play the specified song data according to the song specifying command.

16. The multimedia scoring method according to claim 14, characterized in that: The multimedia scoring method further includes configuring the first communication module to transmit the synchronization request signal multiple times to obtain the synchronization timing signal multiple times.

17. The multimedia scoring method according to claim 14, characterized in that: The multimedia scoring method further includes configuring the second communication module to transmit the synchronization timing signal to the first communication module.

18. The multimedia scoring method according to claim 14, characterized in that: The multimedia scoring method further includes configuring the first communication module and the second communication module to connect to the first timing unit via a network, and obtaining the synchronous timing signal via the network, wherein the synchronous timing signal is a network clock timing signal.

19. The multimedia scoring method according to claim 14, characterized in that: The multimedia scoring method also includes configuring the first processing unit to control an audio playback module of the karaoke main system to play the designated song data.

20. The multimedia scoring method according to claim 19, characterized in that: The multimedia scoring method further comprises: A third communication module configured with the mixing module transmits the vocal data to the karaoke main system; and In response to receiving the vocal data, the first processing unit is configured to mix the designated song data with the vocal data to generate mixed data, and the audio playback module is controlled to play the mixed data.

21. The multimedia scoring method according to claim 14, characterized in that: The multimedia scoring method further comprises: A second timing unit of the karaoke main system is configured to generate another timing synchronization signal; the first communication module and the second communication module are configured to allocate the karaoke main system and the auxiliary scoring system as a main synchronization device and a slave synchronization device according to the first communication protocol to determine whether to synchronize between the karaoke main system and the auxiliary scoring system using the timing synchronization signal or the other timing synchronization signal.

22. The multimedia scoring method according to claim 14, characterized in that: The database is set in a network server, and the multimedia scoring method also includes configuring the first processing unit to control the first communication module to obtain the designated song data and the designated song scoring time series data from the database through the network.

23. The multimedia scoring method according to claim 14, characterized in that: The auxiliary scoring system and the sound mixing module are arranged in a sound mixing attachment device, and the sound mixing attachment device is further electrically connected to a sound receiving unit.

24. The multimedia scoring method according to claim 14, characterized in that: The auxiliary scoring system includes a fifth communication module, the mixing module includes a third communication module and a fourth communication module, and the multimedia scoring method further includes: Configuring the second communication module to connect to the first communication module through a first network interface; Configuring the third communication module to be connected to the karaoke main system; and The fourth communication module is configured to be connected to the fifth communication module via a second network interface.

25. The multimedia scoring method according to claim 14, characterized in that: The mixing module includes a third communication module, a fourth communication module and a sixth communication module, the auxiliary scoring system also includes a fifth communication module, and the multimedia scoring method also includes: Configure the third communication module to be connected to the karaoke main system via a first network interface; configuring the second communication module to be connected to the first communication module via a network; The sixth communication module is configured to be connected to the fifth communication module via a second network interface.

26. The multimedia scoring method according to claim 25, characterized in that: The multimedia scoring method further comprises: A second timing unit of the karaoke main system is configured to generate another timing synchronization signal; and The first communication module and the second communication module are configured to allocate the karaoke main system and the auxiliary scoring system as a main synchronization device and a slave synchronization device through the fourth communication module, the fifth communication module and the sixth communication module according to the first communication protocol to determine whether to synchronize between the karaoke main system and the auxiliary scoring system using the timing synchronization signal or the other timing synchronization signal.

Images