Music karaoke method and device based on IPTV, electronic equipment and storage medium

Abstract

The invention discloses an IPTV-based music karaoke method and device and a medium, and the method comprises the steps: determining a target music file and a corresponding target music identifier in response to a music selection instruction of a user, playing the target music file, and transmitting the target music identifier to a server; determining a corresponding target scoring reference file through the server according to the target music identifier; obtaining a target audio stream of a user through the network sound box, and sending the target audio stream to a server; and scoring the target audio stream through the server according to the target scoring reference file, returning a scoring result, and displaying the scoring result through an EPG page. According to the invention, the real-time scoring of the music karaoke is realized through the IPTV set top box, the user experience is improved, and the method can be widely applied to the technical field of audio processing.

Description

Technical Field

[0001] The present invention relates to the technical field of audio processing, and in particular to an IPTV-based music karaoke method, device, electronic equipment and storage medium. Background Art

[0002] There are two main music karaoke solutions for existing TV boxes: one is a karaoke solution based on Android APK; the other is a karaoke solution based on EPG pages.

[0003] The principle of Karaoke based on Android APK is to transmit the user's voice to the TV box through the microphone. The APK in the box reads the audio data from the microphone through the USB interface and calculates the score in real time through the audio processing algorithm. However, this APK solution has certain limitations:

[0004] 1) IPTV TV boxes in some regions do not support APK installation, so the solution is not available;

[0005] 2) The IPTV TV boxes in some regions do not have a USB port for audio input, so the solution is not available;

[0006] 3) The hardware configuration of some users' IPTV TV boxes is relatively old, and the audio processing algorithm is stuck when running, so the solution is not effective;

[0007] 4) If the user's TV box is based on EPG, and EPG is generally the system desktop of the box, access requires the EPG development team to open the download, installation, and call permissions of APK. However, the EPG application development teams in different provinces are different. In this way, promotion requires special adaptation for each province, which requires a large development workload;

[0008] 5) Due to information security issues, many provinces do not recommend using APK to access IPTV.

[0009] The principle of the EPG-based karaoke solution is to use the multi-channel input function of the speaker; the original accompaniment audio signal output port of the box is connected to the input port 1 of the speaker, and the microphone is connected to the input port 2 of the speaker. The speaker plays the two signals at the same time, and karaoke can be performed. However, in this solution, the microphone signal is directly played through the speaker without being calculated by the audio algorithm, so this solution cannot be used for karaoke scoring.

[0010] In addition, there are still many songs that do not support karaoke scoring in the software with karaoke scoring function on the market. This is because the copyright providers may not be able to develop karaoke scoring algorithms, and karaoke scoring algorithm providers do not have music copyrights. Generally speaking, the music copyrights of a music company are provided by dozens of music copyright providers. If most songs are to be converted into karaoke scoring files, it is necessary to cooperate with many companies, which is a huge workload.

[0011] Therefore, it is urgent to design a music karaoke scoring solution that is adapted to the EPG page of the IPTV box and can independently convert any song audio into scoreable audio.

[0012] Terminology explanation:

[0013] IPTV: Internet Protocol TV, it is based on broadband high-speed IP network, with network video resources as the main body, using televisions, personal computers and handheld devices as display terminals, accessing broadband networks through set-top boxes or computers to realize digital TV, time-shifted TV, interactive TV and other services. The emergence of Internet TV has brought people a new way of watching TV. It has changed the previous passive TV viewing mode and realized a convenient way of watching TV on demand and stopping at any time based on the Internet.

[0014] Karaoke: KTV originated from Japan. KTV is the abbreviation of Karaoke TV. Karaok is a Japanese-English hybrid name. Kara means "empty" in Japanese, and OK is the abbreviation of "orchestra" in English. It evolved into "karaoke" in China. Now it is further shortened to Karaoke.

[0015] EPG: IPTV EPG is actually a portal system of IPTV. The interface of EPG system is similar to that of Web page. The EPG interface generally provides various menus, buttons, links and other components for users to click directly when selecting programs. The EPG interface can also contain various dynamic or static multimedia contents for users to browse.

[0016] APK: APK (full name: Android application package) is an application package file format used by the Android operating system for distributing and installing mobile applications and middleware.

[0017] Music copyright: Music copyright is the legal right enjoyed by the creator of a musical work. Stars, artists, and individuals are not allowed to cover songs, or spoof or adapt original songs in concerts, TV shows, online live broadcasts, public places, etc. without the consent of the original creator, otherwise they will be sued by the original creator company and asked to pay huge infringement fees.

[0018] Sampling rate: Sampling frequency, also known as sampling speed or sampling rate, defines the number of samples extracted from a continuous signal to form a discrete signal per unit time. It is expressed in Hertz (Hz). The reciprocal of the sampling frequency is the sampling period or sampling time, which is the time interval between samples. In layman's terms, sampling frequency refers to how many signal samples a computer can collect per unit time.

[0019] Bit depth: Bit depth refers to the number of bits used by each sample point in digital audio. Bit depth determines the dynamic range of digital audio, which is the distance between the maximum and minimum amplitudes of the sound signal. The higher the bit depth, the greater the dynamic range of digital audio, and the more accurately it can display the details and differences of the sound signal. For example, the bit depth of CD quality sound is 16 bits, which means that each sample point occupies 16 bits. Like the sampling rate, a higher bit depth will also take up more storage space.

[0020] Channel: Sound Channel refers to the independent audio signals collected or played back at different spatial locations when recording or playing sounds, so the number of channels is the number of sound sources when recording sounds or the corresponding number of speakers when playing them back.

[0021] Fourier transform: Fourier transform is a method of analyzing signals. It can analyze the components of a signal or synthesize a signal with these components. Fourier transform plays an extremely critical role in theoretical research and engineering practice. It provides a computational method to convert complex time domain signals into frequency domain signals that are easy to analyze.

[0022] Fast Fourier transform (FFT) algorithm: FFT is an efficient algorithm of DFT, called fast Fourier transform.

[0023] Pitch serial number: refers to various sounds of different pitches, that is, the height of the sound, one of the basic characteristics of the sound.

[0024] Adobe Audition: Adobe Audition (Au for short, formerly known as Cool Edit Pro) is a professional audio editing and mixing environment developed by Adobe. Audition is designed for audio and video professionals working in studios, broadcasting equipment, and post-production equipment, and provides advanced audio mixing, editing, control, and effect processing capabilities.

[0025] FL Studio: FL Studio is the music arrangement software "Fruity Loops Studio", or FL for short, which is also known as the fruit arrangement software by the Chinese people. Users in the circle are used to calling it "Fruity". It is an all-round music production environment or digital audio workstation (DAW). FL Studio can arrange, edit, record and mix music, turning your computer into a full-featured recording studio to help you make your own original music.

[0026] WAV: WAV is one of the most common sound file formats. It is a standard digital audio file developed by Microsoft specifically for Windows. The file can record various mono or stereo sound information and ensure that the sound is not distorted. It complies with the Resource Interchange File Format (RIFF) specification and is used to save audio information resources on the Windows platform. It is widely supported by the Windows platform and its applications. The Wave format supports MSADPCM, CCITT A-law, CCITT μ-law and other compression algorithms, supports multiple audio bit numbers, sampling frequencies and channels, and is the most popular sound file format on PCs; but its file size is large and is mostly used to store short sound clips.

[0027] FLAC: Free Lossless Audio Codec (FLAC) is a free audio compression codec that features lossless compression. Unlike other lossy compression codes such as MP3 and AAC, it does not destroy any original audio information, so it can restore the sound quality of music CDs[1]. Since 2012, it has been supported by many software and hardware audio products (such as CDs, etc.).

[0028] MIDI: A MIDI file is not a recorded sound, but a set of instructions that records the sound information and then tells the sound card how to reproduce the music. Such a MIDI file only takes about 5 to 10KB to store 1 minute of music. MIDI files are mainly used for original instrumental works, amateur performances of popular songs, game soundtracks, and electronic greeting cards.

[0029] TXT: TXT is a text format that comes with Microsoft's operating system. It is the most common file format. It has been widely used as early as the DOS era. It mainly stores text information, that is, text information. Most current operating systems use programs such as Notepad to save it, and most software can view it, such as Notepad, browser, etc. Summary of the invention

[0030] The purpose of the present invention is to solve one of the technical problems existing in the prior art to at least a certain extent.

[0031] To this end, an object of an embodiment of the present invention is to provide a music karaoke method based on IPTV, which realizes real-time scoring of music karaoke through an IPTV set-top box, thereby improving the user experience.

[0032] Another object of an embodiment of the present invention is to provide a music karaoke device based on IPTV.

[0033] In order to achieve the above technical objectives, the technical solutions adopted by the embodiments of the present invention include:

[0034] On the one hand, an embodiment of the present invention provides a music karaoke method based on IPTV, comprising the following steps:

[0035] In response to a music selection instruction from a user, determining a target music file and a corresponding target music identifier, playing the target music file, and sending the target music identifier to a server;

[0036] Determining, by the server, a corresponding target scoring reference file according to the target music identifier;

[0037] Acquire a target audio stream of the user through a network speaker, and send the target audio stream to the server;

[0038] The server scores the target audio stream according to the target scoring reference file, returns the scoring result and displays it through an EPG page.

[0039] Further, in one embodiment of the present invention, the music karaoke method further includes a step of pre-binding the network speaker, which specifically includes:

[0040] In response to the user's device binding instruction, display the device binding QR code through the EPG page;

[0041] Scanning the device binding QR code through the user terminal causes the user terminal to jump to the device binding page;

[0042] Scanning the device QR code of the network speaker through the user terminal, and obtaining the device code of the network speaker according to the device QR code;

[0043] In response to the user's binding confirmation instruction, the network speaker is bound to the user's IPTV account according to the device code.

[0044] Further, in one embodiment of the present invention, in response to the user's music selection instruction, determining the target music file and the corresponding target music identifier, playing the target music file, and sending the target music identifier to the server specifically includes:

[0045] Display the list of alternative music through the EPG page;

[0046] When the music selection instruction is detected, the target music name selected by the user is determined according to the music selection instruction, and the corresponding target music file and the target music identifier are obtained according to the target music name;

[0047] Display the playback preparation screen through the EPG page, and play the target music file after the preset countdown ends;

[0048] The target music identifier is sent to the server via the IPTV set-top box.

[0049] Further, in one embodiment of the present invention, the step of determining the corresponding target scoring reference file according to the target music identifier specifically includes:

[0050] Get the preset scoring file library;

[0051] Perform a matching search in the scoring file library according to the target music identifier to obtain the target scoring reference file;

[0052] The target scoring reference file includes the start time, duration and pitch sequence number of each target pronunciation.

[0053] Furthermore, in one embodiment of the present invention, the music karaoke method further includes the step of pre-building the scoring file library, which specifically includes:

[0054] Acquire multiple candidate music files, separate the vocals from the background sounds in the candidate music files, and obtain vocal files;

[0055] Converting the vocal file into a pronunciation file in MIDI format, and removing noise from the pronunciation file to obtain a scoring file;

[0056] Generate a scoring array corresponding to each pronunciation according to the start time, duration and pitch sequence number of each pronunciation in the scoring file, and generate a scoring reference file according to the scoring array;

[0057] The music identifier of the candidate music file corresponding to each of the scoring reference files is determined, a key-value pair is generated according to the scoring reference file and the music identifier, and then the scoring file library is constructed according to the key-value pair.

[0058] Further, in one embodiment of the present invention, scoring the target audio stream according to the target scoring reference file, returning the scoring result and displaying it through an EPG page specifically includes:

[0059] Determine the start time, duration and pitch sequence number of each target pronunciation according to the target scoring reference file;

[0060] Determine a plurality of audio stream data packets of the target audio stream and corresponding timestamps;

[0061] Determine a plurality of target audio stream data packets corresponding to each of the target pronunciations according to the start time and duration of the target pronunciation and the timestamp of the audio stream data packet, and then determine a target audio stream data segment according to the target audio stream data packet;

[0062] Scoring the target audio stream data segment according to the pitch sequence number of the target pronunciation to obtain a user pronunciation score of the target audio stream data segment;

[0063] The real-time scoring result is determined according to the average of the user pronunciation scores of the target audio stream data segments obtained at the current moment, and the real-time scoring result is returned to the IPTV set-top box, and then displayed through the EPG page.

[0064] Further, in one embodiment of the present invention, scoring the target audio stream data segment according to the pitch sequence number of the target pronunciation to obtain the user pronunciation score of the target audio stream data segment specifically includes:

[0065] Performing a fast Fourier transform on the target audio stream data packet to obtain a target frequency domain signal, and determining a target imaginary number array according to the target frequency domain signal;

[0066] Calculate the modulus of each imaginary number in the target imaginary number array, and determine the target sequence number of the imaginary number with the largest modulus in the target imaginary number array, and then calculate the first baud rate of the target audio stream data packet according to the target sequence number;

[0067] Calculate a second baud rate of the target pronunciation according to the pitch sequence number of the target pronunciation;

[0068] calculating an absolute value of a difference between the first baud rate and the second baud rate, determining a baud rate deviation of the target audio stream data packet according to a ratio of the absolute value of the difference to the second baud rate, and further determining whether the target audio stream data packet is out of tune according to the baud rate deviation;

[0069] Determine a first number of the target audio stream data packets that are not out of tune in the target audio stream data segment, and a second number of the target audio stream data packets that are out of tune;

[0070] The user pronunciation score is determined according to the first number and the second number.

[0071] On the other hand, an embodiment of the present invention provides a music karaoke device based on IPTV, comprising:

[0072] A music file playing module, for responding to a user's music selection instruction, determining a target music file and a corresponding target music identifier, playing the target music file, and sending the target music identifier to a server;

[0073] A reference file determination module, used to determine a corresponding target scoring reference file according to the target music identifier through the server;

[0074] An audio stream acquisition module is used to acquire a user's target audio stream through a network speaker and send the target audio stream to the server;

[0075] The audio stream scoring module is used to score the target audio stream according to the target scoring reference file through the server, return the scoring result and display it through the EPG page.

[0076] On the other hand, an embodiment of the present invention provides an electronic device, which includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the IPTV-based music karaoke method as described above is realized.

[0077] On the other hand, an embodiment of the present invention further provides a storage medium, which is a computer-readable storage medium used for computer-readable storage, and the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the IPTV-based music karaoke method as described above.

[0078] The advantages and beneficial effects of the present invention will be partly given in the following description, partly become apparent from the following description, or be understood through the practice of the present invention:

[0079] The embodiment of the present invention responds to the user's music selection instruction, determines the target music file and the corresponding target music identifier, plays the target music file, and sends the target music identifier to the server, determines the corresponding target scoring reference file by the server according to the target music identifier, obtains the user's target audio stream through the network speaker, and sends the target audio stream to the server, scores the target audio stream according to the target scoring reference file by the server, returns the scoring result and displays it through the EPG page. The embodiment of the present invention binds the IPTV account to the network speaker, and the network speaker pushes the target audio stream to the server while playing the user's target audio stream, scores the target audio stream in real time through the scoring reference file stored in the server, and transmits it to the EPG page for display, thereby realizing real-time scoring of music karaoke through the IPTV set-top box, overcoming the limitation problem that the karaoke solution based on APK in the prior art cannot be widely promoted on the IPTV set-top box, and also overcomes the problem that the karaoke solution based on the EPG page in the prior art cannot be scored, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] In order to more clearly illustrate the technical solution in the embodiments of the present invention, the following introduction is made to the drawings required for use in the embodiments of the present invention. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solution of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0081] Figure 1 A step flow chart of a music karaoke method based on IPTV provided by an embodiment of the present invention;

[0082] Figure 2 A schematic diagram of an application scenario of the IPTV-based music karaoke method provided in an embodiment of the present invention;

[0083] Figure 3 A flowchart of a step of pre-binding a network speaker provided in an embodiment of the present invention;

[0084] Figure 4 A user operation flow chart of pre-binding a network speaker provided in an embodiment of the present invention;

[0085] Figure 5 A schematic diagram of data interaction of a music karaoke method based on IPTV provided by an embodiment of the present invention;

[0086] Figure 6 A step flow chart of step S101 provided in an embodiment of the present invention;

[0087] Figure 7A step flow chart of step S102 provided in an embodiment of the present invention;

[0088] Figure 8 A flowchart of the steps of pre-building a scoring file library provided in an embodiment of the present invention;

[0089] Fig. 9 A step flow chart of step S104 provided in an embodiment of the present invention;

[0090] Fig.10 A step flow chart of step S1044 provided in an embodiment of the present invention;

[0091] Fig.11 A schematic diagram of the structure of an IPTV-based music karaoke device provided in an embodiment of the present invention;

[0092] Fig.12 A schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention;

[0093] Fig.13 A schematic diagram of the structure of a storage medium provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0094] Embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limitations on the present application. It should be noted that, although the functional module division is performed in the system schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described may be performed in a different order from the module division in the system schematic diagram or the flow chart. For the step numbers in the following embodiments, they are only set for the convenience of explanation, and the order between the steps is not limited in any way. The execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0095] In the description of the present invention, the meaning of "a plurality" is two or more. If there is a description of the first or the second, it is only for the purpose of distinguishing the technical features, and it cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features. In addition, unless otherwise defined, all technical and scientific terms used in this document have the same meaning as those commonly understood by technicians in the technical field of this application. The terms used in this document are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0096] The IPTV-based music karaoke method provided in the embodiment of the present application can be applied to the terminal, can also be applied to the server side, and can also be software running in the terminal or the server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a set-top box, etc.; the server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the IPTV-based music karaoke method, etc., but is not limited to the above forms.

[0097] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0098] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with the relevant laws, regulations, and standards of the relevant countries and regions. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0099] like Figure 1 FIG. 1 is a flowchart of a method for singing music karaoke based on IPTV provided by an embodiment of the present invention, referring to FIG. Figure 1 The embodiment of the present invention provides a music karaoke method based on IPTV, which specifically includes the following steps:

[0100] S101, in response to a music selection instruction of a user, determining a target music file and a corresponding target music identifier, playing the target music file, and sending the target music identifier to a server;

[0101] S102, determining a corresponding target scoring reference file according to the target music identifier by the server;

[0102] S103, obtaining the user's target audio stream through the network speaker, and sending the target audio stream to the server;

[0103] S104: The server scores the target audio stream according to the target scoring reference file, returns the scoring result and displays it on the EPG page.

[0104] Specifically, Figure 2 FIG. 1 is a schematic diagram of an application scenario of the IPTV-based music karaoke method provided in an embodiment of the present invention, referring to FIG. Figure 2 The embodiment of the present invention binds the IPTV account to the network speaker. While playing the microphone signal, the speaker pushes the audio stream in the custom format to the karaoke server in the background of the IPTV box EPG page scoring system, scores the audio through a custom singing audio scoring algorithm, and transmits it to the EPG page, thereby realizing the music karaoke scoring function.

[0105] like Figure 3 FIG. 1 is a flowchart of a step of pre-binding a network speaker provided by an embodiment of the present invention, referring to FIG. Figure 3 As an optional implementation, the music karaoke method further includes a step of pre-binding a network speaker, which specifically includes:

[0106] S201, in response to a device binding instruction from a user, displaying a device binding QR code through an EPG page;

[0107] S202, the user terminal scans the device binding QR code, so that the user terminal jumps to the device binding page;

[0108] S203, scanning the device QR code of the network speaker through the user terminal, and obtaining the device code of the network speaker according to the device QR code;

[0109] S204. In response to the user's binding confirmation instruction, the network speaker is bound to the user's IPTV account according to the device code.

[0110] Specifically, Figure 4 The figure shows a user operation flow chart of pre-binding a network speaker provided by an embodiment of the present invention, and the specific operations are as follows:

[0111] 1) Users need to rent or purchase customized network speakers. The customized content of the speakers is only the software (it can start or stop sending audio streams in a specified format to the server after receiving specific instructions). The hardware part is no different from ordinary network speakers.

[0112] 2) The user needs to bind the IPTV account to the network speaker first (this step can also be assisted by the IPTV installation and maintenance engineer);

[0113] 3) The user opens the music channel karaoke section on the IPTV box EPG page;

[0114] 4) The user clicks the "Enable Scoring Function" button in the Karaoke area, and the device binding QR code pops up on the Karaoke area interface, prompting the user to scan the code with their mobile phone;

[0115] 5) The user scans the QR code with a mobile phone, and the phone jumps to the "Device Binding Page";

[0116] 6) After the user scans the device QR code on the speaker with a mobile phone according to the page prompt, the "Device Binding Page" automatically fills in the device code according to the QR code information;

[0117] 7) The user clicks the "Bind" button on the "Device Binding Page". If the binding is successful, the "Device Binding Page" pops up a message "Binding Successful";

[0118] 8) At this time, the KTV area page pops up a prompt "Device binding successful, please turn on the speaker and connect to the network";

[0119] 9) At this time, the speaker has not been turned on or connected to the network. The user needs to plug in the power supply and turn on the speaker, and connect the network speaker to the network. The network speaker can be connected to the network by directly plugging in the network cable and connecting to WIFI. The subsequent steps 10) to 13) are described using connecting to WIFI as an example.

[0120] 10) The user finds the instruction manual of the speaker and presses the switch on the speaker according to the instruction manual to switch the speaker to hotspot mode;

[0121] 11) The user connects the speaker's hotspot with a mobile phone, and according to the instructions in the manual, uses a browser to open the speaker configuration link (such as 192.168.1.1, the specific network address will be indicated in the manual), and jumps to the speaker's WIFI configuration interface (similar to the configuration page of a router);

[0122] 12) There is a WIFI list in the WIFI configuration interface of the network speaker. The user selects the name of his home WIFI in the list and enters the password in the WIFI password input box below. Click OK and the speaker is successfully connected to the WIFI.

[0123] 13) The network connection is successful and the speaker automatically connects to the karaoke server.

[0124] It should be noted that, under normal circumstances, the user only needs to perform the above-mentioned operations of binding the IPTV to the speaker and connecting the speaker to WIFI once. Subsequently, the user can directly sing karaoke and score by simply turning on the TV box and the speaker. After the above-mentioned IPTV account is successfully bound to the network speaker and the network speaker is connected to WIFI, the user can start singing karaoke.

[0125] like Figure 5 FIG. 1 is a schematic diagram of data interaction of a music karaoke method based on IPTV provided by an embodiment of the present invention, which shows a complete process of a music karaoke method based on IPTV. Figure 5 The implementation process of the music karaoke method according to the embodiment of the present invention is described in detail with other drawings.

[0126] like Figure 6 FIG. 1 is a flowchart of step S101 provided in an embodiment of the present invention, referring to FIG. Figure 6 As an optional implementation, in response to a user's music selection instruction, determining a target music file and a corresponding target music identifier, playing the target music file, and sending the target music identifier to a server, specifically includes:

[0127] S1011, displaying a list of alternative music via an EPG page;

[0128] S1012, when a music selection instruction is detected, determining the target music name selected by the user according to the music selection instruction, and acquiring the corresponding target music file and target music identifier according to the target music name;

[0129] S1013, displaying a playback preparation screen through the EPG page, and playing the target music file after the preset countdown ends;

[0130] S1014: Send the target music identifier to the server via the IPTV set-top box.

[0131] Specifically, the user selects a piece of music with a karaoke scoring icon in the karaoke area of ​​the EPG page and clicks it, and the karaoke area jumps to the karaoke preparation page; on the karaoke preparation page, the interface will display a countdown of "3, 2, 1" within 3 seconds; during the countdown, the EPG page sends a karaoke instruction to the server, informing the server that the user is about to start karaoke, and informing the server of the unique identifier of the song selected by the user and the karaoke start time T1 (Beijing time, accurate to milliseconds).

[0132] like Figure 7 FIG. 1 is a flow chart of step S102 provided in an embodiment of the present invention, referring to FIG. Figure 7As an optional implementation, the corresponding target scoring reference file is determined according to the target music identifier, which specifically includes:

[0133] S1021, obtaining a preset scoring file library;

[0134] S1022, performing a matching search in a scoring file library according to the target music identifier to obtain a target scoring reference file;

[0135] The target scoring reference file includes the start time, duration and pitch sequence number of each target pronunciation.

[0136] Specifically, the server finds the corresponding karaoke scoring reference file based on the unique identifier of the song. The scoring reference file contains the start time t1 of each word (relative time, accurate to milliseconds, for example: the start time of the song is 0, and it is 10000 after 10 seconds of playback), the duration d1 of a word (accurate to milliseconds, the duration of a word is generally between 200 and 500 milliseconds), and the pitch serial number.

[0137] After determining the scoring reference file, the server sends a command to notify the speaker device to start sending audio data; after receiving the command, the speaker starts to send the microphone's audio data in segments to the server background in real time; each segment of audio data contains a data header frame and a data tail frame, audio sampling rate, sampling bit depth, number of channels, data length, and acquisition time T2 (Beijing time, accurate to milliseconds).

[0138] like Figure 8 FIG. 1 is a flowchart of a step of pre-building a scoring file library provided by an embodiment of the present invention, referring to Figure 8 As an optional implementation, the music karaoke method further includes the step of pre-building a scoring file library, which specifically includes:

[0139] S301, obtaining a plurality of candidate music files, separating the vocals from the background sounds in the candidate music files, and obtaining vocal files;

[0140] S302, converting the vocal file into a pronunciation file in MIDI format, and removing noise from the pronunciation file to obtain a scoring file;

[0141] S303, generating a scoring array corresponding to each pronunciation according to the start time, duration and pitch sequence number of each pronunciation in the scoring file, and generating a scoring reference file according to the scoring array;

[0142] S304: Determine the music identifier of the candidate music file corresponding to each scoring reference file, generate a key-value pair according to the scoring reference file and the music identifier, and then construct a scoring file library according to the key-value pair.

[0143] Specifically, the embodiment of the present invention creates a scoring reference file based on the candidate music file, and uses the music identifier of the candidate music file as the key value and the corresponding scoring reference file as the value value to form a key-value pair, thereby constructing a scoring file library. The process of creating the scoring reference file is as follows:

[0144] 1) Use Adobe Audition to separate the audio of the karaoke video and export it as an "original sound.wav" file;

[0145] 2) Use AI capabilities (AI models provided by existing technologies) to separate human voice from background sound, retain only human voice, and export it as a "human voice.wav" file;

[0146] 3) Use FL Studio to convert "voice.wav" into a MIDI format file with pitch signals. A MIDI format file is a file that uses binary to save the start time, end time, international pitch number, loudness and other information of each tone;

[0147] 4) Use FL Studio to remove the noises such as breath and sibilance from the MIDI file (if not removed, the sound of breathing and sibilance will also become scoring points, which is obviously unreasonable);

[0148] 5) Export the MIDI file and name it "score.mid";

[0149] 6) Write a program to parse "score.mid" according to the format description of the MIDI file, and convert the pressing time t1 and lifting time t2 of each pronunciation pitch number h into a txt file in the format of "t1 d1 h", with one line for one pronunciation, where d1 = t2-t1;

[0150] 7) Save the txt file as "score.txt". The karaoke scoring reference file is now ready.

[0151] like Fig. 9 FIG. 1 is a flow chart of step S104 provided in an embodiment of the present invention, referring to FIG. Fig. 9 As an optional implementation, the target audio stream is scored according to the target scoring reference file, and the scoring result is returned and displayed through the EPG page, which specifically includes:

[0152] S1041, determining the start time, duration and pitch sequence number of each target pronunciation according to the target scoring reference file;

[0153] S1042, determining multiple audio stream data packets of the target audio stream and corresponding timestamps;

[0154] S1043, determining a plurality of target audio stream data packets corresponding to each target pronunciation according to the start time, duration and timestamp of the audio stream data packet of the target pronunciation, and then determining a target audio stream data segment according to the target audio stream data packet;

[0155] S1044, scoring the target audio stream data segment according to the pitch sequence number of the target pronunciation to obtain a user pronunciation score of the target audio stream data segment;

[0156] S1045, determine the real-time scoring result according to the average of the user pronunciation scores of each target audio stream data segment obtained at the current moment, and return the real-time scoring result to the IPTV set-top box, and then display the real-time scoring result through the EPG page.

[0157] Specifically, the process of scoring the target audio stream on the server side is as follows:

[0158] 1) After receiving the audio stream data packet, the server calculates T2-T1. If the pronunciation of a word meets t1<(T2-T1)<(t1+d1), it means that this audio stream is the user's pronunciation of the word. Then the scoring algorithm is used to calculate whether the pitch is accurate and record it. Generally, there will be 10-25 packets that need to be scored during the pronunciation time of a word in a song. The scoring algorithm will combine all the scoring results of the entire word to obtain a final score for the word.

[0159] 2) After the server obtains the final score of each word's pronunciation, it calculates the singing score at the current time and sends the singing score to the karaoke content playback page every 5 seconds;

[0160] 3) The karaoke content playback page gets the singing score data and displays it on the interface, and displays "SSS", "SS", "S", "A", "B", "C" according to the score;

[0161] 4) When the song ends, the EPG page sends a karaoke ending instruction to the server;

[0162] 5) The server receives the karaoke end instruction and sends an audio stream stop instruction to the network speaker;

[0163] 6) After receiving the audio stream stop instruction, the network speaker stops sending the audio stream to the server;

[0164] 7) Karaoke and scoring are finished;

[0165] 8) A pop-up prompt will appear on the karaoke content page, asking whether to save the song;

[0166] 9) The user clicks "Don't save the work" on the remote control, and the server deletes the cached data;

[0167] 10) The user clicks "Save Work" on the remote control, and the server takes the audio data sent by the speaker, extracts the data segment of each data packet, and splices all the data segments together according to the time sequence, which is all the audio data of the user's karaoke song;

[0168] 11) The server transcodes the audio data into FLAC format audio and saves it, and notifies the karaoke content page after successful saving;

[0169] 12) At the same time, the server will also upload the audio to the risk control interface to check whether the recording contains sensitive content;

[0170] 13) The karaoke content page receives the successful saving result returned by the server, pops up a prompt to inform the user that the saving is successful, and informs the path to view the playback;

[0171] 14) Users can find their own karaoke recordings in the karaoke area - My Works, click the play button, and play back their recordings through the TV and speakers;

[0172] 15) If the risk control interface verifies that there is no sensitive content, the server will notify the KTV area to pop up a prompt, "Your work has passed the review and can be shared with friends";

[0173] 16) Users can find their own karaoke recordings in the karaoke area - My Works. There is an additional share button on the right side of the approved works. Clicking the button will pop up a sharing QR code.

[0174] 17) Users can scan the shared QR code with their mobile phones to jump to the WeChat mini program, replay their own performance on the WeChat mini program, or share the mini program to their friends or share it with other users.

[0175] like Fig.10 FIG. 1 is a flow chart of step S1044 provided in an embodiment of the present invention, referring to FIG. Fig.10 As an optional implementation, the target audio stream data segment is scored according to the pitch sequence number of the target pronunciation to obtain the user pronunciation score of the target audio stream data segment, which specifically includes:

[0176] S10441, performing fast Fourier transform on the target audio stream data packet to obtain a target frequency domain signal, and determining a target imaginary number array according to the target frequency domain signal;

[0177] S10442, calculating the modulus of each imaginary number in the target imaginary number array, and determining the target sequence number of the imaginary number with the largest modulus in the target imaginary number array, and then calculating the first baud rate of the target audio stream data packet according to the target sequence number;

[0178] S10443, calculating a second baud rate of the target pronunciation according to the pitch sequence number of the target pronunciation;

[0179] S10444, calculating the absolute value of the difference between the first baud rate and the second baud rate, determining the baud rate deviation of the target audio stream data packet according to the ratio of the absolute value of the difference to the second baud rate, and then determining whether the target audio stream data packet is out of tune according to the baud rate deviation;

[0180] S10445, determining a first number of target audio stream data packets that are not out of tune in the target audio stream data segment, and a second number of target audio stream data packets that are out of tune;

[0181] S10446. Determine the user pronunciation score based on the first quantity and the second quantity.

[0182] Specifically, the audio transmitted from the speaker to the server is in PCM format, with a sampling rate of fs=44100 Hz, a number of channels c=1, and a bit depth d=16; the speaker samples data per second and occupies a memory space of SP=fs*c*d=44100*1*16=705600 bits of data; the pronunciation time of a word in a song at an ordinary speed is approximately between 0.2 and 0.5 seconds, and the pronunciation of each word requires at least 10 scoring points to more accurately judge whether the pronunciation is accurate, so at least 10 scores are required within 0.2 seconds and at least 50 scores are required within 1 second.

[0183] The embodiment of the present invention defines that one data packet is scored once, so the audio needs to send num=50 data packets to the server in one second; the data segment of each audio packet (excluding the audio header and tail frames and configuration information) occupies a space size of sp=SP / num=705600 / 50=14112 bits; because the PCM signal records a time domain signal, the baud rate of the audio cannot be obtained based on the time domain signal alone, so it is necessary to use Fourier transform to convert the PCM time domain signal into a frequency domain signal; since the general Fourier transform algorithm has a huge amount of calculation and a time complexity of O(N^2), the embodiment of the present invention uses a fast Fourier transform (FFT) algorithm with a time complexity of O(nlogn) to calculate the PCM signal.

[0184] FFT algorithm: For a given complex sequence x[n], its discrete Fourier transform X[k] is defined as:

[0185]

[0186] Where N is the length of the sequence x[n], i is the imaginary unit (satisfying i 2 =-1), e is the base of natural logarithms, k = 0, 1,…, N-1.

[0187] The sample space size of the FFT algorithm is N=sp / d=14112 / 16=882; because the sample space size N=882, after the PCM signal is converted into a frequency domain signal using the FFT algorithm, the result is an imaginary number array with a length of 882; according to the characteristics of the Fourier transform, the first half and the second half of the imaginary number array of the result are mirror images, and the embodiment of the present invention needs to pay attention to the first 441 imaginary numbers in the imaginary number array, and the second half can be directly ignored; the modulus of each imaginary number is calculated using the imaginary number modulus formula for the above 441 imaginary numbers, the largest value among the 441 moduli is found, and its serial number n is recorded.

[0188] In Fourier transform, the formula corresponding to the serial number and the baud rate is f=fs*n / N (for example, the modulus of the imaginary number with serial number 10 is the maximum modulus of the entire array, f=fs*n / N=44100*10 / 882=500, then the pronunciation frequency of the audio packet is 500Hz); therefore, the baud rate corresponding to the serial number n is the baud rate f1 of the PCM audio packet.

[0189] In the international pitch scale, the baud rate corresponding to the 69th semitone is 440Hz (since the baud rate corresponding to the 69th semitone is exactly an integer, the 69th semitone is selected). According to the standard, the ratio of every two semitones is 2^(1 / 12) (12th root of 2, approximately equal to 1.05946, the larger the semitone number, the larger the baud rate corresponding to it. For example, the baud rate of the 70th semitone divided by the baud rate of the 69th semitone is equal to the 12th root of 2). From this, the semitone pitch to baud rate calculation formula can be derived:

[0190] F=440*(2 (h-69) / 12 )

[0191] Among them, h is the pitch serial number corresponding to the semitone.

[0192] Since the song playback start time T1 has been recorded in the above karaoke process, each audio package has a timestamp T2, and the format of a scoring unit (one word per unit) in the scoring reference file is "t1 d1 h" (start time, duration, pitch sequence number). Each scoring unit in the scoring file is traversed, and according to the time judgment formula t1<(T2-T1)<(t1+d1), the pitch sequence number h of the audio package that meets the time requirement is found in the scoring reference file.

[0193] If the audio package does not have a corresponding scoring reference pitch serial number h, no score will be given (since the recording is continuous, and there is usually a gap between each line of lyrics in a song, it is normal that some data cannot find the corresponding value in the scoring reference file); if the corresponding pitch serial number h is found, the pitch-to-baud rate formula is used to calculate the baud rate f2 corresponding to h.

[0194] Since the ratio of the baud rates of two adjacent semitones is 2 1 / 12 , the embodiment of the present invention calculates the absolute value of the difference between f1 and f2 and the ratio of f2 a=|f1-f2| / f2, when a<2^ (1 / 12) When -1, the audio package is not "out of tune" (out of tune means singing in a different tone, such as a word that originally needs to be sung at the 69th note is sung at the 68th note, the 70th note or other notes).

[0195] The pitch calculation results of all audio packages of a word are counted, the number of not out of tune is n1, and the number of out of tune is n2; if n1=0, it means that the user did not sing it correctly at all, but considering the user's feelings, 0 points cannot be given. The embodiment of the present invention gives a "bottom line score" to the pronunciation of the word, and the default bottom line score is 20 points. Then, the user's singing score for the word, when n1=0, s=20; if n1>0, it means that at least one audio package in a word is sung accurately, and the minimum score is set to 50 points. The larger n1 is, the higher the score is, and if the proportion of n1 needs to be greater than n2, a weight value can be added to n1. Therefore, the score of a single word is defined in the embodiment of the present invention as, when n1>0, s=50+50*(n1*W / (n1*W+n2)), where W is the weight value of n1, which defaults to 2.

[0196] In summary, the scoring formula for a word (pronunciation) is: when n1=0, s=20; when n1>0, s=50+50*(n1*W / (n1*W+n2)), W=2.

[0197] The real-time score is the average score of all scored words. Suppose the score of each word is s1, s2, s3...sn, then the real-time score is S=(s1+s2+s3+...+sn) / n, where n represents the total number of scored audio packages.

[0198] In addition, when S≥90, the karaoke content page displays the evaluation as "SSS", when 90>S≥80, the evaluation is "SS", when 80>S≥70, the evaluation is "S", when 70>S≥60, the evaluation is "A", when 60>S≥50, the evaluation is "B", and when S<50, the evaluation is "C".

[0199] The above is a description of the method flow of the embodiment of the present invention. It can be recognized that the embodiment of the present invention binds the IPTV account to the network speaker, and while the network speaker plays the user's target audio stream, it pushes the target audio stream to the server, and scores the target audio stream in real time through the scoring reference file stored in the server, and transmits it to the EPG page for display, thereby realizing real-time scoring of music karaoke through the IPTV set-top box, overcoming the limitation problem that the karaoke solution based on APK in the prior art cannot be widely promoted on the IPTV set-top box, and also overcoming the problem that the karaoke solution based on the EPG page in the prior art cannot be scored, thereby improving the user experience.

[0200] Compared with the prior art, the embodiments of the present invention also have the following advantages:

[0201] 1) The IPTV box’s EPG page innovatively implements karaoke, real-time scoring, playback storage, and social sharing functions;

[0202] 2) It is compatible with a wide range of devices, as long as it is a normal TV box, it can be used;

[0203] 3) The effect is stable, and the scoring is not limited by the performance of the user's box;

[0204] 4) The workload of adaptation is small. When promoting in provinces, there is no need to request the EPG development team in each province to enable APK download, installation, and call permissions, nor is there any need to adapt the EPG page specifically for each province;

[0205] 5) It can accurately judge whether the user is out of tune;

[0206] 6) The karaoke results do not need to be uploaded again, and a sharing link can be quickly generated to share with friends or send to Moments;

[0207] 7) This solution provides a detailed scoring algorithm and a method for producing scoring reference files. In theory, all song videos can be independently converted into scoreable karaoke videos without being restricted by partners.

[0208] like Fig.11 FIG. 1 is a schematic diagram of a music karaoke device based on IPTV provided by an embodiment of the present invention, referring to FIG. Fig.11 The embodiment of the present invention provides a music karaoke device based on IPTV, comprising:

[0209] The music file playing module is used to respond to the user's music selection instruction, determine the target music file and the corresponding target music identifier, play the target music file, and send the target music identifier to the server;

[0210] A reference file determination module, used to determine a corresponding target scoring reference file according to a target music identifier through a server;

[0211] The audio stream acquisition module is used to acquire the user's target audio stream through the network speaker and send the target audio stream to the server;

[0212] The audio stream scoring module is used to score the target audio stream according to the target scoring reference file through the server, return the scoring result and display it through the EPG page.

[0213] The contents of the above method embodiments are all applicable to the present device embodiments. The functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0214] The embodiment of the present invention further provides an electronic device, the electronic device comprising: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory, and the program is executed by the processor to realize the above-mentioned music karaoke method based on IPTV. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0215] like Fig.12 FIG. 1 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. Fig.12 , an embodiment of the present invention provides an electronic device, including:

[0216] The processor 1201 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention;

[0217] The memory 1202 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1202 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1202, and the processor 1201 calls and executes the IPTV-based music karaoke method of the embodiment of the present invention;

[0218] Input / output interface 1203, used to implement information input and output;

[0219] The communication interface 1204 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0220] A bus 1205 that transmits information between the various components of the device (e.g., the processor 1201, the memory 1202, the input / output interface 1203, and the communication interface 1204);

[0221] The processor 1201 , the memory 1202 , the input / output interface 1203 and the communication interface 1204 are connected to each other in communication within the device via the bus 1205 .

[0222] like Fig.13 FIG. 1 is a schematic diagram of a storage medium according to an embodiment of the present invention. Fig.13 The embodiment of the present invention further provides a storage medium, which is a computer-readable storage medium used for computer-readable storage. The storage medium stores one or more programs 1301, and the one or more programs 1301 can be executed by one or more processors to implement the above-mentioned music karaoke method based on IPTV.

[0223] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0224] The embodiment of the present invention also discloses a computer program product or a computer program, wherein the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes Figure 1 The method shown.

[0225] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the above-mentioned boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part for which is described as a larger operation is performed independently.

[0226] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified to the contrary, one or more of the above-mentioned functions and / or features can be integrated into a single physical device and / or software module, or one or more functions and / or features can be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional skills of the engineer. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0227] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the above methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0228] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0229] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and editable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the above-mentioned program is printed, since the above-mentioned program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or processing in other suitable ways as necessary, and then stored in a computer memory.

[0230] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0231] In the above description of this specification, the description with reference to the terms "one embodiment / example", "another embodiment / example" or "certain embodiments / examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0232] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0233] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A music karaoke method based on IPTV, characterized in that: The following steps are involved: In response to a music selection instruction from a user, determining a target music file and a corresponding target music identifier, playing the target music file, and sending the target music identifier to a server; Determining, by the server, a corresponding target scoring reference file according to the target music identifier; Acquire a target audio stream of the user through a network speaker, and send the target audio stream to the server; The server scores the target audio stream according to the target scoring reference file, returns the scoring result and displays it through the EPG page.

2. The method for singing music karaoke based on IPTV according to claim 1, characterized in that: The music karaoke method further includes the step of pre-binding the network speaker, which specifically includes: In response to the user's device binding instruction, display the device binding QR code through the EPG page; Scanning the device binding QR code through the user terminal causes the user terminal to jump to the device binding page; Scanning the device QR code of the network speaker through the user terminal, and obtaining the device code of the network speaker according to the device QR code; In response to the user's binding confirmation instruction, the network speaker is bound to the user's IPTV account according to the device code.

3. The method for singing music karaoke based on IPTV according to claim 1, characterized in that: The method of responding to the user's music selection instruction, determining a target music file and a corresponding target music identifier, playing the target music file, and sending the target music identifier to the server specifically includes: Display the list of alternative music through the EPG page; When the music selection instruction is detected, the target music name selected by the user is determined according to the music selection instruction, and the corresponding target music file and the target music identifier are obtained according to the target music name; Display the playback preparation screen through the EPG page, and play the target music file after the preset countdown ends; The target music identifier is sent to the server via the IPTV set-top box.

4. The method for singing music karaoke based on IPTV according to claim 1, characterized in that: The step of determining the corresponding target scoring reference file according to the target music identifier specifically includes: Get the preset scoring file library; Perform a matching search in the scoring file library according to the target music identifier to obtain the target scoring reference file; The target scoring reference file includes the start time, duration and pitch sequence number of each target pronunciation.

5. The method for singing music karaoke based on IPTV according to claim 4, characterized in that: The music karaoke method further comprises the step of pre-building the scoring file library, which specifically comprises: Acquire multiple candidate music files, separate the vocals from the background sounds in the candidate music files, and obtain vocal files; Converting the vocal file into a pronunciation file in MIDI format, and removing noise from the pronunciation file to obtain a scoring file; Generate a scoring array corresponding to each pronunciation according to the start time, duration and pitch sequence number of each pronunciation in the scoring file, and generate a scoring reference file according to the scoring array; The music identifier of the candidate music file corresponding to each of the scoring reference files is determined, a key-value pair is generated according to the scoring reference file and the music identifier, and then the scoring file library is constructed according to the key-value pair.

6. The method for singing music karaoke based on IPTV according to claim 1, characterized in that: Scoring the target audio stream according to the target scoring reference file, returning the scoring result, and displaying it through the EPG page specifically includes: Determine the start time, duration and pitch sequence number of each target pronunciation according to the target scoring reference file; Determine a plurality of audio stream data packets of the target audio stream and corresponding timestamps; Determine a plurality of target audio stream data packets corresponding to each of the target pronunciations according to the start time and duration of the target pronunciation and the timestamp of the audio stream data packet, and then determine a target audio stream data segment according to the target audio stream data packet; Scoring the target audio stream data segment according to the pitch sequence number of the target pronunciation to obtain a user pronunciation score of the target audio stream data segment; The real-time scoring result is determined according to the average of the user pronunciation scores of the target audio stream data segments obtained at the current moment, and the real-time scoring result is returned to the IPTV set-top box, and then displayed through the EPG page.

7. The IPTV-based music karaoke method according to claim 6, characterized in that: The step of scoring the target audio stream data segment according to the pitch sequence number of the target pronunciation to obtain a user pronunciation score of the target audio stream data segment specifically includes: Performing a fast Fourier transform on the target audio stream data packet to obtain a target frequency domain signal, and determining a target imaginary number array according to the target frequency domain signal; Calculate the modulus of each imaginary number in the target imaginary number array, and determine the target sequence number of the imaginary number with the largest modulus in the target imaginary number array, and then calculate the first baud rate of the target audio stream data packet according to the target sequence number; Calculate a second baud rate of the target pronunciation according to the pitch sequence number of the target pronunciation; calculating an absolute value of a difference between the first baud rate and the second baud rate, determining a baud rate deviation of the target audio stream data packet according to a ratio of the absolute value of the difference to the second baud rate, and further determining whether the target audio stream data packet is out of tune according to the baud rate deviation; Determine a first number of the target audio stream data packets that are not out of tune in the target audio stream data segment, and a second number of the target audio stream data packets that are out of tune; The user pronunciation score is determined according to the first number and the second number.

8. A music karaoke device based on IPTV, characterized in that: include: A music file playing module, for responding to a user's music selection instruction, determining a target music file and a corresponding target music identifier, playing the target music file, and sending the target music identifier to a server; A reference file determination module, used to determine a corresponding target scoring reference file according to the target music identifier through the server; An audio stream acquisition module is used to acquire a user's target audio stream through a network speaker and send the target audio stream to the server; The audio stream scoring module is used to score the target audio stream according to the target scoring reference file through the server, return the scoring result and display it through the EPG page.

9. An electronic device, characterized in that: The electronic device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the steps of the IPTV-based music karaoke method as described in any one of claims 1 to 7 are realized.

10. A storage medium, the storage medium being a computer-readable storage medium, used for computer-readable storage, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the IPTV-based music karaoke method as described in any one of claims 1 to 7.

Images