Karaoke equipment

Abstract

This invention provides a karaoke device capable of determining whether or not death metal vocals are included in a karaoke performance. [Solution] A karaoke device comprising: an acquisition unit that acquires singing voice signals associated with a singer's karaoke singing at predetermined intervals; and a determination unit that determines whether the singer's karaoke singing is death metal vocal singing based on feature quantities corresponding to the maximum value, local maximum, and local minimum of the amplitude obtained from the singing voice signals at predetermined intervals acquired by the acquisition unit.

Description

Technical Field

[0001] The present invention relates to a karaoke apparatus.

Background Art

[0002] Among singers who perform karaoke singing using a karaoke apparatus, there are some who imitate the singing of professional singers and sing using special singing techniques such as vibrato, note dropping, melismatic ornamentation, and shouting. Patent Documents 1 to 4 disclose techniques for detecting such special singing techniques and evaluating singing.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Patent Document 3

Patent Document 4

Summary of the Invention

Problem to be Solved by the Invention

[0004] An object of the present invention is to provide a karaoke apparatus capable of determining whether or not death voice singing (described later) is included in karaoke singing.

Means for Solving the Problem

[0005] Death growl singing is a singing technique that produces a raspy or growling voice by narrowing the vocal tract with the soft palate and false vocal cords and vibrating them. The inventors of this invention analyzed the frequency spectrum of singing voice signals that are perceived as death growl singing and found that a certain feature quantity is smaller in death growl singing than in normal singing. This invention was completed based on this discovery and is a technology that can determine death growl singing by detecting this unique feature quantity of singing. The feature quantity is the value obtained by dividing the average of the differences between a predetermined number of amplitude maxima and the average of the two adjacent minimums by the maximum amplitude.

[0006] Specifically, the invention for achieving the above objective is a karaoke device comprising: an acquisition unit that acquires singing voice signals associated with a singer's karaoke singing at predetermined intervals; and a determination unit that determines whether the singer's karaoke singing is death metal vocal singing based on feature quantities corresponding to the maximum value, local maximum, and local minimum of the amplitude obtained from the singing voice signals at predetermined intervals acquired by the acquisition unit. Other features of the present invention will be revealed in the specification and drawings described below. [Effects of the Invention]

[0007] According to the present invention, it is possible to determine whether or not death metal vocals are included in karaoke singing. [Brief explanation of the drawing]

[0008] [Figure 1] This figure shows an example of the hardware configuration of a karaoke device according to an embodiment. [Figure 2A] This figure shows an example of the frequency spectrum of normal singing according to the embodiment. [Figure 2B] This figure shows an example of the frequency spectrum of death metal vocals according to the embodiment. [Figure 3] This figure shows an example of the software configuration of a karaoke unit according to the embodiment. [Figure 4] This is a flowchart showing the processing performed by the karaoke device according to the embodiment. [Modes for carrying out the invention]

[0009] <Embodiment> A karaoke device 1 according to an embodiment will be described with reference to Figures 1 to 4.

[0010] ==Karaoke Equipment== Karaoke device 1 is a device for karaoke performance and for singers to sing karaoke. Karaoke device 1 registers the karaoke songs selected by the singer in a reservation queue and performs the karaoke in order. As shown in Figure 1, karaoke device 1 comprises a karaoke unit 10, speakers 20, a display device 30, a microphone 40, and a remote control device 50.

[0011] The speaker 20 is configured to emit sound based on the sound signal emitted from the karaoke unit 10. The display device 30 is configured to display video and images on the screen based on the signal from the karaoke unit 10. The microphone 40 is configured to convert the singer's voice (sound input to the microphone 40) into an analog singing voice signal and input it to the karaoke unit 10. The remote control device 50 is a device for performing various operations on the karaoke unit 10. The singer can use the remote control device 50 to select the karaoke song they wish to sing. Icons for inputting instructions for various operations are displayed on the display screen of the remote control device 50.

[0012] The karaoke unit 10 performs various controls related to karaoke singing, such as controlling the playback of selected karaoke songs, controlling the display of lyrics and background images, and processing the singing voice signal input through the microphone 40. As shown in Figure 1, the karaoke unit 10 comprises a control unit 11, a communication unit 12, a storage unit 13, an audio processing unit 14, a display processing unit 15, and an operation unit 16. Each component is connected to bus B via an interface (not shown).

[0013] The control unit 11 includes a CPU 11a and a memory 11b. The CPU 11a implements various control functions by executing programs stored in the memory 11b. The memory 11b is a storage device that stores programs to be executed by the CPU 11a, and temporarily stores various types of information during program execution.

[0014] The communication unit 12 provides an interface for connecting the karaoke main body 10 to a communication line via a router (not shown).

[0015] The storage unit 13 is a large-capacity storage device that stores various types of data, such as a hard disk drive. The storage unit 13 stores a plurality of pieces of music data for performing karaoke performances by the karaoke apparatus 1.

[0016] Each piece of music data is assigned a music ID for identifying an individual karaoke piece of music. The music data includes accompaniment data, reference data, and the like. The accompaniment data is the base data for karaoke performance sounds. The accompaniment data includes information indicating the tempo during the karaoke performance. A predetermined tempo value is set for each piece of music. The reference data is data used as a reference when scoring a karaoke singing performance by a singer.

[0017] In addition, the storage unit 13 stores lyrics telop data for causing the display device 30 or the like to display lyrics corresponding to each karaoke piece of music, background image data such as background images displayed on the display device 30 or the like during karaoke performances, performance time data indicating the karaoke performance time for each piece of music, and music attribute information (information related to the music such as singer name, lyricist / composer name, and genre).

[0018] Here, the storage unit 13 according to the present embodiment stores a trained model.

[0019] The trained model according to the present embodiment outputs a determination result as to whether or not a karaoke singing performance based on a singing voice signal is a death metal vocal performance.

[0020] The trained model is constructed by training it using data that associates features obtained from samples of singing audio signals with singing methods (normal singing or death metal singing) as training data.

[0021] The feature is obtained by dividing the average of the differences between a predetermined number of amplitude maxima and the average of the two adjacent local minimums by the maximum amplitude, as shown in the following formula (1).

[0022]

number

[0023] More specifically, the feature G can be obtained using the following procedure. (1) Apply a Han window to the singing voice signal for a predetermined section and perform a Fast Fourier Transform to obtain the resulting frequency spectrum sequence x = {x0, x1, ..., xN}. N is the window length included in the sequence x. (2) Maximum value x in the series x max We seek. (3) Find the sequence X = {X0, X1, X2…, XN} obtained by arranging the local maximums within the sequence x in descending order. n is a predetermined number. (4) Compare each local maximum of series X with each local maximum of series x and identify the local maximum that has the same value. Find the series Y={Y0,Y1,Y2…,YN} by averaging the local minimums of the identified local maximum and two adjacent local minimums. (5) The feature G is obtained by substituting the values ​​obtained in (2) to (4) into equation (1).

[0024] Figure 2A shows an example of the frequency spectrum of normal singing, and Figure 2B shows an example of the frequency spectrum of death metal vocals. In each figure, the horizontal axis represents frequency, and the vertical axis represents amplitude. The solid line represents the frequency spectrum, and the dashed line connects the maximum value to two adjacent minimum values. Circles represent maximum values, and triangles represent minimum values. As is clear from these figures, the difference in amplitude between the maximum value and the average of two adjacent minimum values ​​in a given frequency band is smaller for death metal vocals than for normal singing.

[0025] The singing audio signal samples used to build the trained model will be selected from karaoke performances that differ in song genre, singing style, singer's age, gender, etc. Furthermore, to avoid bias, singing audio signals from multiple singers performing the same song will be used.

[0026] Furthermore, the singing style included in the training data (normal singing or death metal vocals) is determined by listening to samples of the singing audio signals from which features have been extracted.

[0027] The server device (not shown) constructs a pre-trained model using training data. In this example, the server device (not shown) constructs a pre-trained model of a neural network that takes features as input and outputs singing methods by performing supervised learning to learn predetermined values ​​in the features.

[0028] The predetermined value is the difference in amplitude between a single local maximum and the average of two adjacent local minimums in a given frequency band. A neural network, such as a CNN (Convolutional Neural Network), has an input layer that accepts feature inputs, an output layer that outputs singing methods, and a hidden layer that extracts the predetermined value. The input layer passes the input feature to the hidden layer. The hidden layer passes the extracted predetermined value to the output layer. The output layer identifies the singing method based on the predetermined value output from the hidden layer.

[0029] Furthermore, the server device (not shown) inputs feature quantities into the input layer and obtains the singing method results from the output layer. The singing method results output from the output layer are compared with the ground truth data, and the parameters are optimized so that the singing method results approach the ground truth data. Parameters include, for example, the weights (connection coefficients) between neurons and the coefficients of the activation function used in each neuron. The method of parameter optimization is not particularly limited, but for example, various parameters can be optimized using backpropagation.

[0030] The server device (not shown) transmits the constructed trained model to the karaoke device. The storage unit 13 of the karaoke device 1 stores the received trained model.

[0031] Furthermore, the trained model may be constructed using other learning algorithms, not limited to neural networks, such as SVM (Support Vector Machine), Bayesian networks, or random forests.

[0032] The sound processing unit 14 controls the performance of karaoke songs and processes the singing voice signals input through the microphone 40, based on the control of the control unit 11. The display processing unit 15 processes various displays on the display device 30 and the remote control device 50, based on the control of the control unit 11. For example, the display processing unit 15 controls the display device 30 to display an image on which lyrics and various icons are superimposed on the background image during the performance of a karaoke song. Alternatively, the display processing unit 15 displays various icons for operation input on the display screen of the remote control device 50. The operation unit 16 consists of a panel switch and a remote control receiving circuit, and outputs operation signals such as song selection signals and performance stop signals to the control unit 11 in response to the singer's operation of the panel switch of the karaoke device 1 or the remote control device 50. The control unit 11 detects the operation signals from the operation unit 16 and executes the corresponding processing.

[0033] (Software configuration) Figure 3 shows an example of the software configuration of the karaoke unit 10 according to this embodiment. The karaoke unit 10 comprises an acquisition unit 100, a determination unit 200, and a presentation unit 300. The acquisition unit 100, the determination unit 200, and the presentation unit 300 are realized by the CPU 11a executing a program stored in memory 11b.

[0034] [Acquisition Department] The acquisition unit 100 acquires the singing audio signal associated with the singer's karaoke performance at predetermined intervals.

[0035] A predetermined interval is set in advance, for example, as one frame (1024 samples).

[0036] A singer using the karaoke machine 1 operates the remote control device 50 to select the song they wish to sing. The karaoke machine 1 plays the karaoke version of the song selected by the singer. The singer sings along to the karaoke version. The acquisition unit 100 acquires the singing voice signal input through the microphone 40 at predetermined intervals in conjunction with the karaoke singing. The acquisition of the singing voice signal may be performed sequentially as the karaoke singing of the song progresses, or it may be performed all at once after the karaoke singing of the song has been completed.

[0037] [Judgment section] The determination unit 200 determines whether the singer's karaoke singing is death metal vocals based on feature quantities corresponding to the maximum, local maximum, and local minimum amplitude values ​​obtained from the singing audio signals for each predetermined section acquired by the acquisition unit 100.

[0038] The determination unit 200 calculates a feature by dividing the average of the differences between a predetermined number of amplitude maxima and the average of the two adjacent minimums by the maximum amplitude. The feature can be calculated using the method described above.

[0039] The determination unit 200 determines, based on the calculated feature quantities, whether the singer's karaoke performance is death metal vocals.

[0040] In this embodiment, the determination unit 200, upon receiving feature quantities obtained from the singing voice signals for each predetermined section acquired by the acquisition unit 100, uses a pre-trained model that has been trained to output a determination result on whether the karaoke singing based on the singing voice signals is death metal singing to determine whether the singer's karaoke singing is death metal singing.

[0041] Specifically, the determination unit 200 inputs the calculated feature quantities (i.e., feature quantities in a predetermined interval) to the trained model stored in the memory unit 13. The trained model determines whether the input feature quantities are those of normal singing or those of death metal singing, and outputs a determination result indicating that it is either normal singing or death metal singing.

[0042] If the trained model outputs a result indicating that the singing is death metal vocals, the determination unit 200 determines that the karaoke singing in the predetermined section is death metal vocals. On the other hand, if the trained model outputs a result indicating that the singing is normal, the determination unit 200 determines that the karaoke singing in the predetermined section is normal singing. The determination unit 200 outputs the determination result to the presentation unit 300.

[0043] [Presentation part] If the display unit 300 determines that a singer's karaoke performance is death metal vocals, it displays the determination result or information based on that determination result to the singer.

[0044] For example, suppose death metal vocals are used in a predetermined section of the song. In this case, the display unit 300 can display an icon on the display device 30 indicating that death metal vocals have been used. The display unit 300 may also notify the singer of the death metal vocals through the speaker 20 using sound (applause, cheers, etc.). The display unit 300 may also display images that evoke death metal instead of the background video of the song. These are examples of "presenting the judgment result to the singer."

[0045] Alternatively, when the display unit 300 displays the scoring result for karaoke singing, it can display a scoring result that includes points added according to the number of predetermined sections in which death metal vocals were sung. This is an example of "presenting information based on the judgment result to the singer."

[0046] ==Regarding the operation of karaoke machine 1== Next, a specific example of the operation of the karaoke device 1 in this embodiment will be described with reference to Figure 4. Figure 4 is a flowchart of an example of the operation of the karaoke device 1. In this example, the memory unit 13 stores a trained model that, when a feature quantity obtained from the singing voice signal is input, outputs a determination result of whether the karaoke singing based on the singing voice signal is death metal singing. In this example, it is assumed that the determination of whether or not it is death metal singing is performed at predetermined intervals.

[0047] The singer operates the remote control device 50 to select the song they wish to sing karaoke to. The karaoke device 1 starts playing the selected song (karaoke performance starts. Step 10).

[0048] The acquisition unit 100 acquires the singing audio signal associated with the singer's karaoke performance at predetermined intervals (acquisition of singing audio signal at predetermined intervals; step 11).

[0049] The determination unit 200 receives the feature quantities obtained from the singing audio signal acquired in step 11 as input to the trained model. The trained model outputs a determination result indicating whether the karaoke singing based on the singing audio signal is death metal singing. Based on the output determination result, the determination unit 200 determines whether the singer's karaoke singing is death metal singing (determination of whether it is death metal singing; step 12).

[0050] If the singer's karaoke performance is determined to be death metal vocals (the result is Y in step 13), the presentation unit 300 presents the determination result to the singer (presentation of determination result; step 14).

[0051] The karaoke device 1 repeatedly performs the processes from step 11 to step 14 until the karaoke performance of the song is finished (if the result is Y in step 15).

[0052] Specifically, singer S operates the remote control device 50 to select the song M that they wish to sing karaoke to. The karaoke device 1 starts playing the selected song M. Singer S begins singing karaoke along with the karaoke music.

[0053] The acquisition unit 100 acquires the singing audio signal for the first frame (1024 samples) as the singer S sings karaoke. The acquisition unit 100 outputs the acquired singing audio signal to the determination unit 200.

[0054] The determination unit 200 performs a fast Fourier transform on the singing audio signal output from the acquisition unit 100 to obtain a sequence of frequency spectra x = {x1, x2, ..., x1024}.

[0055] In this example, the sampling frequency is assumed to be 44100Hz. The window length is 1024. Therefore, the determination unit 200 calculates the amplitude at 1024 frequency points approximately every 43Hz using the Fast Fourier Transform. Specifically, the determination unit 200 calculates the amplitude x0 at frequency 0, the amplitude x1 at frequency 43Hz, the amplitude x2 at frequency 86Hz, the amplitude x3 at frequency 129Hz, the amplitude x4 at frequency 172Hz, the amplitude x5 at frequency 215Hz, ... and the amplitude x1024 at frequency 44056.466Hz (a total of 1024 points).

[0056] The determination unit 200 selects the maximum amplitude x from the sequence x of the frequency spectrum. max We seek.

[0057] The determination unit 200 finds a sequence X = {X0, X1, X2…, Xn} by arranging a predetermined number of (in this case, n+1) local maximum values ​​within the sequence x in descending order.

[0058] The determination unit 200 determines the maximum value X0 of the sequence X (i.e., x maxThe maximum value that exhibits the same amplitude as ( ) is identified from the sequence x. In this example, assume that the amplitude xj has been identified from the sequence x.

[0059] The determination unit 200 identifies the minimum values ​​of two points adjacent to the amplitude xj. For example, the determination unit 200 searches for a frequency band lower than the amplitude xj and identifies amplitudes xj-1, xj-2, and xj-3. If the amplitude values ​​are in the relationship xj-1 > xj-3 > xj-2, the determination unit 200 identifies amplitude xj-2 as the minimum value adjacent to amplitude xj (maximum value) in the frequency band lower than the amplitude xj. Similarly, the determination unit 200 searches for a frequency band higher than the amplitude xj and identifies amplitudes xj+1 and xj+2. If the amplitude values ​​are in the relationship xj+2 > xj+1, the determination unit 200 identifies amplitude xj+1 as the minimum value adjacent to amplitude xj (maximum value) in the frequency band higher than the amplitude xj.

[0060] The determination unit 200 calculates the average Yj of the amplitude values ​​of the two local minimums, amplitude xj-2 and xj+1, which are adjacent to the identified amplitude xj.

[0061] The determination unit 200 repeatedly performs the above process to calculate a predetermined number of averages Y0 to YN. The determination unit 200 then obtains a series Y={Y0,Y1,Y2…,YN} by arranging the calculated averages.

[0062] The determination unit 200 determines x max The feature vector G is obtained by substituting the values ​​of the sequence X={X0,X1,X2…,Xn} and the sequence Y={Y0,Y1,Y2…,YN} into equation (1).

[0063] The determination unit 200 inputs the obtained feature quantity G into the trained model stored in the memory unit 13. The trained model outputs a determination result indicating whether the input feature quantity G is death metal vocals. Based on the output determination result, the determination unit 200 determines whether the singer's karaoke performance is death metal vocals.

[0064] For example, if the trained model outputs a result indicating that feature G is death metal vocals, the determination unit 200 determines that singer S's karaoke performance is death metal vocals. In this case, singer S is performing death metal vocals in the first frame of the karaoke performance of song M.

[0065] If singer S's karaoke performance is determined to be death metal vocals, the presentation unit 300 presents the determination result to singer S.

[0066] Karaoke device 1 performs the above process every frame until the karaoke performance of song M is finished (until singer S finishes singing karaoke), determining whether or not it is death metal vocals and displaying the determination result.

[0067] As is clear from the above, the karaoke device 1 according to this embodiment includes an acquisition unit 100 that acquires singing voice signals associated with a singer's karaoke singing at predetermined intervals, and a determination unit 200 that determines whether the singer's karaoke singing is death metal singing based on feature quantities corresponding to the maximum value, local maximum, and local minimum of the amplitude obtained from the singing voice signals at predetermined intervals acquired by the acquisition unit 100.

[0068] More specifically, the determination unit 200 calculates feature quantities by dividing the average of the differences between a predetermined number of amplitude maxima and the average of the two adjacent minimums by the maximum amplitude. Furthermore, when the determination unit 200 receives feature quantities obtained from the singing voice signals for each predetermined interval acquired by the acquisition unit 100, it uses a pre-trained model that has been trained to output a determination result on whether the karaoke singing based on the singing voice signals is death metal singing to determine whether the singer's karaoke singing is death metal singing.

[0069] In other words, according to the karaoke device 1 of this embodiment, it is possible to determine whether or not death metal vocals are included in the karaoke singing using a trained model.

[0070] Furthermore, the karaoke device 1 according to this embodiment has a display unit 300 that, when it is determined that the singer's karaoke performance is death metal vocal performance, presents the determination result or information based on the determination result to the singer. With such a karaoke device 1, the result of the determination that the singing is death metal vocal performance can be presented to the singer directly or indirectly.

[0071] <Variation> The output of a trained model is not limited to the examples of the embodiment. For example, a trained model can be used to output trend data indicating whether a karaoke performance based on a singing voice signal is a death metal vocal performance, given features obtained from a singing voice signal as input.

[0072] Trend data indicates the tendency for a song to be sung using death growls. For example, trend data can be represented by a value of "0" for normal singing and "1" for death growls.

[0073] The determination unit 200 inputs the obtained features into the trained model stored in the memory unit 13. The trained model outputs trend data indicating whether the input features are death metal vocals. Based on the output trend data, the determination unit 200 determines whether the singer's karaoke performance is death metal vocals.

[0074] The determination of whether or not singing is death metal vocals can be made based on a pre-set threshold. For example, as described above, if the trend data is shown as a value between 0 and 1, the determination unit 200 checks whether the output trend data value is 0.5 or greater than the pre-set threshold. If the trend data value is 0.5 or greater, the determination unit 200 determines that the singer's karaoke performance is death metal vocals.

[0075] As is clear from the above, the determination unit 200 in the karaoke device 1 according to this modified version can determine whether a singer's karaoke performance is death metal vocals by using a pre-trained model that has been trained to output trend data indicating whether the karaoke performance based on the singing voice signal is death metal vocals, when it receives feature quantities obtained from the singing voice signal for each predetermined section acquired by the acquisition unit 100 as input. In other words, the karaoke device 1 according to this modified version can determine whether a karaoke performance includes death metal vocals using a pre-trained model.

[0076] <Other> It is also possible to supply the program to a computer using a non-transitory computer-readable medium (with an executable program thereon) on which the above program is stored. Examples of non-transitory computer-readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), CD-ROMs (Read Only Memory), etc.

[0077] The above embodiments are presented as examples and do not limit the scope of the invention. The above configurations can be combined as appropriate, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. The above embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents. [Explanation of Symbols]

[0078] 1. Karaoke machine 10 Karaoke machine 100 Acquisition Department 200 Judgment section 300 Presentation section

Claims

1. An acquisition unit that acquires the singing audio signal associated with the singer's karaoke performance at predetermined intervals, A determination unit determines whether the singer's karaoke singing is death metal vocals, based on feature quantities corresponding to the maximum amplitude, local maximum, and local minimum values ​​obtained from the singing audio signal for each predetermined section acquired by the acquisition unit. A karaoke machine having the following features.

2. The karaoke apparatus according to claim 1, characterized in that the determination unit calculates the feature quantity by dividing the average of the differences between a predetermined number of amplitude maxima and the average of two adjacent minimums by the maximum amplitude.

3. The karaoke apparatus according to claim 1, characterized in that the determination unit, upon input of feature quantities obtained from singing voice signals for predetermined intervals acquired by the acquisition unit, uses a pre-trained model that has been trained to output a determination result of whether the karaoke singing based on the singing voice signals is death metal singing, to determine whether the singer's karaoke singing is death metal singing.

4. The karaoke apparatus according to claim 1, characterized in that the determination unit, upon input of feature quantities obtained from singing voice signals for each predetermined section acquired by the acquisition unit, determines whether the singer's karaoke singing is death metal singing using a pre-trained model that has been trained to output trend data indicating whether the karaoke singing based on the singing voice signals is death metal singing.

5. The karaoke device according to any one of claims 1 to 4, characterized in that, if the karaoke performance of the singer is determined to be death metal vocals, it has a display unit that presents the determination result or information based on the determination result to the singer.

Images