A humming recognition method, device and medium

Abstract

The application discloses a humming recognition method and device and a medium, and applies to the technical field of song recognition. The method comprises the following steps: extracting the pitch of first audio data and second audio data respectively, obtaining a first pitch sequence corresponding to the first audio data and a second pitch sequence corresponding to the second audio data; performing segmentation on the first pitch sequence and the second pitch sequence according to a preset window length and a preset window shift, obtaining a first pitch vector sequence corresponding to the first pitch sequence and a second pitch vector sequence corresponding to the second pitch sequence; determining the vector distance of the pitch vectors in the first pitch vector sequence and the second pitch vector sequence; determining the sequence distance of the first pitch vector sequence and the second pitch vector sequence based on the vector distance; and determining the humming recognition result corresponding to the first audio data based on the sequence distance. In this way, the humming recognition accuracy can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of song recognition, in particular to a humming recognition method, device and medium. BACKGROUND

[0002] At present, there are many complex scenarios for song recognition, such as with or without accompaniment and lyrics, slight changes in speed, different timbres, gender and age of users, etc. Humming recognition, as one of the important methods of humming recognition, only needs the user to hum the tune, and can match to the corresponding song based on the user's humming tune whether there is a lyric or not. Since the humming recognition algorithm only focuses on the pitch sequence of the user at different times for matching, the robustness is relatively high. Because humming recognition can still accurately recognize in the above various complex scenarios, it has a considerable degree of tolerance for changes in user audio, so it is increasingly widely used.

[0003] The existing humming recognition technical solution is usually composed of pitch extraction and sequence matching. For the user's request audio or audio data in the database, the pitch sequence in the audio needs to be extracted by the pitch extraction algorithm, and then the distance between any two pitch sequences is compared using the sequence matching algorithm. The main disadvantage of this solution is that the distance calculation only considers the difference between a single pitch point in the two sequences. In actual application, single pitch point errors occur from time to time, due to the instability of the pitch extraction algorithm, noise influence, and unstable user voice control, etc., resulting in an inaccurate final recognition result. In summary, in the process of implementing the present application, the inventors have found at least that the prior art has the problem of low humming recognition accuracy. SUMMARY

[0004] Therefore, the present application aims to provide a humming recognition method, device and medium, which can improve the humming recognition accuracy. The specific solutions are as follows:

[0005] In a first aspect, the present application discloses a humming recognition method, comprising:

[0006] extracting the pitches of the first audio data and the second audio data respectively to obtain a first pitch sequence corresponding to the first audio data and a second pitch sequence corresponding to the second audio data;

[0007] dividing the first pitch sequence and the second pitch sequence according to a preset window length and a preset window shift to obtain a first pitch vector sequence corresponding to the first pitch sequence and a second pitch vector sequence corresponding to the second pitch sequence;

[0008] determining the vector distance of the pitch vectors in the first pitch vector sequence and the second pitch vector sequence;

[0009] determine a sequence distance of the first pitch vector sequence and the second pitch vector sequence based on the vector distance;

[0010] determine a humming recognition result corresponding to the first audio data based on the sequence distance.

[0011] Optionally, after the first pitch sequence and the second pitch sequence are divided into the first pitch vector sequence and the second pitch vector sequence respectively according to the preset window length and the preset window shift, the method further comprises:

[0012] determine a target pitch value corresponding to each pitch vector in the first pitch vector sequence and the second pitch vector sequence;

[0013] for each pitch vector, subtract the target pitch value from an element in the vector.

[0014] Optionally, the determining of the target pitch value corresponding to each pitch vector in the first pitch vector sequence and the second pitch vector sequence comprises:

[0015] determining a first element value in each pitch vector as the target pitch value;

[0016] or, calculating a mean value of each element value in each pitch vector respectively, and determining the mean value as the target pitch value;

[0017] or, determining a previous element value of each element in each pitch vector as the target pitch value corresponding to the element.

[0018] Optionally, the determining of the vector distance of the pitch vectors in the first pitch vector sequence and the second pitch vector sequence comprises:

[0019] calculating a cosine similarity and / or a modulus difference value of the pitch vectors in the first pitch vector sequence and the second pitch vector sequence;

[0020] determining the vector distance of the pitch vectors based on the cosine similarity and / or the modulus difference value.

[0021] Optionally, the determining of the vector distance of the pitch vectors in the first pitch vector sequence and the second pitch vector sequence comprises:

[0022] calculating an Euclidean distance of the pitch vectors in the first pitch vector sequence and the second pitch vector sequence to obtain the vector distance.

[0023] Optionally, the determining of the vector distance of the pitch vectors in the first pitch vector sequence and the second pitch vector sequence comprises:

[0024] determine a maximum vector distance;

[0025] determine vector distances of pitch vectors in the first pitch vector sequence and the second pitch vector sequence based on the maximum vector distance; wherein the vector distances are less than or equal to the maximum vector distance.

[0026] Optionally, the determining of the vector distances of the pitch vectors in the first pitch vector sequence and the second pitch vector sequence comprises:

[0027] determining vector distances of the pitch vectors in the first pitch vector sequence and the second pitch vector sequence to obtain a distance matrix;

[0028] Correspondingly, the determining of the sequence distance of the first pitch vector sequence and the second pitch vector sequence based on the vector distances comprises:

[0029] finding a shortest distance path from the distance matrix, and determining the sequence distance of the first pitch vector sequence and the second pitch vector sequence based on the shortest distance path.

[0030] Optionally, the determining of the humming recognition result corresponding to the first audio data based on the sequence distance comprises:

[0031] determining whether the sequence distance satisfies a preset condition;

[0032] if there are multiple sequence distances corresponding to the second audio data that satisfy the preset condition, determining the humming recognition result corresponding to the first audio data according to the second audio data with the smallest sequence distance.

[0033] In a second aspect, a humming recognition apparatus comprises:

[0034] a pitch extraction module configured to extract pitches of first audio data and second audio data respectively to obtain a first pitch sequence corresponding to the first audio data and a second pitch sequence corresponding to the second audio data;

[0035] a sequence segmentation module configured to segment the first pitch sequence and the second pitch sequence according to a preset window length and a preset window shift to obtain a first pitch vector sequence corresponding to the first pitch sequence and a second pitch vector sequence corresponding to the second pitch sequence;

[0036] a vector distance determination module configured to determine vector distances of pitch vectors in the first pitch vector sequence and the second pitch vector sequence;

[0037] a sequence distance determination module, configured to determine a sequence distance of the first pitch vector sequence and the second pitch vector sequence based on the vector distance;

[0038] a recognition result determination module, configured to determine the humming recognition result corresponding to the first audio data based on the sequence distance.

[0039] In a third aspect, the present application discloses an electronic device, comprising a processor and a memory; wherein,

[0040] the memory, configured to save a computer program;

[0041] the processor, configured to execute the computer program to implement the humming recognition method.

[0042] In a fourth aspect, the present application discloses a computer readable storage medium, configured to save a computer program, wherein the computer program is executed by a processor to implement the humming recognition method.

[0043] It can be seen that, in the present application, the pitches of the first audio data and the second audio data are extracted respectively to obtain a first pitch sequence corresponding to the first audio data and a second pitch sequence corresponding to the second audio data, then the first pitch sequence and the second pitch sequence are cut according to a preset window length and a preset window shift to obtain a first pitch vector sequence corresponding to the first pitch sequence and a second pitch vector sequence corresponding to the second pitch sequence, the vector distance of the pitch vectors in the first pitch vector sequence and the second pitch vector sequence is determined, the sequence distance of the first pitch vector sequence and the second pitch vector sequence is determined based on the vector distance, and finally the humming recognition result corresponding to the first audio data is determined based on the sequence distance. That is, in the present application, when performing humming recognition, the pitch sequences corresponding to the first audio data and the second audio data are cut to obtain pitch vector sequences, the vector distance of the pitch vectors in the two pitch vector sequences is determined, the sequence distance of the pitch vector sequences is determined based on the vector distance, and finally the recognition result is obtained. Since the pitch vector includes pitch change and front and back pitch information, compared with the scheme of recognizing based on the distance of a single pitch point, the accuracy of humming recognition can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor based on the provided drawings.

[0045] Figure 1 A system framework schematic diagram applicable to the humming recognition scheme provided in the present application;

[0046] Figure 2 A humming recognition method flowchart provided in the present application;

[0047] Figure 3 A DTW distance matrix schematic diagram provided in the present application;

[0048] Figure 4 A specific humming recognition method flowchart provided in the present application;

[0049] Figure 5 A vector schematic diagram provided in the present application;

[0050] Figure 6 A specific humming recognition interface schematic diagram provided in the present application;

[0051] Figure 7 A humming recognition device structure schematic diagram provided in the present application;

[0052] Figure 8 A structure diagram of an electronic device provided in the present application. DETAILED DESCRIPTION

[0053] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0054] The existing humming recognition technical scheme is usually composed of pitch extraction and sequence matching. For the request audio of the user or the audio data in the database, the pitch sequence in the audio needs to be extracted by the pitch extraction algorithm, and then the distance between any two pitch sequences is compared by using the sequence matching algorithm. The main disadvantage of this scheme is that the distance calculation only considers the difference between a single pitch point in the two sequences. In actual application, errors in single pitch points occur from time to time, and the reasons are instability of the pitch extraction algorithm, noise influence, and unstable user voice control, etc., which leads to an inaccurate final recognition result. In summary, in the process of implementing the present application, the inventors have at least found that the existing technology has the problem of low humming recognition accuracy. Therefore, the present application provides a humming recognition scheme, which can improve the accuracy of humming recognition.

[0055] In the humming recognition scheme of the present application, the system framework can be specifically seen from Figure 1As shown, specifically can include: background server and a number of user terminals establishing communication connection with the background server. Wherein, the user terminal includes but not limited to tablet computer, notebook computer, smart phone, personal computer (PC), not limited here.

[0056] In this application, the background server executes the steps of the humming recognition scheme method, including extracting the pitch of the first audio data and the second audio data respectively, obtaining the first pitch sequence corresponding to the first audio data and the second pitch sequence corresponding to the second audio data; according to the preset window length and the preset window shift, the first pitch sequence and the second pitch sequence are cut respectively, and the first pitch vector sequence corresponding to the first pitch sequence and the second pitch vector sequence corresponding to the second pitch sequence are obtained; the vector distance of the pitch vector in the first pitch vector sequence and the second pitch vector sequence is determined; the sequence distance of the first pitch vector sequence and the second pitch vector sequence is determined based on the vector distance; the humming recognition result corresponding to the first audio data is determined based on the sequence distance, and finally the humming recognition result is pushed to the user terminal display.

[0057] Referring to Figure 2 As shown, the embodiment of the present application discloses a humming recognition method, comprising:

[0058] Step S11: Extracting the pitch of the first audio data and the second audio data respectively, obtaining the first pitch sequence corresponding to the first audio data and the second pitch sequence corresponding to the second audio data.

[0059] In a specific embodiment, the pitch of the first audio data and the second audio data can be extracted based on a preset pitch extraction algorithm. Wherein, the preset pitch extraction algorithm can include but not limited to Yin, Pyin (i.e. Probabilisticyin, probabilistic Yin), Crepe (i.e. Convolutional representation for pitch estimation, convolutional representation for pitch estimation) and the like. The first audio data can be the user's request audio, or other audio to be identified. The second audio data can be the audio data with accurate song information in the database.

[0060] Step S12: According to the preset window length and the preset window shift, the first pitch sequence and the second pitch sequence are cut respectively, and the first pitch vector sequence corresponding to the first pitch sequence and the second pitch vector sequence corresponding to the second pitch sequence are obtained.

[0061] For example, two pitch sequences X1=[x 1,1 ,x1,2 ..., x 1,N ] and X2 = [x 1,1 , x 1,2 ..., x 1,M ], are divided into multiple pitch vectors according to the window length L and the window shift K, where x 1,1 , x 1,2 ..., x 1,N , x 1,1 , x 1,2 ..., x 1,M represent the pitch values at different time points. For example, vector 1 is [x 1,1 , x 1,2 ..., x 1,L ] and vector 2 is [x 1,1+K , x 1,2+K ..., x 1,L+K ], and so on. In this way, the pitch sequence is expressed in the form of vectors. The window length and the window shift can be selected through experiments to obtain better results.

[0062] Step S13: determining the vector distance of the pitch vectors in the first pitch vector sequence and the second pitch vector sequence.

[0063] In a specific embodiment, the vector distance of the pitch vectors in the first pitch vector sequence and the second pitch vector sequence can be directly determined.

[0064] Step S14: determining the sequence distance of the first pitch vector sequence and the second pitch vector sequence based on the vector distance.

[0065] It should be noted that when comparing any two pitch sequences, since the lengths of the pitch sequences extracted from different audio signals are different, a better method for matching unequal length sequences is DTW (Dynamic Time Warping). The DTW algorithm is directly based on the pitch sequence to calculate the distance function. For example, for two pitch sequences X1 = [x 1,1 , x 1,2 ..., x 1,N ] and X2 = [x 1,1 , x 1,2 ..., x 1,M ] with lengths N and M respectively, the i, j element in the DTW distance matrix D is defined as the distance between x 1,i and x 2,j . It is usually defined as L1-Norm, the absolute value of the difference between the two, i.e. D i,j = |x 1,i - x 2,j |, as shown in formula (1). Figure 3 Figure 3 ​For a DTW distance matrix diagram in the prior art, find the shortest path in the distance matrix D to get the distance between the sequences. This scheme only considers the difference between a single pitch point in two sequences when calculating the distance. However, the embodiments of the present application can use the pitch vectors obtained by cutting the sequences to better utilize the pitch changes, the information of the previous and subsequent pitches, and the like, thereby improving the recognition accuracy.

[0066] Step S15: Determine the humming recognition result corresponding to the first audio data based on the sequence distance.

[0067] In a specific embodiment, it can be determined whether the sequence distance satisfies a preset condition. If there are multiple second audio data corresponding to the sequence distance satisfying the preset condition, the humming recognition result corresponding to the first audio data is determined according to the second audio data with the smallest sequence distance.

[0068] The preset condition can be a distance threshold. When the sequence distance is less than the distance threshold, it is determined that the preset condition is satisfied, indicating that the similarity between the two sequences is high. When there are multiple second audio data corresponding to the sequence distance satisfying the preset condition, the song information of the second audio data with the smallest distance is taken as the humming recognition result. If there is no second audio data satisfying the condition, the humming recognition result is determined to be unrecognizable.

[0069] As can be seen, the embodiments of the present application first extract the pitches of the first audio data and the second audio data to obtain the first pitch sequence corresponding to the first audio data and the second pitch sequence corresponding to the second audio data. Then, the first pitch sequence and the second pitch sequence are cut according to a preset window length and a preset window shift to obtain a first pitch vector sequence corresponding to the first pitch sequence and a second pitch vector sequence corresponding to the second pitch sequence. The vector distance of the pitch vectors in the first pitch vector sequence and the second pitch vector sequence is determined. Then, the sequence distance of the first pitch vector sequence and the second pitch vector sequence is determined based on the vector distance. Finally, the humming recognition result corresponding to the first audio data is determined based on the sequence distance. That is, in the embodiments of the present application, when performing humming recognition, the pitch sequences corresponding to the first audio data and the second audio data are cut to obtain pitch vector sequences. Then, the vector distance of the pitch vectors in the two pitch vector sequences is determined. Then, the sequence distance of the pitch vector sequences is determined based on the vector distance, and finally the recognition result is obtained. Since the pitch vector includes pitch changes and information of the previous and subsequent pitches, the accuracy of humming recognition can be improved compared to the scheme based on the distance between single pitch points.

[0070] Referring to Figure 4 As shown in the figure, the embodiments of the present application disclose a specific humming recognition method, which comprises:

[0071] Step S21: Extracting the pitch of the first audio data and the second audio data respectively to obtain a first pitch sequence corresponding to the first audio data and a second pitch sequence corresponding to the second audio data.

[0072] Step S22: Splitting the first pitch sequence and the second pitch sequence according to a preset window length and a preset window shift to obtain a first pitch vector sequence corresponding to the first pitch sequence and a second pitch vector sequence corresponding to the second pitch sequence.

[0073] Step S23: Determining a target pitch value corresponding to each pitch vector in the first pitch vector sequence and the second pitch vector sequence.

[0074] Step S24: Subtracting the target pitch value from each element in the vector for each pitch vector.

[0075] In the first implementation, the first element value in each pitch vector can be determined as the target pitch value. In this way, each element in the pitch vector is subtracted by the first element value.

[0076] In the second implementation, the mean value of each element value in each pitch vector can be calculated, and the mean value is determined as the target pitch value. In this way, each element in the pitch vector is subtracted by the mean value of the element values.

[0077] In the third implementation, the previous element value of each element in each pitch vector can be determined as the target pitch value corresponding to the element. In this way, each subsequent element in the pitch vector is subtracted by the previous element value.

[0078] Step S25: Determining the vector distance of the pitch vectors in the first pitch vector sequence and the second pitch vector sequence.

[0079] That is, in the embodiments of the present application, the elements in the pitch vectors in the pitch vector sequence can be subtracted by the target pitch value first to obtain the pitch vectors that can better reflect the pitch change, and then the vector distance of the pitch vectors in the sequence is calculated.

[0080] In specific implementations, the cosine similarity and / or the modulus difference value of the pitch vectors in the first pitch vector sequence and the second pitch vector sequence can be calculated; and the vector distance of the pitch vectors is determined based on the cosine similarity and / or the modulus difference value.

[0081] The vector distance of the pitch vectors can be determined based on a preset weight coefficient, the cosine similarity and the modulus difference value. It should be noted that if the cosine similarity and the modulus difference value are used to determine the vector distance of the pitch vectors, the recognition accuracy can be improved because the angle difference and the modulus difference information are included.

[0082] In another embodiment, the Euclidean distance between the pitch vectors in the first pitch vector sequence and the second pitch vector sequence may be calculated to obtain the vector distance.

[0083] Furthermore, in a specific embodiment, a maximum vector distance may be determined; and based on the maximum vector distance, the vector distances of the pitch vectors in the first pitch vector sequence and the second pitch vector sequence are determined; wherein the vector distances are less than or equal to the maximum vector distance. This prevents some distance values ​​from being too large and affecting matching.

[0084] Step S26: Determine the sequence distance between the first pitch vector sequence and the second pitch vector sequence based on the vector distance.

[0085] In a specific embodiment, the vector distances between the pitch vectors in the first pitch vector sequence and the second pitch vector sequence can be determined to obtain a distance matrix; a shortest distance path can be found in the distance matrix, and a sequence distance between the first pitch vector sequence and the second pitch vector sequence can be determined based on the shortest distance path. The sequence distance is the sum of the distances in the shortest distance path.

[0086] For example, two pitch sequences X1=[x 1,1 ,x 1,2 ,...,x 1,N ] and X2=[x 1,1 ,x 1,2 ,...,x 1,M ] are divided into multiple pitch vectors according to the window length L and window shift K, where x 1,1 ,x 1,2 ,...,x 1,N 、x 1,1 ,x 1,2 ,...,x 1,M , represents the pitch point at each moment, that is, the pitch value at each moment. If vector 1 is [x 1,1 ,x 1,2 ,...,x 1,L ], vector 2 is [x 1,1+K ,x 1,2+K ,...,x 1,L+K ], etc. In this way, the pitch sequence is expressed as a vector. For each pitch vector, each element in the vector is subtracted from the value of the first element. For example, vector 1 is [x 1,1 -x 1,1 ,x 1,2 -x 1,1 ,...,x 1,L -x 1,1 ], vector 2 is [x 1,1+K -x 1,1+K ,x1,2+K -x 1,1+K ,...,x 1,L+K -x 1,1+K ], etc., subtracting the value of the first element can pay more attention to the pitch change and convert the pitch vector into a pitch change vector. In this way, the pitch sequence X1 = [x 1,1 ,x 1,2 ,...,x 1,N ] is converted into a pitch vector sequence X'1=[X' 1,1 ,X' 1,2 ,...,X' 1,i ], where X' 1,1 For vector [x 1,1 -x 1,1 ,x 1,2 -x 1,1 ,...,x 1,L -x 1,1 ],X' 1,2 For vector [x 1,1+K -x 1,1+K ,x 1,2+K -x 1,1+K ,...,x 1,L+K -x 1,1+K ],X' 1,i For vector [x 1,1+(i-1)K -x 1,1+(i-1)K ,x 1,2+(i-1)K -x 1,1+(i-1)K ,...,x 1,L+(i-1)K -x 1,1+(i-1)K ],X' 1,i represents the i-th vector in the pitch vector sequence X'1. For the pitch sequence X2, the corresponding pitch vector sequence X'2 can also be obtained.

[0087] It should be noted that in the prior art, for pitch sequences X1 and X2, the L1-Norm of each pitch point in the sequence is used to calculate the distance to obtain the DTW distance matrix. The embodiment of the present application calculates the DTW distance matrix for the pitch vector sequences X'1 and X'2. This takes into account the similarity of short-term pitch changes and uses the pitch change vector with a window length of K to calculate the distance. Compared with the original solution that directly uses the L1-Norm of the pitch point, the accuracy is greatly improved.

[0088] In one embodiment, the cosine similarity is used to calculate the DTW distance matrix. The i,jth element D in the DTW distance matrix D is i,j Defined as vector X' 1,i With X' 2,j Distance, X' 2,j is the jth vector in X'2. We can use 1 minus cosine similarity, that is

[0089]

[0090] For example Figure 5 , Figure 5 is a vector diagram provided by an embodiment of the present application, including vectors a, b, and the calculation formula of cosine similarity is as follows:

[0091]

[0092] It can be understood that the cosine similarity range is [-1, 1], and the larger the value is, the more similar the two vectors are, so D i,j The range is [0, 2], and the larger the value is, the farther the distance between the two vectors is, and the greater the difference is.

[0093] In another embodiment, the DTW distance matrix can be calculated by using the cosine similarity and the modulus length difference value. In this way, in addition to considering the angle difference between the two vectors, the modulus length difference between the two vectors is additionally added, and the calculation formula of the modulus length difference value is as follows:

[0094]

[0095] Wherein, a is a coefficient, indicating the weight ratio between the angle difference and the modulus length difference, the numerator is the modulus length difference, and the denominator is the normalization. The modulus length difference value is combined with the angle difference to form D i,j .

[0096] Further, D i,j The range is [0, 2], but in actual application, the value of D i,j should not be too large (such as more than 1) to avoid affecting the matching due to the too large distance value, such as making it not more than β, D i,j =min(β,D i,j ).

[0097] Step S27: determining the humming recognition result corresponding to the first audio data based on the sequence distance.

[0098] It can be seen that, after the first pitch sequence and the second pitch sequence are divided according to the preset window length and the preset window shift to obtain the first pitch vector sequence corresponding to the first pitch sequence and the second pitch vector sequence corresponding to the second pitch sequence, the target pitch value corresponding to each pitch vector in the first pitch vector sequence and the second pitch vector sequence is determined, and then for each pitch vector, the elements in the vector are subtracted by the target pitch value. Such operation makes the pitch vector more capable of reflecting the short-time pitch change, and fully utilizes the pitch change to improve the accuracy of the final recognition result.

[0099] Next, taking a certain music APP as an example, the technical scheme of the present application is described.

[0100] Assuming that the user installs the music APP on the user terminal, when the humming recognition is needed, the music APP is opened, and the humming recognition interface is entered, as shown in Figure 6 Figure 6 A specific humming recognition interface provided by the embodiment of the present application is shown. After the start recognition is clicked, the user hums, and after the humming is finished, the stop recognition is clicked. The music APP transmits the obtained user humming audio to the background server of the music APP. The background server extracts the pitch of the user humming audio and the database audio in the database of the music APP, obtains the first pitch sequence corresponding to the user humming audio and the second pitch sequence corresponding to the database audio, then respectively cuts the first pitch sequence and the second pitch sequence according to the preset window length and the preset window shift, obtains the first pitch vector sequence corresponding to the first pitch sequence and the second pitch vector sequence corresponding to the second pitch sequence, then for each pitch vector in the first pitch vector sequence and the second pitch vector sequence, each element is subtracted by the first element value, then the distance matrix of the two sequences is calculated, then the shortest distance path is calculated, the sum of distances on the shortest distance path is determined, the sequence distance is obtained, when the sequence distance is less than the distance threshold, it indicates that the similarity of the two sequences is high, when there are multiple database audios, the sequence distance corresponding to the user humming audio is less than the distance threshold, then the sequence distance is sorted, the song information of the database audio with the smallest distance is determined as the humming recognition result, if there is no database audio, the sequence distance corresponding to the user humming audio is less than the distance threshold, then it is determined that the humming recognition result is unable to be recognized, after the humming recognition result is determined, the humming recognition result is returned to the music APP for display.

[0101] Of course, in some embodiments, the pitch vector sequence can also be generated in advance for the audio in the database, so as to improve the matching speed. Moreover, the present scheme can not only improve the humming recognition scene in which the user does not remember the lyrics and only remembers the melody, but also find the song name in the audio, and can also improve other song recognition scenes, such as the two songs with the same melody but with new lyrics, which can also be recognized and matched; the user's karaoke, song recognition in song library sorting, etc.

[0102] As shown in Figure 7 The embodiment of the present application discloses a humming recognition device, which comprises:

[0103] The pitch extraction module 11 is configured to extract the pitches of the first audio data and the second audio data respectively, and obtain a first pitch sequence corresponding to the first audio data and a second pitch sequence corresponding to the second audio data.

[0104] ​a sequence cutting module 12, configured to cut the first pitch sequence and the second pitch sequence according to a preset window length and a preset window shift, to obtain a first pitch vector sequence corresponding to the first pitch sequence and a second pitch vector sequence corresponding to the second pitch sequence;

[0105] a vector distance determining module 13, configured to determine a vector distance of a pitch vector in the first pitch vector sequence and the second pitch vector sequence;

[0106] a sequence distance determining module 14, configured to determine a sequence distance of the first pitch vector sequence and the second pitch vector sequence based on the vector distance;

[0107] a recognition result determining module 15, configured to determine a humming recognition result corresponding to the first audio data based on the sequence distance.

[0108] It can be seen that, in the embodiments of the present application, the pitch of the first audio data and the second audio data is extracted respectively to obtain a first pitch sequence corresponding to the first audio data and a second pitch sequence corresponding to the second audio data, then the first pitch sequence and the second pitch sequence are cut according to a preset window length and a preset window shift to obtain a first pitch vector sequence corresponding to the first pitch sequence and a second pitch vector sequence corresponding to the second pitch sequence, and a vector distance of a pitch vector in the first pitch vector sequence and the second pitch vector sequence is determined, then a sequence distance of the first pitch vector sequence and the second pitch vector sequence is determined based on the vector distance, and finally a humming recognition result corresponding to the first audio data is determined based on the sequence distance. That is, in the present application, when humming recognition is performed, the pitch sequence corresponding to the first audio data and the second audio data is cut to obtain a pitch vector sequence, then a vector distance of a pitch vector in the two pitch vector sequences is determined, then a sequence distance of the pitch vector sequences is determined based on the vector distance, and finally a recognition result is obtained. Since the pitch vector includes pitch change and front and back pitch information, compared with a scheme of recognizing based on the distance of a single pitch point, the accuracy of humming recognition can be improved.

[0109] The apparatus further includes:

[0110] a target pitch value determining module, configured to, after the sequence cutting module 12 cuts the first pitch sequence and the second pitch sequence according to a preset window length and a preset window shift to obtain a first pitch vector sequence corresponding to the first pitch sequence and a second pitch vector sequence corresponding to the second pitch sequence, determine a target pitch value corresponding to each pitch vector in the first pitch vector sequence and the second pitch vector sequence;

[0111] a sequence processing module configured to, for each pitch vector, subtract the target pitch value from each element in the vector.

[0112] In a first implementation, the target pitch value determining module is configured to determine a first element value in each pitch vector as the target pitch value.

[0113] In a second implementation, the target pitch value determining module is configured to calculate a mean value of each element value in each pitch vector respectively, and determine the mean value as the target pitch value.

[0114] In the second implementation, the target pitch value determining module is configured to determine a previous element value of each element in each pitch vector as the target pitch value corresponding to the element.

[0115] In an implementation, the vector distance determining module 13 specifically comprises:

[0116] a first calculating unit configured to calculate a cosine similarity and / or a modulus difference value of the pitch vectors in the first pitch vector sequence and the second pitch vector sequence;

[0117] a second calculating unit configured to determine a vector distance of the pitch vectors based on the cosine similarity and / or the modulus difference value.

[0118] In another implementation, the vector distance determining module 13 is specifically configured to:

[0119] calculate a Euclidean distance of the pitch vectors in the first pitch vector sequence and the second pitch vector sequence to obtain the vector distance.

[0120] In a specific implementation, the vector distance determining module 13 specifically comprises:

[0121] a maximum vector distance determining unit configured to determine a maximum vector distance;

[0122] a vector distance calculating unit configured to determine a vector distance of the pitch vectors in the first pitch vector sequence and the second pitch vector sequence based on the maximum vector distance, wherein the vector distance is less than or equal to the maximum vector distance.

[0123] Further, in a specific implementation, the vector distance determining module 13 is specifically configured to determine vector distances of the pitch vectors in the first pitch vector sequence and the second pitch vector sequence to obtain a distance matrix.

[0124] Correspondingly, the sequence distance determining module 14 is specifically configured to find a shortest distance path from the distance matrix, and determine a sequence distance of the first pitch vector sequence and the second pitch vector sequence based on the shortest distance path.

[0125] The recognition result determination module 15 specifically comprises:

[0126] A condition determination unit configured to determine whether the sequence distance satisfies a preset condition.

[0127] A recognition result determination unit configured to, if the condition determination unit determines that there are multiple sequence distances corresponding to the second audio data satisfying the preset condition, determine the humming recognition result corresponding to the first audio data according to the second audio data with the smallest sequence distance.

[0128] Further, the embodiment of the present application further provides an electronic device. Figure 8 The electronic device 20 is shown in the structure diagram according to an exemplary embodiment, and the content in the diagram cannot be considered as any limitation on the use range of the present application.

[0129] Figure 8 The electronic device 20 is shown in the structure diagram according to an exemplary embodiment, and the content in the diagram cannot be considered as any limitation on the use range of the present application. The electronic device 20 specifically can comprise at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25 and a communication bus 26. The memory 22 is configured to store a computer program, the computer program is loaded and executed by the processor 21 to realize the related steps in the humming recognition method disclosed in any of the preceding embodiments. In addition, the electronic device 20 in the embodiment can be a server.

[0130] In the embodiment, the power supply 23 is configured to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol followed by the communication interface 24 can be any communication protocol applicable to the technical solution of the present application, which is not limited here; the input / output interface 25 is configured to obtain external input data or output data to the outside, and the specific interface type can be selected according to the specific application needs, which is not limited here.

[0131] In addition, the memory 22 as a carrier for resource storage can be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., and the resources stored thereon can include an operating system 221, a computer program 222 and audio data 223, etc., and the storage mode can be temporary storage or permanent storage.

[0132] The operating system 221 is used to manage and control each hardware device on the electronic device 20 and the computer program 222, so as to realize the operation and processing of the processor 21 on the audio data 223 in the memory 22, which can be Windows Server, Netware, Unix, Linux, etc. In addition to the computer program capable of completing the humming recognition method disclosed in any of the above embodiments, the computer program 222 can further include a computer program capable of completing other specific work.

[0133] Further, the embodiment of the present application further discloses a storage medium, wherein the storage medium stores a computer program, and the computer program is loaded and executed by a processor to realize the humming recognition method steps disclosed in any of the above embodiments.

[0134] Each embodiment in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts of each embodiment can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the related parts can be referred to the method part.

[0135] Finally, it should be noted that in this paper, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitation, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or device including the element.

[0136] The above provides a humming recognition method, device and medium, and the principle and implementation manner of the present application are described by applying specific examples. The above embodiment is only used to help understand the method and its core idea; meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manner and application range can be changed. In conclusion, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A humming recognition method, characterized in that: include: Extracting pitches of the first audio data and the second audio data respectively to obtain a first pitch sequence corresponding to the first audio data and a second pitch sequence corresponding to the second audio data; The first pitch sequence and the second pitch sequence are respectively segmented according to a preset window length and a preset window shift to obtain a first pitch vector sequence corresponding to the first pitch sequence and a second pitch vector sequence corresponding to the second pitch sequence; Determining a vector distance between pitch vectors in the first pitch vector sequence and the second pitch vector sequence; Determine a sequence distance between the first pitch vector sequence and the second pitch vector sequence based on the vector distance; determining a humming recognition result corresponding to the first audio data based on the sequence distance; Wherein, after the first pitch sequence and the second pitch sequence are respectively segmented according to the preset window length and the preset window shift to obtain a first pitch vector sequence corresponding to the first pitch sequence and a second pitch vector sequence corresponding to the second pitch sequence, the method further includes: Determining a target pitch value corresponding to each pitch vector in the first pitch vector sequence and the second pitch vector sequence; For each pitch vector, subtract the target pitch value from the elements in the vector; The target pitch value includes the first element value in each pitch vector, the average of the element values ​​in each pitch vector, or the preceding element value of each element in each pitch vector.

2. The humming recognition method according to claim 1, wherein: Determining the vector distance between the pitch vectors in the first pitch vector sequence and the second pitch vector sequence includes: Calculating cosine similarity and / or modulus difference values ​​of pitch vectors in the first pitch vector sequence and the second pitch vector sequence; The vector distance of the pitch vectors is determined based on the cosine similarity and / or the modulus difference value.

3. The humming recognition method according to claim 1, wherein: Determining the vector distance between the pitch vectors in the first pitch vector sequence and the second pitch vector sequence includes: The Euclidean distance between the pitch vectors in the first pitch vector sequence and the second pitch vector sequence is calculated to obtain the vector distance.

4. The humming recognition method according to claim 1, wherein: Determining the vector distance between the pitch vectors in the first pitch vector sequence and the second pitch vector sequence includes: Determine the maximum vector distance; The vector distance between the pitch vectors in the first pitch vector sequence and the second pitch vector sequence is determined based on the maximum vector distance; wherein the vector distance is less than or equal to the maximum vector distance.

5. The humming recognition method according to claim 1, wherein: Determining the vector distance between the pitch vectors in the first pitch vector sequence and the second pitch vector sequence includes: Determining vector distances between pitch vectors in the first pitch vector sequence and the second pitch vector sequence to obtain a distance matrix; Accordingly, determining the sequence distance between the first pitch vector sequence and the second pitch vector sequence based on the vector distance includes: A shortest distance path is searched from the distance matrix, and a sequence distance between the first pitch vector sequence and the second pitch vector sequence is determined based on the shortest distance path.

6. The humming recognition method according to any one of claims 1 to 5, characterized in that: The determining, based on the sequence distance, a humming recognition result corresponding to the first audio data includes: Determining whether the sequence distance meets a preset condition; If there are multiple second audio data corresponding to which the sequence distances satisfy the preset condition, the humming recognition result corresponding to the first audio data is determined according to the second audio data having the smallest sequence distance.

7. An electronic device, characterized in that: comprising a processor and a memory; wherein, The memory is used to store computer programs; The processor is configured to execute the computer program to implement the humming recognition method according to any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that Used to store a computer program, wherein when the computer program is executed by a processor, the humming recognition method according to any one of claims 1 to 6 is implemented.

Images