Audio matching method, device and equipment

Abstract

The present application discloses an audio matching method, apparatus and device. In the scheme, the first audio fingerprint corresponding to the target position in the first song is first extracted; then, according to the target position, the second audio fingerprint corresponding to the corresponding position in the second song is extracted; and then the consistency ratio and similarity ratio of the first audio fingerprint and the second audio fingerprint are determined. If the consistency ratio is greater than the first threshold and the similarity ratio is greater than the second threshold, it is determined that the first song and the second song are the same song. In this scheme, the audio fingerprints in the two songs are used as matching objects, and the two audio fingerprints are matched using the dual judgment conditions of consistency ratio and similarity ratio, which can improve the accuracy and precision of song matching. Correspondingly, the audio matching device and equipment provided by the present application also have the above-mentioned technical effects.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an audio matching method, device and equipment. Background Art

[0002] At present, id3 (id3Tag) information can be used to determine whether two songs are the same song. ID3 information is a tag that represents an MP3 track and is mostly used in MP3 format song audio files. It is generally located at the beginning or end of a song audio file and stores the singer, title, album name, year, etc. of the song. ID3 information is easily modified by users, which may cause misjudgment. Summary of the invention

[0003] In view of this, the purpose of this application is to provide an audio matching method, device and equipment to improve the accuracy of song matching. The specific scheme is as follows:

[0004] To achieve the above objectives, on the one hand, the present application provides an audio matching method, comprising:

[0005] Extracting a first audio fingerprint corresponding to a target position in a first song;

[0006] According to the target position, extracting a second audio fingerprint corresponding to a corresponding position in a second song; the song tags of the second song and the first song match each other;

[0007] Determine respectively a consistency ratio and a similarity ratio of the first audio fingerprint and the second audio fingerprint;

[0008] If the consistency ratio is greater than a first threshold and the similarity ratio is greater than a second threshold, it is determined that the first song and the second song are the same song.

[0009] On the other hand, the present application also provides an audio matching device, comprising:

[0010] A first extraction module, used to extract a first audio fingerprint corresponding to a target position in a first song;

[0011] A second extraction module is used to extract a second audio fingerprint corresponding to a corresponding position in a second song according to the target position; the song tags of the second song and the first song match each other;

[0012] A first determination module, configured to respectively determine a consistency ratio and a similarity ratio of the first audio fingerprint and the second audio fingerprint;

[0013] The second determination module is used to determine that the first song and the second song are the same song if the consistency ratio is greater than a first threshold and the similarity ratio is greater than a second threshold.

[0014] On the other hand, the present application also provides an electronic device, comprising a processor and a memory; wherein the memory is used to store a computer program, and the computer program is loaded and executed by the processor to implement the aforementioned audio matching method.

[0015] This application first extracts the first audio fingerprint corresponding to the target position in the first song; then extracts the second audio fingerprint corresponding to the corresponding position in the second song according to the target position; and then determines the consistency ratio and similarity ratio of the first audio fingerprint and the second audio fingerprint respectively; if the consistency ratio is greater than the first threshold and the similarity ratio is greater than the second threshold, it is determined that the first song and the second song are the same song. In this scheme, the audio fingerprints in the two songs are used as matching objects, and the two audio fingerprints are matched using the dual judgment conditions of consistency ratio and similarity ratio, which can improve the accuracy and precision of song matching and avoid misjudgment using id3 information.

[0016] Correspondingly, the audio matching device and equipment provided by this application also have the above-mentioned technical effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0018] Figure 1 A schematic diagram of a physical architecture applicable to the present application provided for the present application;

[0019] Figure 2 A flow chart of the first audio matching method provided in this application;

[0020] Figure 3 A spectrogram and a schematic diagram of peak points therein provided for this application;

[0021] Figure 4 A schematic diagram of a set of adjacent peak points provided for this application;

[0022] Figure 5 A flow chart of the second audio matching method provided in this application;

[0023] Figure 6A flow chart of the third audio matching method provided in this application;

[0024] Figure 7 A schematic diagram of an audio matching device provided in this application;

[0025] Figure 8 A server structure diagram provided for this application;

[0026] Fig. 9 A terminal structure diagram provided for this application. DETAILED DESCRIPTION

[0027] The ID3 information of songs can be easily modified by users, so using the ID3 information to determine whether two songs are the same song may cause misjudgment. In view of the above problems currently existing, the present application proposes an audio matching solution, which can improve the accuracy and precision of song matching.

[0028] For ease of understanding, the physical framework to which this application applies is first introduced.

[0029] Figure 1 A physical framework applicable to the present application is shown. The physical framework includes: a terminal and a server. The terminal runs a platform that supports running audio, and the platform can be any one of an audio player program or applet (a program that relies on a host program to run), an audio player web page, a video player program or applet, and a video player web page. The terminal is connected to the server via a wireless network or a wired network.

[0030] Terminal refers to one or more terminals. For example, the terminal may be only one, or may be more than ten or hundreds, or a greater number. The present application embodiment only uses the terminal as an example for explanation. The types of terminals include: at least one of a smart phone, a tablet computer, an e-book reader, an MP3 player, an MP4 player, a laptop computer, and a desktop computer. The present application embodiment does not limit the number and type of terminals.

[0031] The server includes at least one of a single server, multiple servers, a cloud computing platform, and a virtualization center. The server is provided with a song library and an audio matching method provided by the present application. If a user transmits a song to the server through a platform on a terminal, the server can match the song with any song in the song library, thereby determining whether the song uploaded by the user is already in the song library.

[0032] Specifically, the audio matching method provided in the embodiment of the present application can be applied to song playback or recommendation scenarios. In this scenario, the server uses the audio matching method provided in the present application to match the songs uploaded by the user and any song in the song library, that is: extract the first audio fingerprint corresponding to the target position in the first song (that is, the song uploaded by the user); then, according to the target position, extract the second audio fingerprint corresponding to the corresponding position in the second song (that is, any song in the song library); then determine the consistency ratio of the first audio fingerprint and the second audio fingerprint; if the consistency ratio is greater than the first threshold, further determine the similarity ratio of the first audio fingerprint and the second audio fingerprint; if the similarity ratio is greater than the second threshold, determine that the first song and the second song are the same song, and the song consistent with the song uploaded by the user can be found.

[0033] Of course, the above application scenarios are only for illustrative purposes. The method provided in the embodiment of the present application can be applied to other scenarios that require audio matching (such as audio retrieval, audio comparison, etc.), and this embodiment does not limit this.

[0034] Combined with the above commonalities, please see Figure 2 , Figure 2 The first audio matching method flow chart provided in the embodiment of the present application. Figure 2 As shown, the audio matching method may include the following steps:

[0035] S201. Extract a first audio fingerprint corresponding to a target position in a first song.

[0036] S202. Extracting a second audio fingerprint corresponding to a corresponding position in the second song according to the target position.

[0037] The song tags of the second song and the first song match each other. For example, the first song and the second song can be two songs matched using id3 information (i.e., song tags). That is, based on the first song, the id3 information is first used to match the second song, and then S201-S205 are further performed to verify the matching result of the id3 information.

[0038] The target position is generally represented by the playing time of the song. Each song has a certain playing time, so the playing time period of the song (such as the 10th to 20th second) can be determined as the target position. Based on this, the audio fingerprints corresponding to the 10th to 20th second of the first song and the second song can be extracted and processed for subsequent matching.

[0039] In a specific embodiment, extracting the first audio fingerprint corresponding to the target position in the first song includes: extracting the audio data corresponding to the target position and converting the audio data into a spectrogram; determining each peak point in the spectrogram, and selecting a set of adjacent peak points from each peak point, and encoding the set of adjacent peak points as the first audio fingerprint. Wherein, selecting a set of adjacent peak points from each peak point includes: taking any peak point in the spectrogram as the center of a circle, and determining a coverage range with a preset distance threshold as the radius, and combining all peak points within the coverage range whose time points are greater than the time points of the center of the circle into the set of adjacent peak points. Wherein, the set of adjacent peak points only includes: peak points within a certain range and whose time points are greater than the time points of the center of the circle (that is, peak points behind the center of the circle), so that repeated sub-fingerprints can be avoided.

[0040] Of course, other strategies may be used to select adjacent peak point sets. Encoding the adjacent peak point sets as the first audio fingerprint includes: encoding the adjacent peak point sets as the first audio fingerprint using a hash algorithm to reduce the possibility of fingerprint collision.

[0041] The peak points in the spectrogram can be found in Figure 3 . Figure 3 The horizontal axis is time, and the vertical axis is frequency. Frequency can be understood as the pitch heard by the human ear. The peak points in the spectrogram represent the most representative frequency value of a song at each moment. Each peak point corresponds to a mark (f, t) composed of frequency and time, which can be understood as a coordinate. Each peak point and its n neighboring peak points (such as Figure 4 ) corresponding to the tags can form a set of adjacent peak points, which can then be encoded into an audio fingerprint in the form of hash coding. The specific process of extracting the second audio fingerprint from the second song can refer to the above introduction.

[0042] Specifically, if the peak point as the center of the circle is represented as (f0, t0), and its n adjacent peak point sets are represented as (f1, t1), (f2, t2), …, (fn, tn), then (f0, t0) is combined with each of its adjacent peak points to obtain pairs of combined information, such as (f0, f1, t1-t0), (f0, f2, t2-t0), …, (f0, fn, tn-t0), and then each pair of combined information can be encoded into a sub-fingerprint in the form of hash coding.

[0043] S203: Determine a consistency ratio between the first audio fingerprint and the second audio fingerprint.

[0044] In a specific implementation, determining the consistency ratio of the first audio fingerprint and the second audio fingerprint includes: arranging the sub-fingerprints included in the first audio fingerprint and the second audio fingerprint in chronological order to obtain a first sequence and a second sequence; determining a matching position, the matching position being the position of the sub-fingerprint in the first sequence, or the position of the sub-fingerprint in the second sequence; if there is no vacancy in the two sub-fingerprints corresponding to the matching position, and the two sub-fingerprints corresponding to the matching position are the same, determining the matching result of the matching position as a first weight value; otherwise, determining the matching result of the matching position as a second weight value; the first weight value> the second weight value; and calculating the consistency ratio according to the matching results of each matching position. The first weight value and the second weight value can be flexibly selected.

[0045] The first audio fingerprint and the second audio fingerprint each include a plurality of sub-fingerprints, such as c, i, h, k, and t in the fingerprint B in the following example.

[0046] Specifically, the consistency ratio refers to: the ratio of information that exists in both the first audio fingerprint and the second audio fingerprint to the total amount of information in the first audio fingerprint or the second audio fingerprint.

[0047] The process of determining the consistency ratio can be referred to as follows: Assuming that fingerprint A is arranged in chronological order as cvhkabt, and fingerprint B is arranged in chronological order as cihkt, then fingerprint A and fingerprint B are compared bit by bit, see Table 1, where "-" in Table 1 indicates a blank.

[0048] Table 1

[0049]

[0050]

[0051] As can be seen from Table 1, in fingerprint A and fingerprint B, the fingerprints at matching positions 1, 3, 4, and 7 are the same, and the fingerprints at matching positions 2, 5, and 6 are different. That is, the information that exists in fingerprint A and fingerprint B is: chkt. If the fingerprints are the same, the weight value is 1 (i.e., the first weight value), and the fingerprints are different. The weight value is 0 (i.e., the second weight value), then the matching results of fingerprint A and fingerprint B bit by bit are shown in Table 2.

[0052] Table 2

[0053] 1 2 3 4 5 6 7 Fingerprint A c v h k a b t Fingerprint B c i h k - - t Matching results 1 0 1 1 0 0 1

[0054] According to Table 2, consistency ratio = (1+0+1+1+0+0+1) / 7 = 4 / 7. 7 is the ideal value when fingerprint A and fingerprint B are completely consistent, that is, the sum of the weight values ​​when the sub-fingerprints at the 7 matching positions correspond to the same.

[0055] It should be noted that, in the process of comparing two fingerprints, the two fingerprints can be aligned using the NW (Needleman-Wunsch) algorithm or the LCS algorithm so that the common information (such as chkt) included in the two fingerprints is aligned bit by bit. As shown in Table 1, two vacancies are inserted before the "t" in fingerprint B so that the "t" in fingerprint A is aligned with the "t" in fingerprint B, the "k" in fingerprint A is aligned with the "k" in fingerprint B, the "h" in fingerprint A is aligned with the "h" in fingerprint B, and the "c" in fingerprint A is aligned with the "c" in fingerprint B.

[0056] In a specific implementation, before determining the matching position, the method further includes: determining a common sub-fingerprint (such as chkt) in the first sequence and the second sequence, and aligning the common sub-fingerprint bit by bit.

[0057] In a specific implementation, the consistency ratio may also be calculated using the Longest Common Subsequence (LCS) algorithm, and the calculated longest common subsequence is “chkt” in the above example.

[0058] It should be noted that the LCS algorithm has limited matching accuracy for some songs with strong rhythms, such as heavy metal. This is because songs with strong rhythms have strong energy distribution in each frequency band, which increases the number of peak points extracted and calculates more audio fingerprints. Therefore, due to the dense fingerprints and rich fingerprint information, it is easy to have the same fingerprints as other songs, so it is easy to misjudge the same song when comparing the longest common subsequence. The NW algorithm has higher matching accuracy for some songs with strong rhythms, such as heavy metal, because it can better discover the relationship between audio fingerprints and detect gaps in audio fingerprints.

[0059] S204: If the consistency ratio is greater than the first threshold, determine the similarity ratio between the first audio fingerprint and the second audio fingerprint.

[0060] In this embodiment, the consistency ratio and the similarity ratio can be determined in any order. Whether the consistency ratio is determined first or the similarity ratio is determined first, it will not affect the implementation of this embodiment.

[0061] In a specific implementation, determining the similarity ratio between the first audio fingerprint and the second audio fingerprint includes: arranging the sub-fingerprints included in the first audio fingerprint and the second audio fingerprint in chronological order to obtain a first sequence and a second sequence; determining a matching position, where the matching position is the position of the sub-fingerprint in the first sequence or the position of the sub-fingerprint in the second sequence; determining a weight value corresponding to the matching position according to the above judgment result, wherein the weight value corresponding to the two sub-fingerprints corresponding to the matching position when there is no vacancy is greater than the weight value corresponding to the vacancy; and calculating the similarity ratio according to the weight values ​​corresponding to each matching position.

[0062] Among them, if there is no vacancy in the two sub-fingerprints corresponding to the matching position, and the two sub-fingerprints corresponding to the matching position are the same, then the matching result of the matching position is determined to be the third weight value; if there is no vacancy in the two sub-fingerprints corresponding to the matching position, and the two sub-fingerprints corresponding to the matching position are different, then the matching result of the matching position is determined to be the fourth weight value; if there is a vacancy in the two sub-fingerprints corresponding to the matching position, then the matching result of the matching position is determined to be the fifth weight value; the third weight value> the fourth weight value> the fifth weight value. The third weight value, the fourth weight value, and the fifth weight value can be flexibly selected. The third weight value can be the same as the first weight value, and the fourth weight value can be the same as the second weight value.

[0063] Specifically, the similarity ratio can be understood as: the similarity between the first audio fingerprint and the second audio fingerprint.

[0064] The process of determining the similarity ratio can be referred to as follows: Assuming that fingerprint A is arranged in chronological order as cvhkabt, and fingerprint B is arranged in chronological order as cihkt, then fingerprint A and fingerprint B are compared bit by bit. If the fingerprints at the same matching position are the same, a weight value of 3 (i.e., the third weight value) is assigned; if the fingerprints at the same matching position are different, a weight value of 1 (i.e., the fourth weight value) is assigned; if there is a vacancy in the fingerprint at the same matching position, a weight value of -1 (i.e., the fifth weight value) is assigned. The matching results of the bit-by-bit comparison of fingerprint A and fingerprint B can be found in Table 3, where "-" in Table 3 indicates a vacancy.

[0065] Table 3

[0066] 1 2 3 4 5 6 7 Fingerprint A c v h k a b t Fingerprint B c i h k - - t Matching results 3 1 3 3 -1 -1 3

[0067] According to Table 3, similarity ratio = (3+1+3+3-1-1+3) / (3×7) = 11 / 21. 3×7 is the ideal value for fingerprint A and fingerprint B to be completely consistent, that is, the sum of the weight values ​​when the sub-fingerprints at the 7 matching positions are all the same.

[0068] When calculating the similarity ratio, three different weight values ​​are used to represent three matching results, that is, the two situations of vacancies (the situation corresponding to matching positions 5 and 6) and two different fingerprints (the situation corresponding to matching position 2) are distinguished. This method can avoid mismatching of music with strong rhythm. Because the consistency of the common information (i.e. the longest common subsequence) can be screened, but there are many intervals in the longest common subsequence (i.e. "t" in fingerprint A in the above example is separated by "a, b"), the similarity in this case is generally low.

[0069] S205: If the similarity ratio is greater than a second threshold, it is determined that the first song and the second song are the same song.

[0070] It can be seen that this embodiment uses the audio fingerprints in the two songs as the matching objects, and uses the dual judgment conditions of consistency ratio and similarity ratio to match the two audio fingerprints, which can improve the accuracy and precision of song matching and avoid misjudgment caused by using id3 information.

[0071] See also Figure 5 , Figure 5 The second audio matching method flow chart provided in the embodiment of the present application. Figure 5 As shown, the audio matching method may include the following steps:

[0072] S501, extracting a first audio fingerprint corresponding to a target position in a first song;

[0073] S502, extracting a second audio fingerprint corresponding to a corresponding position in the second song according to the target position;

[0074] S503: Determine the consistency ratio between the first audio fingerprint and the second audio fingerprint;

[0075] S504, determine whether the consistency ratio is greater than a first threshold; if so, execute S505; if not, execute S508;

[0076] S505: Determine a similarity ratio between the first audio fingerprint and the second audio fingerprint;

[0077] S506, determining whether the similarity ratio is greater than a second threshold; if so, executing S507; if not, executing S508;

[0078] S507, determining that the first song and the second song are the same song;

[0079] S508: Determine that the first song and the second song are not the same song.

[0080] The first threshold and the second threshold may be the same or different, and the sizes of the two are not specifically limited in this embodiment.

[0081] In this embodiment, steps S503, S504 and S508 can be implemented using the NW algorithm or the LCS algorithm, and steps S505-S508 can be implemented using the NW algorithm. For details, please refer to the relevant introduction of the above embodiment. The other steps in this embodiment can also refer to the relevant introduction of the above embodiment, and this embodiment will not be repeated here. Among them, the NW algorithm includes two judgment steps, namely, the similarity ratio and the consistency ratio. The LCS algorithm only includes the judgment step of the consistency ratio.

[0082] The execution order of the two judgment steps of judging the consistency ratio and judging the similarity ratio can be interchanged, and the interchange does not affect the implementation of this application.

[0083] It can be seen that this embodiment uses the audio fingerprints in the two songs as the matching objects, and uses the dual judgment conditions of consistency ratio and similarity ratio to match the two audio fingerprints, which can improve the accuracy and precision of song matching and avoid misjudgment caused by using id3 information.

[0084] The following describes the audio matching solution provided by this application through a specific application scenario example. Figure 6 , Figure 6 The third audio matching method flow chart provided in the embodiment of the present application. Figure 6 As shown, the audio matching method may include the following steps:

[0085] (1) The user uploads a local song to the server, and the server uses the ID3 information to match the song with the song library and finds a matching song from the song library;

[0086] (2) Extract 30s to 40s of audio from the local song and the matching song, and calculate fingerprints for each, obtaining two fingerprints;

[0087] (3) Use the Needleman-Wunsch algorithm to match the two fingerprints in step (2) to determine whether the local song and the matching song are the same song. If they are the same song, the corresponding ID3 information is displayed; if they are not the same song, other processing operations are performed on the local song (for example, fingerprint retrieval and matching in the music library, etc.) to improve the user's physical examination.

[0088] In this embodiment, the specific implementation of steps (2) and (3) can refer to the relevant introduction of any of the above embodiments. This embodiment has a better matching effect for songs with strong rhythms. Because the NW algorithm can better discover the relationship between audio fingerprints (match, mismatch, gap, etc.), detect the gap in the audio fingerprint, thereby improving the accuracy of song matching.

[0089] See also Figure 7 , Figure 7 A schematic diagram of an audio matching device provided in an embodiment of the present application includes:

[0090] A first extraction module 701 is used to extract a first audio fingerprint corresponding to a target position in a first song;

[0091] The second extraction module 702 is used to extract the second audio fingerprint corresponding to the corresponding position in the second song according to the target position;

[0092] A first determination module 703, configured to respectively determine a consistency ratio and a similarity ratio of the first audio fingerprint and the second audio fingerprint;

[0093] The second determination module 704 is used to determine that the first song and the second song are the same song if the consistency ratio is greater than a first threshold and the similarity ratio is greater than a second threshold.

[0094] In a specific embodiment, the first extraction module includes:

[0095] An extraction unit, used to extract audio data corresponding to a target position and convert the audio data into a spectrogram;

[0096] The encoding unit is used to determine each peak point in the spectrogram, select a set of adjacent peak points from each peak point, and encode the set of adjacent peak points into a first audio fingerprint.

[0097] In a specific implementation, the encoding unit is specifically used for:

[0098] Taking any peak point in the spectrogram as the center of the circle and a preset distance threshold as the radius to determine the coverage range, all peak points within the coverage range whose time points are greater than the time point of the center of the circle are combined into the adjacent peak point set.

[0099] In a specific implementation, the encoding unit is specifically used for:

[0100] A hash algorithm is used to encode the neighboring peak point set into a first audio fingerprint.

[0101] In a specific implementation, the first determining module includes:

[0102] A first arrangement unit, configured to arrange the sub-fingerprints included in the first audio fingerprint and the second audio fingerprint in chronological order to obtain a first sequence and a second sequence;

[0103] A first determining unit, configured to determine a matching position, where the matching position is a position of the sub-fingerprint in the first sequence, or a position of the sub-fingerprint in the second sequence;

[0104] a matching unit, configured to determine that a matching result of the matching position is a first weight value if there is no gap in the two sub-fingerprints corresponding to the matching position and the two sub-fingerprints corresponding to the matching position are the same; otherwise, determine that a matching result of the matching position is a second weight value; and the first weight value is greater than the second weight value;

[0105] The first calculation unit is used to calculate the consistency ratio according to the matching results of each matching position.

[0106] In a specific implementation, the first determining module includes:

[0107] A second arrangement unit, configured to arrange the sub-fingerprints included in the first audio fingerprint and the second audio fingerprint in chronological order to obtain a first sequence and a second sequence;

[0108] A second determining unit is used to determine a matching position, where the matching position is a position of the sub-fingerprint in the first sequence or a position of the sub-fingerprint in the second sequence;

[0109] A judging unit, used to judge whether there is a gap in the two sub-fingerprints corresponding to the matching position;

[0110] A third determining unit is used to determine the weight value corresponding to the matching position according to the above judgment result, wherein the weight value corresponding to the two sub-fingerprints corresponding to the matching position when there is no gap is greater than the weight value corresponding to the gap;

[0111] The second calculation unit is used to calculate the similarity ratio according to the matching results of each matching position.

[0112] In a specific embodiment,

[0113] If the two sub-fingerprints corresponding to the matching position are the same, determining the matching result of the matching position to be a third weight value;

[0114] If the two sub-fingerprints corresponding to the matching position are different, the matching result of the matching position is determined to be a fourth weight value.

[0115] If there is a gap between the two sub-fingerprints corresponding to the matching position, the matching result of the matching position is determined to be the fifth weight value; the third weight value> the fourth weight value> the fifth weight value.

[0116] In a specific implementation, the first determining module further includes:

[0117] An alignment unit is used to determine common sub-fingerprints in the first sequence and the second sequence, and align the common sub-fingerprints bit by bit.

[0118] In a specific embodiment, it also includes:

[0119] The output module is used to determine that the first song and the second song are not the same song if the consistency ratio is not greater than a first threshold value and / or the similarity ratio is not greater than a second threshold value.

[0120] Among them, for more specific working processes of each module and unit in this embodiment, reference can be made to the corresponding contents disclosed in the aforementioned embodiments, which will not be repeated here.

[0121] It can be seen that this embodiment provides an audio matching device, which takes the audio fingerprints in two songs as matching objects, and uses the dual judgment conditions of consistency ratio and similarity ratio to match the two audio fingerprints, which can improve the accuracy and precision of song matching, and can also avoid misjudgment caused by using id3 information.

[0122] Furthermore, the present application also provides an electronic device. The electronic device can be Figure 8 The server 50 shown may also be Fig. 9 Terminal 60 is shown. Figure 8 and Fig. 9 All of them are structural diagrams of electronic devices according to an exemplary embodiment, and the contents in the diagrams cannot be regarded as any limitation on the scope of use of the present application.

[0123] Figure 8 A schematic diagram of the structure of a server provided in an embodiment of the present application. The server 50 may specifically include: at least one processor 51, at least one memory 52, a power supply 53, a communication interface 54, an input / output interface 55, and a communication bus 56. The memory 52 is used to store a computer program, which is loaded and executed by the processor 51 to implement the relevant steps in the audio matching disclosed in any of the aforementioned embodiments.

[0124] In this embodiment, the power supply 53 is used to provide working voltage for each hardware device on the server 50; the communication interface 54 can create a data transmission channel between the server 50 and external devices, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 55 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0125] In addition, the memory 52, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon include an operating system 521, a computer program 522 and data 523, etc. The storage method can be temporary storage or permanent storage.

[0126] The operating system 521 is used to manage and control the hardware devices and computer programs 522 on the server 50, so as to realize the operation and processing of the data 523 in the memory 52 by the processor 51, and it can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the audio matching method disclosed in any of the aforementioned embodiments, the computer program 522 can further include a computer program that can be used to complete other specific tasks. In addition to data such as song audio, the data 523 can also include data such as developer information of the application.

[0127] Fig. 9 This is a schematic diagram of the structure of a terminal provided in an embodiment of the present application. The terminal 60 may specifically include but is not limited to a smart phone, a tablet computer, a laptop computer or a desktop computer.

[0128] Generally, the terminal 60 in this embodiment includes: a processor 61 and a memory 62 .

[0129] Among them, the processor 61 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 61 can be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 61 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 61 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 61 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0130] The memory 62 may include one or more computer-readable storage media, which may be non-transitory. The memory 62 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 62 is at least used to store the following computer program 621, wherein, after the computer program is loaded and executed by the processor 61, it can implement the relevant steps in the audio matching method performed by the terminal side disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 62 may also include an operating system 622 and data 623, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 622 may include Windows, Unix, Linux, etc. Data 623 may include but is not limited to song audio information.

[0131] In some embodiments, the terminal 60 may further include a display screen 63 , an input / output interface 64 , a communication interface 65 , a sensor 66 , a power supply 67 , and a communication bus 68 .

[0132] Those skilled in the art will understand that Fig. 9 The structure shown in the figure does not constitute a limitation on the terminal 60, and the terminal 60 may include more or less components than those shown in the figure.

[0133] It should be pointed out that the above is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

[0134] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0135] Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An audio matching method, characterized in that: include: Extracting a first audio fingerprint corresponding to a target position in a first song; According to the target position, extracting a second audio fingerprint corresponding to a corresponding position in the second song; The song tags of the second song and the first song match each other; Determine respectively the consistency ratio and the similarity ratio of the first audio fingerprint and the second audio fingerprint; the consistency ratio is: the ratio of information commonly present in the first audio fingerprint and the second audio fingerprint to the total amount of information of the first audio fingerprint or the second audio fingerprint; the similarity ratio is: the similarity between the first audio fingerprint and the second audio fingerprint; If the consistency ratio is greater than a first threshold and the similarity ratio is greater than a second threshold, it is determined that the first song and the second song are the same song; The determining of the consistency ratio between the first audio fingerprint and the second audio fingerprint includes: Arranging the sub-fingerprints included in the first audio fingerprint and the second audio fingerprint in chronological order to obtain a first sequence and a second sequence; Determine a matching position, where the matching position is a position of the sub-fingerprint in the first sequence, or a position of the sub-fingerprint in the second sequence; If there is no gap in the two sub-fingerprints corresponding to the matching position, and the two sub-fingerprints corresponding to the matching position are the same, then the matching result of the matching position is determined to be the first weight value; otherwise, the matching result of the matching position is determined to be the second weight value; the first weight value> the second weight value; The consistency ratio is calculated according to the matching results of each matching position.

2. The audio matching method according to claim 1, characterized in that: The extracting a first audio fingerprint corresponding to a target position in the first song includes: Extracting audio data corresponding to the target position and converting the audio data into a spectrogram; Each peak point in the spectrogram is determined, and a set of adjacent peak points is selected from the peak points, and the set of adjacent peak points is encoded as the first audio fingerprint.

3. The audio matching method according to claim 2, characterized in that: The step of selecting a set of adjacent peak points from the peak points includes: Taking any peak point in the spectrogram as the center of the circle and a preset distance threshold as the radius to determine the coverage range, all peak points within the coverage range whose time points are greater than the time point of the center of the circle are combined into the adjacent peak point set.

4. The audio matching method according to claim 2, characterized in that: The step of encoding the adjacent peak point set into the first audio fingerprint comprises: The neighboring peak point set is encoded into the first audio fingerprint by using a hash algorithm.

5. The audio matching method according to claim 1, characterized in that: The determining the similarity ratio between the first audio fingerprint and the second audio fingerprint includes: Arranging the sub-fingerprints included in the first audio fingerprint and the second audio fingerprint in chronological order to obtain a first sequence and a second sequence; Determine a matching position, where the matching position is a position of the sub-fingerprint in the first sequence, or a position of the sub-fingerprint in the second sequence; Determine whether there is a gap in the two sub-fingerprints corresponding to the matching position; Determine the weight value corresponding to the matching position according to the above judgment result, wherein the weight value corresponding to the two sub-fingerprints corresponding to the matching position when there is no gap is greater than the weight value corresponding to the gap; The similarity ratio is calculated according to the weight value corresponding to each matching position.

6. The audio matching method according to claim 5, characterized in that: Also includes: If there is no gap in the two sub-fingerprints corresponding to the matching position, and the two sub-fingerprints corresponding to the matching position are the same, determining that the matching result of the matching position is a third weight value; If there is no gap in the two sub-fingerprints corresponding to the matching position, and the two sub-fingerprints corresponding to the matching position are different, determining that the matching result of the matching position is a fourth weight value; If there is a gap in the two sub-fingerprints corresponding to the matching position, the matching result of the matching position is determined to be the fifth weight value; the third weight value>the fourth weight value>the fifth weight value.

7. The audio matching method according to claim 1 or 5, characterized in that: Before determining the matching position, the method further includes: Common sub-fingerprints in the first sequence and the second sequence are determined, and the common sub-fingerprints are aligned bit by bit.

8. The audio matching method according to any one of claims 1 to 6, characterized in that: Also includes: If the consistency ratio is not greater than the first threshold, and / or the similarity ratio is not greater than the second threshold, it is determined that the first song and the second song are not the same song.

9. An audio matching device, characterized in that: include: A first extraction module, used to extract a first audio fingerprint corresponding to a target position in a first song; A second extraction module, configured to extract a second audio fingerprint corresponding to a corresponding position in a second song according to the target position; The song tags of the second song and the first song match each other; A first determination module is configured to respectively determine a consistency ratio and a similarity ratio of the first audio fingerprint and the second audio fingerprint; the consistency ratio is: a ratio of information commonly present in the first audio fingerprint and the second audio fingerprint to a total amount of information in the first audio fingerprint or the second audio fingerprint; The similarity ratio is: the similarity between the first audio fingerprint and the second audio fingerprint; A second determination module, configured to determine that the first song and the second song are the same song if the consistency ratio is greater than a first threshold and the similarity ratio is greater than a second threshold; Wherein, the first determining module is specifically used for: Arranging the sub-fingerprints included in the first audio fingerprint and the second audio fingerprint in chronological order to obtain a first sequence and a second sequence; Determine a matching position, where the matching position is a position of the sub-fingerprint in the first sequence, or a position of the sub-fingerprint in the second sequence; If there is no gap in the two sub-fingerprints corresponding to the matching position, and the two sub-fingerprints corresponding to the matching position are the same, then the matching result of the matching position is determined to be the first weight value; otherwise, the matching result of the matching position is determined to be the second weight value; the first weight value> the second weight value; The consistency ratio is calculated according to the matching results of each matching position.

10. An electronic device, characterized in that: The electronic device comprises a processor and a memory; wherein the memory is used to store a computer program, and the computer program is loaded and executed by the processor to implement the audio matching method according to any one of claims 1 to 8.

Images