A cover relationship identification method, device and equipment and storage medium

Abstract

The application discloses a cover relationship identification method and device, equipment and a storage medium, and relates to the technical field of cover identification, which comprises the following steps: dividing a target reference audio into a plurality of reference segments, and dividing a target to-be-detected audio into a plurality of initial candidate segments based on a preset singing situation; for any reference segment, selecting a plurality of initial matching segments from the initial candidate segments according to the similarity between the reference segment and each initial candidate segment; selecting a plurality of target matching segments from the initial matching segments based on a preset constraint rule, and generating an evidence chain according to each target matching segment; determining a reference audio coverage degree and a to-be-detected audio coverage degree according to each target matching segment in the evidence chain; and identifying the cover relationship between the target to-be-detected audio and the target reference audio according to the similarity, the reference audio coverage degree and the to-be-detected audio coverage degree. The application can accurately identify multiple types of covers, reduce the misjudgment and missed detection rate, and output interpretable evidence.

Description

Technical Field

[0001] This invention relates to the field of cover song recognition technology, and in particular to a method, apparatus, device, and storage medium for recognizing cover song relationships. Background Technology

[0002] With the development of short video platforms, live streaming platforms, music platforms, and AI (Artificial Intelligence) audio generation technology, users may create covers of existing songs using real people, AI-generated voices, pitch shifting, speed adjustment, different accompaniments, or simply sing the chorus or a snippet. Such content typically retains the main melody or parts of the original song, but changes are made to timbre, tempo, tonality, accompaniment, singing style, and audio structure.

[0003] Traditional audio fingerprinting technology is better suited for identifying audio clips from the same recording version, those with accompanying accompaniment, or those that have undergone slight compression, editing, or speed changes. For cover songs, the original waveform and spectral details may differ significantly, making it difficult to cover real cover song scenarios using only audio fingerprinting.

[0004] Meanwhile, most existing melody matching and neural network recognition solutions only output simple judgment results and cannot provide traceable fragment-level matching evidence, time information and quantitative indicators, which makes it difficult to meet the actual needs of manual review and evidence tracing in copyright review scenarios.

[0005] In conclusion, how to accurately identify multiple types of cover songs, reduce the false positive and false negative rates, and output interpretable evidence are the technical problems that urgently need to be solved. Summary of the Invention

[0006] In view of this, the purpose of this invention is to provide a method, apparatus, device, and storage medium for identifying cover song relationships, capable of accurately identifying multiple types of cover songs, reducing false positives and false negatives, and outputting interpretable evidence. The specific solution is as follows: Firstly, this application provides a method for identifying cover song relationships, including: The target reference audio is divided into several reference segments, and the target audio to be detected is divided into several initial candidate segments based on preset singing conditions; the preset singing conditions include preset fast singing, preset normal speed singing, and preset slow singing. For any of the reference segments, several initial matching segments are selected from each of the initial candidate segments based on the similarity between the reference segment and each of the initial candidate segments; Based on preset constraint rules, several target matching segments are selected from each of the initial matching segments, and an evidence chain is generated based on each of the target matching segments; The reference audio coverage and the detection audio coverage are determined based on each target matching segment in the chain of evidence; the reference audio coverage is the ratio of the combined coverage duration of the target matching segment on the target reference audio to the total duration of the target reference audio, and the detection audio coverage is the ratio of the combined coverage duration of the target matching segment on the target detection audio to the total duration of the target detection audio. The cover song relationship between the target audio to be detected and the target reference audio is identified based on the similarity, the reference audio coverage, and the audio coverage to be detected.

[0007] Optionally, the method for identifying cover song relationships further includes: The fundamental frequency of the target reference audio and the target audio to be detected are estimated respectively to obtain a first fundamental frequency sequence of the target reference audio and a second fundamental frequency sequence of the target audio to be detected; The first fundamental frequency sequence is converted into a first pitch sequence, and the second fundamental frequency sequence is converted into a second pitch sequence; Extract the first harmonic features of the target reference audio and the second harmonic features of the target audio to be detected; the harmonic features are features that characterize the harmonic structure of the audio. A first melody skeleton of the target reference audio is generated based on the first pitch sequence, and a second melody skeleton of the target audio to be detected is generated based on the second pitch sequence, so as to determine the similarity between the reference segment and each of the initial candidate segments based on the first harmonic features of the target reference audio, the first melody skeleton, the second harmonic features of the target audio to be detected, and the second melody skeleton.

[0008] Optionally, the melody skeleton includes a trend skeleton and an interval skeleton; Accordingly, generating a first melody skeleton of the target reference audio based on the first pitch sequence, and generating a second melody skeleton of the target audio to be detected based on the second pitch sequence, includes: A first target note node is generated based on the first pitch sequence, and a second target note node is generated based on the second pitch sequence; the target note node is a note node corresponding to consecutive frames with consistent pitch values. A first trend skeleton and a first interval skeleton of the target reference audio are generated based on the first target note node, and a second trend skeleton and a second interval skeleton of the target audio to be detected are generated based on the second target note node. The trend skeleton is used to describe the direction of pitch change between adjacent target note nodes, and the interval skeleton is used to describe the amplitude of pitch change between adjacent target note nodes.

[0009] Optionally, the step of dividing the target audio to be detected into several initial candidate segments based on a preset singing scenario includes: Determine the appropriate speed ratio based on the preset singing conditions; For any of the reference segments, a reference duration is determined, and a corresponding candidate segment duration is determined based on the product of the reference duration and the speed multiplier. The candidate segment duration is then used to slide and truncate the target audio to be detected to obtain each of the initial candidate segments corresponding to each of the reference segments. Accordingly, determining the corresponding speed multiplier based on the preset singing conditions includes: For a preset fast singing mode, a first speed multiplier is generated; the first speed multiplier is less than 1. For a preset normal singing speed, a second speed multiplier is generated; the second speed multiplier is equal to 1. For a preset slow singing speed, a third speed multiplier is generated; the third speed multiplier is greater than 1.

[0010] Optionally, determining the similarity between the reference segment and each of the initial candidate segments based on the first harmonic feature of the target reference audio, the first melody skeleton, and the second harmonic feature and the second melody skeleton of the target audio to be detected includes: The first target harmonic feature, the first target trend skeleton, and the first target interval skeleton of the reference segment are determined from the first harmonic feature, the first trend skeleton, and the first interval skeleton of the target reference audio. From the second harmonic features, the second trend skeleton, and the second interval skeleton of the target audio to be detected, determine the second target harmonic features, the second target trend skeleton, and the second target interval skeleton of each of the initial candidate segments; Determine a first similarity between the first target harmonic feature and the second target harmonic feature of the initial candidate segment; Determine the first degree of overlap between the first target trend skeleton and the second target trend skeleton of the initial candidate segment on a continuous sequence of a first preset order; Determine the second degree of overlap between the first target interval skeleton and the second target interval skeleton of the initial candidate segment on a continuous sequence of a second preset order; The first similarity, the first overlap, and the second overlap are weighted and summed to obtain the first target similarity. Then, a number of first candidate segments are selected from each of the initial candidate segments according to the order of the first target similarity, so as to select a number of initial matching segments from the first candidate segments.

[0011] Optionally, selecting a plurality of the initial matching segments from the first candidate segments includes: The second similarity between the first target trend skeleton and the second target trend skeleton of the first candidate segment is determined based on the dynamic time warping algorithm, and the third similarity between the first target interval skeleton and the second target interval skeleton of the first candidate segment is determined based on the dynamic time warping algorithm. The trend skeleton similarity is determined based on the second similarity, and the interval skeleton similarity is determined based on the third similarity. The similarity between the chroma vectors of the first target harmony features and the second target harmony features of the first candidate segment is determined as the harmony structure similarity. The trend skeleton similarity, the interval skeleton similarity, and the harmonic structure similarity are weighted and summed to obtain the second target similarity between the reference segment and the first candidate segment; Based on the second target similarity, several initial matching segments that meet the preset similarity conditions are selected from each of the first candidate segments.

[0012] Optionally, the cover song relationship includes existing cover song relationship, suspected cover song relationship, or no cover song relationship found; Accordingly, after identifying the cover song relationship between the target audio to be detected and the target reference audio based on the similarity, the reference audio coverage, and the audio coverage to be detected, the method further includes: If the cover song relationship between the target audio to be detected and the target reference audio is identified as existing or suspected to be existing, then based on each target matching segment in the evidence chain, the differences in singing style between the target audio to be detected and the target reference audio are analyzed; the differences in singing style include differences in high notes, differences in ending notes, and differences in vibrato amplitude.

[0013] Secondly, this application provides a cover song relationship identification device, including: The audio segmentation module is used to divide the target reference audio into several reference segments, and to divide the target audio to be detected into several initial candidate segments based on preset singing conditions; the preset singing conditions include preset fast singing, preset normal speed singing, and preset slow singing. The matching segment selection module is used to select a number of initial matching segments from each of the initial candidate segments based on the similarity between the reference segment and each of the initial candidate segments for any one of the reference segments; The evidence chain generation module is used to select several target matching segments from each of the initial matching segments based on preset constraint rules, and generate an evidence chain based on each of the target matching segments; The coverage determination module is used to determine the reference audio coverage and the detection audio coverage based on each target matching segment in the evidence chain; the reference audio coverage is the ratio of the combined coverage duration of the target matching segment on the target reference audio to the total duration of the target reference audio, and the detection audio coverage is the ratio of the combined coverage duration of the target matching segment on the target detection audio to the total duration of the target detection audio. The cover song relationship identification module is used to identify the cover song relationship between the target audio to be detected and the target reference audio based on the similarity, the reference audio coverage, and the audio coverage to be detected.

[0014] Thirdly, this application provides an electronic device, comprising: Memory, used to store computer programs; A processor is used to execute the computer program to implement the aforementioned method for identifying cover song relationships.

[0015] Fourthly, this application provides a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned method for identifying cover song relationships.

[0016] In this application, the target reference audio is first divided into several reference segments, and the target audio to be detected is divided into several initial candidate segments based on preset singing conditions. The preset singing conditions include preset fast singing, preset normal speed singing, and preset slow singing. Then, for any reference segment, several initial matching segments are selected from each initial candidate segment based on the similarity between the reference segment and each initial candidate segment. Subsequently, several target matching segments are selected from each initial matching segment based on preset constraint rules, and an evidence chain is generated based on each target matching segment. Next, the reference audio coverage and the audio to be detected coverage are determined based on each target matching segment in the evidence chain. The reference audio coverage is the ratio of the combined coverage duration of the target matching segment on the target reference audio to the total duration of the target reference audio, and the audio to be detected coverage is the ratio of the combined coverage duration of the target matching segment on the target audio to the total duration of the target audio to be detected. Finally, the cover song relationship between the target audio to be detected and the target reference audio is identified based on the similarity, the reference audio coverage, and the audio to be detected coverage. As can be seen from the above, this application can locate suspected cover segments at any time position in long videos or audio files, avoiding direct misses due to the presence of numerous irrelevant segments in the audio to be detected. Furthermore, the evidence chain of multiple reference segments can reduce false positives caused by accidental similarities between different songs. In this way, this application can accurately identify multiple types of cover songs, reduce the false positive and false negative rates, and output interpretable evidence. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0018] Figure 1 A flowchart of a method for identifying cover song relationships provided in this application; Figure 2 A flowchart illustrating a specific method for identifying cover song relationships provided in this application; Figure 3 This application provides a schematic diagram of the structure of a cover song relationship identification device; Figure 4 This application provides a structural diagram of an electronic device. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] With the development of short video platforms, live streaming platforms, music platforms, and AI audio generation technology, users may create covers of existing songs using real people, AI-generated voices, pitch shifting, speed adjustment, different accompaniments, or simply the chorus or a fragment. Such content typically retains the main melody or a segment of the original song, but changes are made in timbre, tempo, tonality, accompaniment, singing style, and audio structure. Traditional audio fingerprinting technology is better suited for identifying audio segments from the same recording version, with similar accompaniments, or those that have undergone slight compression, editing, or speed adjustments. For re-recorded cover audio, the original waveform and spectral details may differ significantly, making it difficult to cover real cover scenarios using only audio fingerprinting. Furthermore, existing melody matching and neural network recognition solutions mostly output simple judgment results, failing to provide traceable segment-level matching evidence, time information, and quantitative indicators, thus failing to meet the actual needs of manual review and evidence tracing in copyright verification scenarios. Therefore, this application provides a cover song relationship identification scheme that can accurately identify multiple types of covers, reduce false positives and false negatives, and output interpretable evidence.

[0021] See Figure 1 As shown in the figure, an embodiment of the present invention discloses a method for identifying cover song relationships, which may include: Step S11: Divide the target reference audio into several reference segments, and divide the target audio to be detected into several initial candidate segments based on preset singing conditions; the preset singing conditions include preset fast singing, preset normal speed singing, and preset slow singing.

[0022] In this embodiment, reference audio A and audio to be detected B are first obtained. If the object to be detected is a video, the audio stream is extracted from the video first. Then, the audio undergoes preprocessing such as mono conversion, resampling, silence trimming, and energy normalization to obtain a unified format audio signal, including the target audio to be detected and the target reference audio. The reference audio serves as the original singer, copyright holder sample, or registered music audio as a comparison benchmark. The audio to be detected is the audio for which cover song relationship identification is required; it can come from song files, short videos, long videos, live stream recordings, or other media files.

[0023] It should be noted that in this embodiment, the fundamental frequency of both the reference audio and the audio to be detected is estimated separately to obtain a frame-by-frame fundamental frequency sequence. Invalid fundamental frequency positions can be filtered or interpolated. The fundamental frequencies are then converted into a pitch sequence. Simultaneously, this embodiment can extract harmonic features from the audio signal, such as chroma features; these harmonic features represent the energy distribution of the audio at different pitch levels and are not used as the sole basis for judging cover song relationships, but rather as auxiliary evidence of the melodic structure. The specific process includes: firstly, fundamental frequency estimation is performed on the target reference audio and the target audio to be detected to obtain a first fundamental frequency sequence of the target reference audio and a second fundamental frequency sequence of the target audio to be detected; then, the first fundamental frequency sequence is converted into a first pitch sequence, and the second fundamental frequency sequence is converted into a second pitch sequence; next, the first harmonic features of the target reference audio and the second harmonic features of the target audio to be detected are extracted; the harmonic features are features that characterize the harmonic structure of the audio; finally, a first melody skeleton of the target reference audio is generated based on the first pitch sequence, and a second melody skeleton of the target audio to be detected is generated based on the second pitch sequence, so as to determine the similarity between the reference segment and each initial candidate segment based on the first harmonic features, the first melody skeleton, the second harmonic features, and the second melody skeleton of the target audio to be detected.

[0024] It should be noted that in this embodiment, smoothing, octave anomaly correction, quantization, short-term jitter filtering, and merging of consecutive identical pitches can be performed on the continuous pitch sequence to obtain stable note nodes. Then, two types of melody skeletons are constructed based on the stable note nodes: a trend skeleton and an interval skeleton. That is, in this embodiment, a first target note node of the target reference audio can be generated based on the first pitch sequence, and a second target note node of the target audio to be detected can be generated based on the second pitch sequence; the target note node is the note node corresponding to consecutive frames with consistent pitch values; then, a first trend skeleton and a first interval skeleton of the target reference audio are generated based on the first target note node, and a second trend skeleton and a second interval skeleton of the target audio to be detected are generated based on the second target note node; wherein, the trend skeleton is used to describe the direction of pitch change between adjacent target note nodes, and the interval skeleton is used to describe the amplitude of pitch change between adjacent target note nodes. Specifically, the trend skeleton is used to record the ascending, descending, or sustaining relationships between adjacent stable note nodes, thereby increasing tolerance for out-of-tune, transposition, or slight pitch deviations; the interval skeleton is used to record the pitch variation between adjacent stable note nodes, thereby preserving the differences in melodic structure and preventing misjudgment of different songs due to similar rising and falling trends.

[0025] In this embodiment, the trend skeleton similarity, interval skeleton similarity, and harmonic structure similarity of the reference audio and the audio to be detected at the overall level can be calculated separately, and these similarities can be fused into a global fusion similarity. The global similarity is used as an auxiliary indicator and is not used as the sole criterion for judgment.

[0026] It should be noted that in this embodiment, the reference audio is divided into multiple reference segments according to a preset segment length and step size. For each reference segment, it is not required to align with the same time position of the audio to be detected, but rather candidate segments are generated within the complete time range of the audio to be detected.

[0027] In this embodiment, the target audio to be detected can be divided based on a preset singing scenario to obtain several initial candidate segments corresponding to each reference segment. The specific process may include: first, determining the corresponding speed ratio based on the preset singing scenario; then, for any reference segment, determining the reference duration of the reference segment, and determining the corresponding candidate segment duration based on the product of the reference duration and the speed ratio; and sliding and truncating the target audio to be detected based on the candidate segment duration to obtain each initial candidate segment corresponding to each reference segment.

[0028] Accordingly, the process of determining the corresponding speed multiplier based on the preset singing conditions can include: generating a first speed multiplier for preset fast singing; the first speed multiplier is less than 1; generating a second speed multiplier for preset normal singing; the second speed multiplier is equal to 1; generating a third speed multiplier for preset slow singing; the third speed multiplier is greater than 1.

[0029] Specifically, to accommodate fast singing, slow singing, or local speed variations, for a reference segment of length L, initial candidate segments of length L×r can be generated according to multiple candidate window length ratios r, where r can include multiple values ​​less than 1, equal to 1, and greater than 1. That is, in this embodiment, the reference audio is divided into multiple reference segments. For each reference segment, candidate segments are generated within the complete time range of the audio to be detected according to a preset search step size. To accommodate fast singing, slow singing, or local speed variations in cover songs, multiple candidate window length ratios are set for the same reference segment, allowing the length of the candidate segment to be detected to be shorter than, equal to, or longer than the length of the reference segment. Let the length of the first candidate segment be... The reference fragment is Its length is For candidate start times in the audio to be detected and candidate window length multiplier Generate initial candidate fragments: ; in, It can include multiple values, such as less than 1, equal to 1, and greater than 1, to cover fast singing, normal speed singing, and slow singing respectively.

[0030] Step S12: For any of the reference segments, select several initial matching segments from the initial candidate segments based on the similarity between the reference segment and each of the initial candidate segments.

[0031] Understandably, in long video, live stream recording, or long audio scenarios, the duration of the audio to be detected is usually quite long. Directly performing fine sequence matching on each reference segment and all candidate segments along the entire timeline of the audio to be detected would result in a large computational load. Therefore, in this embodiment, a candidate coarse screening stage is set up before fine matching to quickly retain the first candidate segment that is most likely to have a melodic correspondence with the reference segment from a large number of initial candidate segments.

[0032] Specifically, in the first-stage candidate coarse screening process, the first target harmonic features, first target trend skeleton, and first target interval skeleton of the reference segment are first determined from the first harmonic features, first trend skeleton, and first interval skeleton of the target reference audio; then, the second target harmonic features, second target trend skeleton, and second target interval skeleton of each initial candidate segment are determined from the second harmonic features, second trend skeleton, and second interval skeleton of the target audio to be detected; next, the first similarity between the first target harmonic features of the reference segment and the second target harmonic features of the initial candidate segments is determined; subsequently, the first target trend skeleton of the reference segment and the second target interval skeleton of the initial candidate segments are determined. The first overlap of the second target trend skeleton of the initial candidate segment on a continuous sequence of a first preset order is determined; then the second overlap of the first target interval skeleton of the reference segment and the second target interval skeleton of the initial candidate segment on a continuous sequence of a second preset order is determined; finally, the first similarity, the first overlap and the second overlap are weighted and summed to obtain the first target similarity, and several first candidate segments are selected from each initial candidate segment according to the order of the first target similarity, so as to select several initial matching segments from the first candidate segments; wherein, the similarity between each initial matching segment and the reference segment is greater than the preset similarity threshold.

[0033] Specifically, in the candidate coarse screening stage, full dynamic time warping is not performed; instead, a candidate screening score is calculated based on lightweight features. The candidate screening score, i.e., the first target similarity, includes the overlap of short sequences with the trend skeleton. (First degree of overlap), Short sequence overlap in interval direction (Second degree of overlap) and rapid harmonic similarity (First similarity).

[0034] It should be noted that, in order to calculate the overlap of short sequences in the trend skeleton... (First degree of overlap), in this embodiment, a reference segment is provided. Trend skeleton sequence for: ; in, The total number of nodes contained in the trend skeleton sequence of the reference segment.

[0035] Initial candidate fragments to be detected Trend skeleton sequence for: ; in, This represents the total number of nodes in the trend skeleton sequence of the initial candidate segment. The trend value is used to indicate the ascending, descending, or holding relationship between adjacent stable note nodes.

[0036] Define any sequence of Set of continuous short sequences of order for: ; The overlap of short-sequence trend skeleton It can be represented as: ; in, A value of 6 is acceptable. This metric is used to measure the local consistency between the reference segment and the initial candidate segment in the melodic rise and fall trend.

[0037] To calculate the overlap of short sequences in the interval direction In this embodiment, a reference segment is provided. interval skeleton sequence for: ; in, The single-step pitch difference within the interval framework of the reference segment. This represents the total number of nodes in the interval skeleton sequence of the reference segment.

[0038] Initial candidate fragments interval skeleton sequence for: ; in, The single-step pitch difference within the interval skeleton of the initial candidate segment. This represents the total number of nodes in the interval skeleton sequence of the initial candidate segment.

[0039] Next, the interval skeleton sequence is converted into an interval direction sequence: ; ; in, It indicates the direction of the interval change, i.e., ascending, descending, or remaining still.

[0040] Then the overlap of short sequences in the interval direction It can be represented as: ; in, A score of 7 is acceptable. This index does not directly compare the size of specific intervals, but rather the direction of pitch change. Therefore, it is suitable for quickly eliminating candidate segments with obviously inconsistent melodic directions during the initial screening stage.

[0041] To calculate fast harmonic similarity In this embodiment, a reference segment is provided. chromaticity feature matrix for: ; Initial candidate fragments chromaticity feature matrix for: ; Calculate the average chromaticity vectors of the two over the time dimension respectively. and : ; ; in, The total number of frames in the reference segment. The total number of frames for the initial candidate segments. This represents the chroma value of the f-th frame across all pitch levels of the matrix.

[0042] To accommodate key-shifting covers, the average chroma vector of the initial candidate segments can be cyclically shifted in 12 semitone directions, and the maximum cosine similarity can be taken to obtain a fast harmonic similarity. : ; in, This indicates a cyclic translation of the chroma vector. Each semitone. This index is used to quickly measure the similarity between a reference segment and a candidate segment in terms of pitch energy distribution or harmonic structure.

[0043] Next, the overlap of short-series trend skeletons was analyzed. , overlap of short sequences in pitch direction and rapid harmonic similarity The first target similarity is obtained by weighted summation. : ; in, , , This refers to the weighting parameter. In one specific implementation, , , These values ​​can be set to 0.45, 0.25, and 0.30 respectively. These values ​​are just examples; in actual applications, they can be adjusted according to the audio type, application scenario, or labeled samples.

[0044] Finally, the initial candidate fragments are sorted according to their initial screening scores, and several top-scoring candidate fragments are retained for each reference fragment. This allows for the retention of the top-ranked fragments at each candidate window length multiplier. A candidate pool is formed by selecting 10 candidate segments and creating a candidate pool corresponding to the reference segment.

[0045] Through the aforementioned candidate coarse screening, this embodiment can retain the most likely first candidate segment from a large number of initial candidate segments across the entire timeline of the audio to be detected without performing costly fine matching. Specifically, firstly, this embodiment can reduce the computational load in long video or long audio scenarios. It eliminates the need to perform full dynamic time warping on all candidate segments; instead, it quickly retains candidates using lightweight features. Secondly, it improves the recall capability of fragmented cover songs. Since candidate segments are generated across the entire timeline of the audio to be detected, the reference audio does not need to start at the same point as the audio to be detected, thus enabling the detection of cover segments appearing at any time position. Thirdly, it improves the adaptability to covers with varying speeds. By setting multiple candidate window length ratios, it can cover fast singing, slow singing, or local speed variations. Fourthly, it provides a candidate pool for subsequent fine-tuning. The number of candidate segments output in the coarse screening stage is significantly less than the total number of candidates across the entire timeline, allowing subsequent fine-tuning to perform more refined similarity calculations with controllable computational load.

[0046] It should be noted that after obtaining several first candidate segments in the candidate pool through the first stage of candidate coarse screening, this embodiment performs a second stage of candidate fine sorting on the first candidate segments in the candidate pool. Unlike the coarse screening stage, the fine sorting stage performs more refined melody and harmony structure similarity calculations on the retained candidate segments to obtain segment-level matching results that can be used for subsequent construction of time monotonic evidence chains. That is, several initial matching segments are selected from the first candidate segments. The specific process may include: First, based on the dynamic time warping algorithm, the second similarity between the first target trend skeleton of the reference segment and the second target trend skeleton of the first candidate segment, and the third similarity between the first target interval skeleton of the reference segment and the second target interval skeleton of each first candidate segment are determined; then, the trend skeleton similarity between the reference segment and the first candidate segment is determined based on the second similarity, and the interval skeleton similarity between the reference segment and the first candidate segment is determined based on the third similarity; next, the similarity between the chroma vectors of the first target harmonic features of the reference segment and the second target harmonic features of the first candidate segment is determined as the harmonic structure similarity between the reference segment and the first candidate segment; then, the trend skeleton similarity, interval skeleton similarity, and harmonic structure similarity are weighted and summed to obtain the second target similarity between the reference segment and the first candidate segment; finally, based on the second target similarity, several initial matching segments that meet the preset similarity conditions are selected from each first candidate segment; the preset similarity conditions are used to control that the second target similarity corresponding to the initial matching segments is greater than the preset similarity threshold.

[0047] Specifically, regarding the reference fragment and each first candidate fragment in its candidate pool The similarity scores for trend skeleton, interval skeleton, and harmonic structure are calculated separately. Trend skeleton similarity measures the overall consistency between the reference and candidate segments in terms of melodic ascending, descending, or sustained relationships. Interval skeleton similarity measures the consistency in pitch variation between adjacent stable note nodes. Harmonic structure similarity measures the similarity in pitch energy distribution or chromaticity characteristics.

[0048] In one specific implementation, trend skeleton similarity It is obtained by fusing the dynamic time-warped similarity of the trend series and the overlap of the short trend series: ; in, This represents the dynamic time-warped similarity of a trend sequence. Indicates the degree of overlap in short-term trend sequences. and These are the weight parameters. and These can be set to 0.65 and 0.35 respectively. The above values ​​are just examples and can be adjusted in actual applications.

[0049] In one specific implementation, interval skeleton similarity It is obtained by fusing the dynamic time warping similarity of the interval sequence, the overlap of short interval sequences, and the overlap of interval direction contours: ; in, This represents the dynamic time warping similarity of the audio sequence. Indicates the degree of overlap in short interval sequences. Indicates the degree of overlap of short sequences of pitch direction profiles. , , These are the weight parameters.

[0050] It should be noted that, in order to calculate the dynamic time warping similarity of the audio sequence... Let the interval skeleton sequences of the reference segment and the first candidate segment be respectively... and Next, dynamic time warping can be performed on both to obtain the cumulative alignment distance: ; Let the path length of the dynamic time warping be... Then the normalized distance for: ; Then convert the distance into similarity. : ; in, This is the scale parameter. A value of 1.35 can be used. This index is used to adjust the alignment of musical sequences under local rhythmic changes or changes in singing speed.

[0051] Then, the overlap of short interval sequences is calculated directly based on the specific interval change values. The formula can be expressed as: ; in, A value of 4 is acceptable. This indicator measures the local consistency between two segments in terms of specific interval change patterns, which can avoid misjudgment based solely on "similar ascending and descending trends".

[0052] Then calculate the overlap of short sequences of pitch direction profiles. First, convert the interval skeleton sequence into a directional contour sequence. and : ; ; Then, the crossover-union ratio between the sets of short sequence directional profiles is calculated to obtain the overlap degree of the short sequence directional profiles. : ; in, A value of 7 is acceptable. This metric is used to measure the local consistency of two segments in the contour of pitch change direction.

[0053] It should be noted that, , , These can be set to 0.55, 0.30, and 0.15 respectively, thus increasing the interval skeleton similarity. This can be specifically expressed as: ; The weights mentioned above are only one implementation method and can be adjusted according to the labeled samples or specific application scenarios.

[0054] In one specific implementation, the harmonic structure similarity can be calculated based on the chromaticity features or pitch level energy distribution of the reference segment and the candidate segment, and the harmonic structure similarity is obtained by calculating the similarity between their chromaticity statistical vectors. It adapts to key-shifting covers by maximizing similarity under different semitone shifts.

[0055] Subsequently, the similarity of the trend skeleton, the similarity of the interval skeleton, and the similarity of the harmonic structure are fused to obtain the second target similarity. : ; in, , , For weight fusion. , , These values ​​can be set to 0.40, 0.45, and 0.15 respectively. These values ​​are just examples; in actual applications, they can be adjusted according to different scenarios.

[0056] After the second stage of candidate sorting is completed, a structured segment matching result is output for each candidate segment. The structured segment matching result may include: the start and end times of the reference segment, the start and end times of the candidate segment to be detected, the candidate window length multiplier, the trend skeleton similarity, the interval skeleton similarity, the harmonic structure similarity, and the second target similarity.

[0057] Finally, the candidate results corresponding to each reference segment can be sorted according to the second target similarity, and the top-ranked candidate segments can be retained as the initial matching segments and as candidate groups for the subsequent construction of the time monotonic evidence chain.

[0058] Through the aforementioned candidate refinement process, this embodiment can further transform the candidate segments obtained in the coarse screening stage into segment-level matching results with sub-item similarity, fusion similarity, and temporal position information. Specifically, firstly, this embodiment can improve the accuracy of candidate segment matching. Compared to the coarse screening stage which only uses lightweight features, the refinement stage introduces dynamic time regularization similarity and multiple melodic structure similarities, which can more accurately measure the melodic correspondence between reference segments and candidate segments. Secondly, it improves the adaptability to live covers, AI covers, key-shifted covers, and tempo-shifted covers. Trend skeleton similarity can tolerate a certain degree of pitch deviation or transposition, interval skeleton similarity can preserve melodic structure differences, and harmonic structure similarity can provide auxiliary musical background evidence. Thirdly, it reduces misjudgments caused by single feature similarity. By simultaneously fusing trend skeleton, interval skeleton, and harmonic structure, it avoids judging candidate segments based solely on a single melodic outline or a single harmonic feature. Fourthly, it provides structured input for the subsequent construction of a temporal monotonic evidence chain. The reference time, time to be detected, candidate window length multiplier, sub-item similarity, and fusion similarity output during the fine-tuning stage can serve as candidate node information for subsequent dynamic programming to select evidence chains. Fifth, improve the interpretability of results. It can show copyright reviewers the trend similarity, interval similarity, harmony similarity, and fusion similarity of each candidate segment, thus providing segment-level evidence for manual review.

[0059] Step S13: Select several target matching segments from each of the initial matching segments based on preset constraint rules, and generate an evidence chain based on each of the target matching segments.

[0060] It should be noted that the preset constraint rules are used to control the temporal order of each reference segment in the target reference audio, ensuring that it is consistent with the temporal order of the corresponding target matching segment in the target audio to be detected, and to control the overlap between each target matching segment to be less than the preset overlap threshold.

[0061] After obtaining the second target similarity in the second stage of candidate ranking, this embodiment further constructs a time-monotonic evidence chain. The purpose of this step is not to simply select a single candidate segment with the highest similarity, but to select a set of target matching segments with a reasonable sequential relationship on the timeline of the audio to be detected from the initial matching segments corresponding to multiple reference segments, forming a segment-level evidence sequence.

[0062] Specifically, the refined candidate results corresponding to each reference segment, i.e., the initial matching segments, are treated as a candidate group. For candidate groups corresponding to different reference segments, the candidate segment sequence with higher cumulative evidence scores and satisfying the time monotonicity constraint is selected based on factors such as the similarity to the second target, the order of the reference segments, the start and end times of the segment to be detected, and the degree of overlap of the candidate segments.

[0063] In one specific implementation, a time-monotonic chain of evidence can be constructed using dynamic programming. For matching fragments... and ,like The corresponding reference segment is located at Following the corresponding reference segment, The start time in the audio to be detected should, in principle, not be significantly earlier than [previous time]. The start time in the audio to be detected.

[0064] If the start time of the later matching segment in the audio to be detected is significantly earlier than that of the previous matching segment, the transition is prohibited, or a time backoff penalty is applied; if the time intervals of the two matching segments in the audio to be detected overlap too much, an overlap penalty is applied to avoid the same segment being counted repeatedly in the chain of evidence.

[0065] Specifically, the transfer penalties between matching segments can include: (1) time rollback penalty, used to suppress candidate combinations that are obviously out of order on the time axis to be detected. (2) time overlap penalty, used to suppress multiple matching segments from repeatedly covering the same region to be detected. (3) low similarity penalty, used to reduce the probability of matching segments with insufficient fusion similarity entering the evidence chain.

[0066] In one specific implementation, the chain of evidence path score It can be represented as: ; in, Indicates the number of matching fragments in the chain of evidence. Indicates the first The second target similarity of the matched segments This represents the transition penalty between adjacent matched segments.

[0067] The above formula is only one possible implementation. In addition to dynamic programming, the chain of evidence can also be constructed using the longest path, shortest path, graph search, bundle search, or other sequence path selection methods. This embodiment is not limited to a specific path search algorithm.

[0068] Through the above processing, one or more time-monotonic evidence chains can be output. Each target matching segment in the evidence chain may include the start and end times of the reference segment, the start and end times of the segment to be detected, the candidate window length multiplier, the trend skeleton similarity, the interval skeleton similarity, the harmonic structure similarity, the second target similarity, and the path score.

[0069] By constructing the aforementioned time-monotonous evidence chain, firstly, this embodiment reduces the risk of misjudgment caused by a single locally highly similar segment. This embodiment does not determine the cover song relationship solely based on the highest local similarity, but requires multiple candidate segments to form an evidence sequence with a reasonable sequential relationship on the timeline of the audio to be detected. Secondly, it improves the reliability of segment-based cover song recognition. Even if the cover song content only appears within a portion of the audio to be detected, this embodiment can still locate the corresponding reference segment and the segment to be detected through the evidence chain. Thirdly, it reduces the impact of duplicate evidence on the judgment result. Through overlap penalty, this embodiment can avoid the evidence strength being overestimated due to multiple reference segments repeatedly hitting the same time region to be detected. Fourthly, it provides a basis for subsequent coverage calculation. The reference segment time range and the segment to be detected time range output by the evidence chain can be used to subsequently calculate the reference audio coverage, the audio to be detected coverage, and the proportion of effective matching segments. Fifthly, it improves the interpretability of the results. This embodiment can output the reference time, the time to be detected, the speed ratio, the sub-item similarity, and the path score for each segment in the evidence chain, providing copyright reviewers with verifiable segment-level evidence.

[0070] Step S14: Determine the reference audio coverage and the detection audio coverage based on each target matching segment in the evidence chain; the reference audio coverage is the ratio of the combined coverage duration of the target matching segment on the target reference audio to the total duration of the target reference audio, and the detection audio coverage is the ratio of the combined coverage duration of the target matching segment on the target detection audio to the total duration of the target detection audio.

[0071] In this embodiment, based on the target matching segments in the evidence chain, the local highest similarity, the proportion of effective matching segments, the reference audio coverage, and the detection audio coverage are calculated.

[0072] The reference audio coverage is the ratio of the combined coverage duration of the target matching segment on the reference audio timeline to the total duration of the target reference audio; the detection audio coverage is the ratio of the combined coverage duration of the target matching segment on the detection audio timeline to the total duration of the target detection audio.

[0073] Step S15: Identify the cover song relationship between the target audio to be detected and the target reference audio based on the similarity, the reference audio coverage and the audio coverage to be detected.

[0074] In this embodiment, the cover song relationship between the target audio to be detected and the target reference audio can be output based on the global similarity, the local highest similarity, the proportion of effective matching segments, the coverage of the reference audio, and the coverage of the audio to be detected; the cover song relationship includes the existence of a cover song relationship, the suspected existence of a cover song relationship, or the absence of a cover song relationship.

[0075] It should be noted that in this embodiment, the cover song relationship determination model can be trained based on labeled samples. The model takes global similarity, local maximum similarity, proportion of effective matching segments, reference audio coverage, coverage of the audio to be detected, trend skeleton similarity, interval skeleton similarity, harmonic structure similarity, evidence chain length, and candidate window length multiplier as inputs, and outputs the confidence score of the cover song relationship. The cover song relationship determination model is used to fuse structured evidence or perform threshold calibration, rather than replacing the cross-temporal evidence chain itself.

[0076] In this embodiment, when the cover song relationship between the target audio to be detected and the target reference audio is identified as existing or suspected to be existing, the differences in singing style between the target audio to be detected and the target reference audio are further analyzed based on each target matching segment in the evidence chain; the differences in singing style include differences in high notes, differences in ending notes, and differences in vibrato amplitude.

[0077] It should be noted that the analysis of differences in singing style is not the sole basis for determining whether a cover song relationship exists, nor is it used for singer identification or voiceprint identification alone. This step is used to further explain the differences in singing style between the reference audio and the audio to be tested, based on the existing or suspected melodic structure correspondence, providing auxiliary verification information for copyright reviewers.

[0078] In this embodiment, based on the pitch sequences of the reference audio and the audio to be detected, the high-pitched stability, the descent of the trailing note, and the vibrato amplitude can be calculated respectively, and the differences in singing style can be further calculated.

[0079] (1) Differences in high-pitched stability

[0080] First, the high-pitched region is determined based on the pitch sequence. Pitch frames in the pitch sequence that are higher than a preset quantile can be identified as the high-pitched region. Then, the degree of pitch fluctuation within the high-pitched region relative to the local center pitch is calculated to characterize the singer's stability in the high-pitched range.

[0081] In one implementation, high-frequency stability It can be represented as: ; in, Indicates the first The pitch value corresponding to the frame. Indicates the center pitch after local smoothing. Indicates the high-pitched range. This represents the standard deviation. The value, expressed in centimeters, indicates the degree of high-pitched fluctuation.

[0082] Difference in treble stability between reference audio and the audio to be tested It can be represented as: ; in, Indicates the high-frequency stability of the reference audio. This indicates the high-frequency stability of the audio being tested.

[0083] (2) Difference in the descent of the final note

[0084] Based on stable note nodes or continuous note segments in the pitch sequence, identify the tail note region at the end of each stable note or musical phrase, and calculate the pitch change at the end of the tail note region relative to the beginning of the tail note.

[0085] In one implementation, for a stable note segment Its trailing sag It can be represented as: ; in, This indicates the starting frame of the region where the note ends. This indicates the end frame of the region where the note ends. The glissando signature of the audio can be obtained by taking the median or average of the glissando measurements from multiple stable note segments. : ; Difference in glissando between the reference audio and the audio to be detected It can be represented as: ; in, This indicates the glissando characteristic at the end of the reference audio. This indicates the glissando feature at the end of the audio being detected.

[0086] Differences in the glissando at the end of a phrase can be used to describe differences in the final consonant, glissando, weakening, or singing habits of different versions.

[0087] (3) Difference in tremor amplitude

[0088] Based on pitch sequence and local center pitch, the fluctuation range of pitch residual is calculated to characterize vibrato or natural pitch fluctuation in stable singing regions.

[0089] In one implementation, the amplitude of the jitter It can be represented as: ; in, Represents the 90th quantile. This represents the residual of pitch relative to the local center pitch. This value is used to describe the range of pitch swing around the center pitch.

[0090] Difference in jitter amplitude between reference audio and the audio to be detected It can be represented as: ; in, This indicates the amplitude of the vibration in the reference audio. This indicates the amplitude of the vibration in the audio to be detected.

[0091] (4) Style difference label output

[0092] Based on differences in high-frequency stability Difference in the descent of the final note Difference in vibration amplitude Generate tags for differences in singing styles.

[0093] In one specific implementation, the differences in each feature can be classified as follows: (1) When the difference in high-pitched stability is small, output "high-pitched stability is close"; (2) When the difference in high-pitched stability is in the middle range, output "high-pitched stability is slightly different"; (3) When the difference in high-pitched stability is large, output "high-pitched stability is significantly different"; (4) When the difference in the descent of the final note is small, output "the descent of the final note is close"; (5) When the difference in the descent of the final note is large, output "the descent of the final note is different"; (6) When the difference in the vibration amplitude is small, output "the vibration amplitude is close"; (7) When the difference in the vibration amplitude is large, output "the vibration amplitude is significantly different".

[0094] Next, comprehensive style tags can be generated based on the above sub-tags, including: similar styles; slight differences in style; and significant differences in style.

[0095] In one specific implementation, when at least one sub-feature has a significant difference, "significant style difference" can be output; when one or more sub-features have a moderate difference but do not reach a significant difference, "slight style difference" can be output; when the differences of each sub-feature are small, "similar style" can be output.

[0096] It should be noted that the results of the singing style difference analysis can be output together with the results of the cover song relationship identification. The output content may include: differences in high note stability, differences in the descent of the final note, differences in vibrato amplitude, sub-labels for high note stability, sub-labels for the final note processing, sub-labels for vibrato amplitude, and comprehensive style difference labels.

[0097] Through the above processing, this embodiment can not only output whether a cover song relationship exists, but also further explain the differences in singing behavior between the version to be detected and the reference version. That is, firstly, it improves the interpretability of the cover song recognition results. This embodiment not only outputs "a cover song relationship exists," "a suspected cover song relationship," or "no cover song relationship found," but also outputs information on differences in singing style such as high-pitched stability, sloping trailing notes, and vibrato amplitude, providing copyright reviewers with a more intuitive basis for verification. Secondly, it avoids mistaking differences in singing style for non-cover songs. Real-person covers, AI covers, voice-changing covers, or reinterpretations may differ from the reference audio in timbre, singing method, and singing habits, but still retain the main melody. In this embodiment, style differences are used as auxiliary explanatory information, rather than as the sole basis for denying a cover song relationship, thereby reducing the risk of missed detection. Thirdly, it can help distinguish different cover song performance forms. Differences in high-pitched stability can reflect differences in pitch control in the high-pitched region, differences in sloping trailing notes can reflect differences in phrase ending processing and closing habits, and differences in vibrato amplitude can reflect differences in vibrato or natural pitch fluctuations. Fourthly, it improves the efficiency of manual verification. For samples already identified as having a cover song relationship or suspected cover song relationship, this embodiment can further output labels such as similar style, slight style difference, or significant style difference, enabling reviewers to quickly understand the differences in singing style between the two versions. Fifth, it differs from ordinary voiceprint recognition. Ordinary voiceprint recognition usually focuses on whether it is the same speaker or the same singer, while the singing style difference analysis in this embodiment is based on melody structure matching and cover song relationship determination, focusing on the differences in singing behavior between the two versions. Therefore, it is more suitable for cover song recognition, AI cover song recognition, voice-changing cover song detection, and copyright review scenarios.

[0098] In one specific implementation, see Figure 2As shown, the specific process for cover song relationship recognition can be as follows: input reference audio / audio to be detected; perform audio preprocessing; extract the pitch and harmony features of the main melody; construct stable note nodes; generate trend skeleton and interval skeleton; calculate global similarity; slide segment the reference audio; generate multi-speed candidates for the audio to be detected across time; perform the first stage candidate coarse screening; perform the second stage candidate fine ranking; generate a time monotonic evidence chain through dynamic programming; fuse coverage and similarity to complete the judgment; output the cover song recognition result and style difference label.

[0099] It should be noted that, for the above steps, without changing the core idea of ​​"cross-time candidate retrieval, two-layer melody structure, temporal monotonic evidence chain, and coverage fusion judgment," the following alternative methods can also be used: (1) The segmentation method of the reference audio can be a fixed time window, a beat window, a phrase boundary, a verse / chorus structure boundary, a multi-scale window, or a fixed stable note number window. (2) The candidate segment generation method of the audio to be detected can be a fixed step size sliding window, a beat-aligned sliding window, a vocal activity area detection, a music segment detection, an adaptive speed window, or a secondary fine search near the hit position. (3) The construction method of stable note nodes can be a pitch duration judgment, a local pitch midpoint, a note start point detection, a segmented linear approximation, a main melody node extraction after vocal separation, or a node extraction based on the pitch change slope. (4) The pitch extraction method can be CREPE or other main melody estimation algorithms. (5) The harmonic features can be constant Q transform features, chord sequence features, tonality features, pitch distribution features, or other music structure features. (6) The melody structure expression can be relative pitch sequence, interval category sequence, melody contour encoding, melody phrase token, interval direction sequence, or trend and duration joint encoding. (7) Similarity calculation can use edit distance, longest common subsequence, cosine similarity, correlation coefficient, graph matching method or trained segment matching model. (8) Candidate coarse screening can use inverted index, local sensitive hashing, vector nearest neighbor retrieval, melody phrase index, low information content segment filtering or lightweight machine learning ranking model. (9) Candidate fine ranking can use multi-feature weighted fusion, adaptive weight fusion, melody-first-harmony hierarchical judgment, segment matching model scoring or fusion scoring model trained based on labeled samples. (10) Evidence chain construction can use shortest path, longest path, graph search, bundle search, hidden Markov model, conditional random field or other sequence decision method. (11) Coverage judgment can use reference coverage, detection coverage, number of consecutive hit segments, average similarity of evidence chain, evidence chain length or weighted combination of the above indicators. (12) Fusion judgment can use statistical calibration threshold, logistic regression, decision tree, random forest, gradient boosting tree or neural network model. (13) Style difference features can be expanded to include breath stability, formant features, timbre similarity, vocal embedding vector, rhythm stability, lyric matching degree, pronunciation consistency, or suspected AI voice-changing features. (14) Output results can include cover song relationship tags, suspected cover song confidence, segment-level evidence report, reference audio hit time, target audio hit time, speed multiplier, coverage index, or manual review prompts.

[0100] All of the above alternative methods can achieve the identification, segment localization, and evidence output of the cover song relationship between the reference audio and the audio to be detected without changing the core process of this embodiment.

[0101] As can be seen from the above, this embodiment can locate suspected cover segments at any time position in long videos or long audio recordings, avoiding direct misses due to the presence of numerous irrelevant segments in the audio to be detected. Furthermore, the evidence chain of multiple reference segments can reduce false positives caused by accidental similarities between different songs. Thus, this embodiment can accurately identify multiple types of cover songs, reduce the false positive and false negative rates, and output interpretable evidence.

[0102] Accordingly, see Figure 3 As shown in the embodiment of this application, a cover song relationship identification device is also provided, which may include: The audio segmentation module 11 is used to divide the target reference audio into several reference segments, and to divide the target audio to be detected into several initial candidate segments based on preset singing conditions; the preset singing conditions include preset fast singing, preset normal speed singing, and preset slow singing. The matching segment selection module 12 is used to select a number of initial matching segments from each of the initial candidate segments based on the similarity between the reference segment and each of the initial candidate segments for any one of the reference segments; The evidence chain generation module 13 is used to select several target matching segments from each of the initial matching segments based on preset constraint rules, and generate an evidence chain based on each of the target matching segments; The coverage determination module 14 is used to determine the reference audio coverage and the detection audio coverage based on each of the target matching segments in the evidence chain; the reference audio coverage is the ratio of the combined coverage duration of the target matching segment on the target reference audio to the total duration of the target reference audio, and the detection audio coverage is the ratio of the combined coverage duration of the target matching segment on the target detection audio to the total duration of the target detection audio. The cover song relationship identification module 15 is used to identify the cover song relationship between the target audio to be detected and the target reference audio based on the similarity, the reference audio coverage and the audio coverage to be detected.

[0103] In some specific embodiments, the cover song relationship identification device may further include: A fundamental frequency sequence determination module is used to perform fundamental frequency estimation on the target reference audio and the target audio to be detected, respectively, to obtain a first fundamental frequency sequence of the target reference audio and a second fundamental frequency sequence of the target audio to be detected; A pitch sequence determination module is used to convert the first fundamental frequency sequence into a first pitch sequence and the second fundamental frequency sequence into a second pitch sequence; The harmony feature extraction module is used to extract the first harmony feature of the target reference audio and the second harmony feature of the target audio to be detected; the harmony feature is a feature that characterizes the harmony structure of the audio. The melody skeleton generation module is used to generate a first melody skeleton of the target reference audio based on the first pitch sequence, and to generate a second melody skeleton of the target audio to be detected based on the second pitch sequence, so as to determine the similarity between the reference segment and each of the initial candidate segments based on the first harmonic features of the target reference audio, the first melody skeleton and the second harmonic features of the target audio to be detected, and the second melody skeleton.

[0104] In some specific embodiments, the melody skeleton includes a trend skeleton and an interval skeleton; Accordingly, the melody skeleton generation module may include: The note node generation unit is used to generate a first target note node of the target reference audio according to the first pitch sequence, and to generate a second target note node of the target audio to be detected according to the second pitch sequence; the target note node is the note node corresponding to consecutive frames with consistent pitch values; An interval skeleton generation unit is used to generate a first trend skeleton and a first interval skeleton of the target reference audio based on the first target note node, and to generate a second trend skeleton and a second interval skeleton of the target audio to be detected based on the second target note node; wherein, the trend skeleton is used to describe the pitch change direction between adjacent target note nodes, and the interval skeleton is used to describe the pitch change amplitude between adjacent target note nodes.

[0105] In some specific embodiments, the audio segmentation module 11 may include: The speed ratio determination submodule is used to determine the corresponding speed ratio based on preset singing conditions; The initial candidate segment determination submodule is used to determine the reference duration of any reference segment, determine the corresponding candidate segment duration based on the product of the reference duration and the speed multiplier, and perform sliding segmentation on the target audio to be detected based on the candidate segment duration to obtain each initial candidate segment corresponding to each reference segment. Accordingly, the speed ratio determination submodule may include: The first speed ratio determination unit is used to generate a first speed ratio for a preset fast singing performance; the first speed ratio is less than 1. The second speed multiplier determination unit is used to generate a second speed multiplier for singing at a preset normal speed; the second speed multiplier is equal to 1. The third speed multiplier determination unit is used to generate a third speed multiplier for a preset slow singing speed; the third speed multiplier is greater than 1.

[0106] In some specific embodiments, the melody skeleton generation module may include: The first target harmony feature determination submodule is used to determine the first target harmony feature, the first target trend skeleton, and the first target interval skeleton of the reference segment from the first harmony feature, the first trend skeleton, and the first interval skeleton of the target reference audio. The second target harmony feature determination submodule is used to determine the second target harmony feature, second target trend skeleton, and second target interval skeleton of each of the initial candidate segments from the second harmony feature, second trend skeleton, and second interval skeleton of the target audio to be detected; The first similarity determination submodule is used to determine the first similarity between the first target harmony feature and the second target harmony feature of the initial candidate segment; The first overlap determination submodule is used to determine the first overlap between the first target trend skeleton and the second target trend skeleton of the initial candidate segment on a continuous sequence of a first preset order. The second overlap determination submodule is used to determine the second overlap between the first target interval skeleton and the second target interval skeleton of the initial candidate segment on a continuous sequence of a second preset order. The first candidate segment selection submodule is used to perform a weighted summation of the first similarity, the first overlap, and the second overlap to obtain a first target similarity, and to select a number of first candidate segments from each of the initial candidate segments according to the order of the first target similarity, so as to select a number of initial matching segments from the first candidate segments.

[0107] In some specific implementations, the first candidate fragment selection submodule may include: The second similarity determination unit is used to determine the second similarity between the first target trend skeleton and the second target trend skeleton of the first candidate segment, and the third similarity between the first target interval skeleton and the second target interval skeleton of the first candidate segment, based on the dynamic time warping algorithm. An interval skeleton similarity determination unit is used to determine trend skeleton similarity based on the second similarity and to determine interval skeleton similarity based on the third similarity. The harmonic structure similarity determination unit is used to determine the similarity between the chroma vector of the first target harmonic feature and the second target harmonic feature of the first candidate segment as the harmonic structure similarity. The second target similarity determination unit is used to perform a weighted summation of the trend skeleton similarity, the interval skeleton similarity, and the harmonic structure similarity to obtain the second target similarity between the reference segment and the first candidate segment; The initial matching segment determination unit is used to select a number of initial matching segments that meet preset similarity conditions from each of the first candidate segments based on the second target similarity.

[0108] In some specific implementations, the cover song relationship includes existing cover song relationship, suspected existing cover song relationship, or no cover song relationship found; Correspondingly, the cover song relationship identification device may further include: The singing style difference analysis module is used to analyze the singing style differences between the target audio to be detected and the target reference audio based on each target matching segment in the evidence chain if the cover song relationship between the target audio to be detected and the target reference audio is identified as existing or suspected to be existing. The singing style differences include differences in high notes, differences in ending notes, and differences in vibrato amplitude.

[0109] Furthermore, embodiments of this application also disclose an electronic device, Figure 4 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the cover song relationship identification method disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be a computer.

[0110] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0111] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0112] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the cover song relationship identification method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include a computer program capable of performing other specific tasks.

[0113] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned method for identifying cover song relationships. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0114] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0115] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0116] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0117] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0118] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for identifying cover song relationships, characterized in that, include: The target reference audio is divided into several reference segments, and the target audio to be detected is divided into several initial candidate segments based on the preset singing conditions; The preset singing scenarios include preset fast singing, preset normal speed singing, and preset slow singing. For any of the reference segments, several initial matching segments are selected from each of the initial candidate segments based on the similarity between the reference segment and each of the initial candidate segments; Based on preset constraint rules, several target matching segments are selected from each of the initial matching segments, and an evidence chain is generated based on each of the target matching segments; The reference audio coverage and the detection audio coverage are determined based on each target matching segment in the chain of evidence; the reference audio coverage is the ratio of the combined coverage duration of the target matching segment on the target reference audio to the total duration of the target reference audio, and the detection audio coverage is the ratio of the combined coverage duration of the target matching segment on the target detection audio to the total duration of the target detection audio. The cover song relationship between the target audio to be detected and the target reference audio is identified based on the similarity, the reference audio coverage, and the audio coverage to be detected.

2. The method for identifying cover song relationships according to claim 1, characterized in that, Also includes: The fundamental frequency of the target reference audio and the target audio to be detected are estimated respectively to obtain a first fundamental frequency sequence of the target reference audio and a second fundamental frequency sequence of the target audio to be detected; The first fundamental frequency sequence is converted into a first pitch sequence, and the second fundamental frequency sequence is converted into a second pitch sequence; Extract the first harmonic features of the target reference audio and the second harmonic features of the target audio to be detected; Harmonic features are the characteristics that characterize the harmonic structure of audio. A first melody skeleton of the target reference audio is generated based on the first pitch sequence, and a second melody skeleton of the target audio to be detected is generated based on the second pitch sequence, so as to determine the similarity between the reference segment and each of the initial candidate segments based on the first harmonic features of the target reference audio, the first melody skeleton, the second harmonic features of the target audio to be detected, and the second melody skeleton.

3. The method for identifying cover song relationships according to claim 2, characterized in that, The melody skeleton includes a trend skeleton and an interval skeleton; Accordingly, generating a first melody skeleton of the target reference audio based on the first pitch sequence, and generating a second melody skeleton of the target audio to be detected based on the second pitch sequence, includes: A first target note node is generated based on the first pitch sequence, and a second target note node is generated based on the second pitch sequence; the target note node is a note node corresponding to consecutive frames with consistent pitch values. A first trend skeleton and a first interval skeleton of the target reference audio are generated based on the first target note node, and a second trend skeleton and a second interval skeleton of the target audio to be detected are generated based on the second target note node. The trend skeleton is used to describe the direction of pitch change between adjacent target note nodes, and the interval skeleton is used to describe the amplitude of pitch change between adjacent target note nodes.

4. The method for identifying cover song relationships according to claim 1, characterized in that, The process of dividing the target audio to be detected into several initial candidate segments based on preset singing conditions includes: Determine the appropriate speed ratio based on the preset singing conditions; For any of the reference segments, a reference duration is determined, and a corresponding candidate segment duration is determined based on the product of the reference duration and the speed multiplier. The candidate segment duration is then used to slide and truncate the target audio to be detected to obtain each of the initial candidate segments corresponding to each of the reference segments. Accordingly, determining the corresponding speed multiplier based on the preset singing conditions includes: For a preset fast singing mode, a first speed multiplier is generated; the first speed multiplier is less than 1. For a preset normal singing speed, a second speed multiplier is generated; the second speed multiplier is equal to 1. For a preset slow singing speed, a third speed multiplier is generated; the third speed multiplier is greater than 1.

5. The method for identifying cover song relationships according to claim 3, characterized in that, The step of determining the similarity between the reference segment and each of the initial candidate segments based on the first harmonic features of the target reference audio, the first melody skeleton, and the second harmonic features and the second melody skeleton of the target audio to be detected includes: The first target harmonic feature, the first target trend skeleton, and the first target interval skeleton of the reference segment are determined from the first harmonic feature, the first trend skeleton, and the first interval skeleton of the target reference audio. From the second harmonic features, the second trend skeleton, and the second interval skeleton of the target audio to be detected, determine the second target harmonic features, the second target trend skeleton, and the second target interval skeleton of each of the initial candidate segments; Determine a first similarity between the first target harmonic feature and the second target harmonic feature of the initial candidate segment; Determine the first degree of overlap between the first target trend skeleton and the second target trend skeleton of the initial candidate segment on a continuous sequence of a first preset order; Determine the second degree of overlap between the first target interval skeleton and the second target interval skeleton of the initial candidate segment on a continuous sequence of a second preset order; The first similarity, the first overlap, and the second overlap are weighted and summed to obtain the first target similarity. Then, a number of first candidate segments are selected from each of the initial candidate segments according to the order of the first target similarity, so as to select a number of initial matching segments from the first candidate segments.

6. The method for identifying cover song relationships according to claim 5, characterized in that, The step of selecting a plurality of initial matching segments from the first candidate segments includes: The second similarity between the first target trend skeleton and the second target trend skeleton of the first candidate segment is determined based on the dynamic time warping algorithm, and the third similarity between the first target interval skeleton and the second target interval skeleton of the first candidate segment is determined based on the dynamic time warping algorithm. The trend skeleton similarity is determined based on the second similarity, and the interval skeleton similarity is determined based on the third similarity. The similarity between the chroma vectors of the first target harmony features and the second target harmony features of the first candidate segment is determined as the harmony structure similarity. The trend skeleton similarity, the interval skeleton similarity, and the harmonic structure similarity are weighted and summed to obtain the second target similarity between the reference segment and the first candidate segment; Based on the second target similarity, several initial matching segments that meet the preset similarity conditions are selected from each of the first candidate segments.

7. The method for identifying cover song relationships according to any one of claims 1 to 6, characterized in that, The cover song relationship includes existing cover song relationship, suspected cover song relationship, or no cover song relationship found; Accordingly, after identifying the cover song relationship between the target audio to be detected and the target reference audio based on the similarity, the reference audio coverage, and the audio coverage to be detected, the method further includes: If the cover song relationship between the target audio to be detected and the target reference audio is identified as existing or suspected to be existing, then based on each target matching segment in the evidence chain, the differences in singing style between the target audio to be detected and the target reference audio are analyzed; the differences in singing style include differences in high notes, differences in ending notes, and differences in vibrato amplitude.

8. A cover song relationship identification device, characterized in that, include: The audio segmentation module is used to divide the target reference audio into several reference segments, and to divide the target audio to be detected into several initial candidate segments based on a preset singing scenario. The preset singing scenarios include preset fast singing, preset normal speed singing, and preset slow singing. The matching segment selection module is used to select a number of initial matching segments from each of the initial candidate segments based on the similarity between the reference segment and each of the initial candidate segments for any one of the reference segments; The evidence chain generation module is used to select several target matching segments from each of the initial matching segments based on preset constraint rules, and generate an evidence chain based on each of the target matching segments; The coverage determination module is used to determine the reference audio coverage and the detection audio coverage based on each target matching segment in the evidence chain; the reference audio coverage is the ratio of the combined coverage duration of the target matching segment on the target reference audio to the total duration of the target reference audio, and the detection audio coverage is the ratio of the combined coverage duration of the target matching segment on the target detection audio to the total duration of the target detection audio. The cover song relationship identification module is used to identify the cover song relationship between the target audio to be detected and the target reference audio based on the similarity, the reference audio coverage, and the audio coverage to be detected.

9. An electronic device, characterized in that, The electronic device includes a processor and a memory; wherein the memory is used to store a computer program, which is loaded and executed by the processor to implement the cover song relationship identification method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store a computer program, which, when executed by a processor, implements the cover song relationship identification method as described in any one of claims 1 to 7.

Images