Music detection method, electronic device, storage medium and program product
By performing track separation and feature extraction on music audio and combining it with a neighborhood attention mechanism network, the problem of poor accuracy in detecting AI-generated music in existing technologies has been solved, achieving efficient recognition and accurate classification of AI-generated music.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-04
- Publication Date
- 2026-03-10
AI Technical Summary
In existing technologies, the accuracy of detecting music generated by unknown generative models is poor, making it difficult to effectively identify AI-generated music.
By separating the audio tracks of the music, and using feature extraction networks and neighborhood attention mechanisms, the features of multiple audio tracks are analyzed to capture subtle flaws in the cross-track coordination and temporal naturalness of AI-generated music, which are then accurately identified by a classifier.
It significantly improves the recognition accuracy of AI-generated music, avoids the insufficient detection accuracy caused by relying on specific generation model features, and improves the robustness and recognition accuracy of the model.
Smart Images
Figure CN121641071A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and in particular to a music detection method, electronic device, storage medium, and program product. Background Technology
[0002] With the deep penetration of generative artificial intelligence technology into the field of music creation, music generation models based on deep learning architecture can easily produce musical works. However, the use of copyrighted musical works for model training may infringe on the original works' reproduction rights. Imitating the style of a specific singer or existing works may trigger disputes regarding infringement and unfair competition.
[0003] Existing technologies identify whether music is generated by a model by detecting deterministic artifacts left by a specific generative model in the audio of the music.
[0004] However, for music generated by unknown generative models, the detection accuracy is poor because the characteristics of the artifacts generated by the generative model are not clear. Summary of the Invention
[0005] This application provides a music detection method, electronic device, storage medium, and program product, which can solve the problem of poor accuracy in recognizing music generated by AI models. The technical solution is as follows: On the one hand, a music detection method is provided, the method comprising: Obtain time-spectral data of multiple audio tracks in the music audio to be identified; For each audio track, the time-spectrum data of the audio track is sliced to obtain multiple slice data, and features are extracted from the multiple slice data based on the feature extraction network to obtain multiple first feature data corresponding to the audio track. Perform inter-track neighborhood attention processing on all first feature data corresponding to the multiple audio tracks to obtain second feature data corresponding to each first feature data; Based on the classification and prediction of the plurality of first feature data and the corresponding plurality of second feature data, the classification result of the music audio to be identified is obtained, wherein the classification result is used to indicate whether the music audio to be identified is generated by an artificial intelligence (AI) model.
[0006] In one possible implementation, the time-spectrum data of the audio track is sliced to obtain multiple slice data, including: The time-domain slicing process is performed on the time-spectrum data of the audio track to obtain multiple time-domain slice data corresponding to the audio track, and the frequency-domain slicing process is performed on the time-spectrum data of the audio track to obtain multiple frequency-domain slice data corresponding to the audio track.
[0007] In another possible implementation, the feature extraction network is used to extract features from the multiple slice data to obtain multiple first feature data corresponding to the audio track, including: Based on the temporal order of multiple time-domain slice data corresponding to the audio track and the frequency order of multiple frequency-domain slice data corresponding to the audio track, the multiple time-domain slice data and multiple frequency-domain slice data corresponding to the audio track are combined to obtain a slice data sequence. The sliced data sequence is input into a feature extraction network to obtain the feature data sequence corresponding to the audio track output by the feature extraction network, wherein the feature data sequence includes multiple first feature data.
[0008] In another possible implementation, the feature extraction network is a global attention mechanism network.
[0009] In another possible implementation, the step of performing inter-track neighborhood attention processing on all the first feature data corresponding to the plurality of audio tracks to obtain second feature data corresponding to each first feature data includes: The first feature data corresponding to multiple audio tracks are used to construct a first feature data matrix; The first feature data matrix is input into the neighborhood attention mechanism network to obtain the second feature data matrix output by the neighborhood attention mechanism network, wherein the second feature data matrix includes the second feature data corresponding to each first feature data.
[0010] In another possible implementation, in the first feature data matrix, the first feature data of the same audio track belong to the same row, and the first feature data of the same time or frequency in different audio tracks belong to the same column; or... In the first feature data matrix, the first feature data of the same audio track belong to the same column, and the first feature data of the same time or the same frequency in different audio tracks belong to the same row.
[0011] In another possible implementation, the method further includes: Obtain the original audio of the sample music and generate AI audio of the sample music based on the AI model; Construct a sample dataset, which includes multiple sample audios and training labels corresponding to each sample audio. The multiple sample audios include the original audio and the AI audio. The training labels corresponding to the original audio are non-AI generated, and the training labels corresponding to the AI audio are AI generated. In the sample dataset, the sample audio is obtained, and sample time-spectral data of multiple audio tracks in the sample audio are obtained; For each audio track, the sample time-spectrum data of the audio track is sliced to obtain multiple sample slice data. Based on the feature extraction network to be trained, features are extracted from the multiple sample slice data to obtain multiple first sample feature data corresponding to the audio track. Based on the neighbor attention mechanism network to be trained, inter-track neighborhood attention processing is performed on all first sample feature data corresponding to the multiple audio tracks to obtain second sample feature data corresponding to each first sample feature data. Based on the classifier to be trained, the multiple first sample feature data and the corresponding multiple second sample feature data are classified and predicted to obtain the classification result of the sample audio. The classification result of the sample audio is used to indicate whether the sample audio is generated by an artificial intelligence (AI) model. Based on the classification results of the sample audio and the training labels corresponding to the sample audio, the feature extraction network, the neighbor attention mechanism network, and the classifier to be trained are trained to obtain the trained feature extraction network, the trained neighbor attention mechanism network, and the trained classifier.
[0012] On the other hand, a music detection device is provided, the device comprising: The acquisition module is configured to acquire time-spectral data of multiple audio tracks in the music audio to be identified.
[0013] The extraction module is configured to slice the time-spectrum data of each audio track to obtain multiple slice data, and extract features from the multiple slice data based on a feature extraction network to obtain multiple first feature data corresponding to the audio track.
[0014] The processing module is configured to perform inter-track neighborhood attention processing on all first feature data corresponding to the multiple audio tracks to obtain second feature data corresponding to each first feature data.
[0015] A classification module is configured to perform classification prediction based on the plurality of first feature data and the corresponding plurality of second feature data to obtain a classification result of the music audio to be identified, wherein the classification result is used to indicate whether the music audio to be identified is generated by an artificial intelligence AI model.
[0016] In one possible implementation, the extraction module is used for: The time-domain slicing process is performed on the time-spectrum data of the audio track to obtain multiple time-domain slice data corresponding to the audio track, and the frequency-domain slicing process is performed on the time-spectrum data of the audio track to obtain multiple frequency-domain slice data corresponding to the audio track.
[0017] In another possible implementation, the extraction module is used for: Based on the temporal order of multiple time-domain slice data corresponding to the audio track and the frequency order of multiple frequency-domain slice data corresponding to the audio track, the multiple time-domain slice data and multiple frequency-domain slice data corresponding to the audio track are combined to obtain a slice data sequence. The sliced data sequence is input into a feature extraction network to obtain the feature data sequence corresponding to the audio track output by the feature extraction network, wherein the feature data sequence includes multiple first feature data.
[0018] In another possible implementation, the processing module is used for: The first feature data corresponding to multiple audio tracks are used to construct a first feature data matrix; The first feature data matrix is input into the neighborhood attention mechanism network to obtain the second feature data matrix output by the neighborhood attention mechanism network, wherein the second feature data matrix includes the second feature data corresponding to each first feature data.
[0019] The music detection device also includes: The generation module is configured to acquire the original audio of the sample music and generate AI audio of the sample music based on an AI model. The training module is configured to construct a sample dataset, which includes multiple sample audios and training labels corresponding to each sample audio. The multiple sample audios include the original audio and the AI audio. The training labels corresponding to the original audio are non-AI generated, and the training labels corresponding to the AI audio are AI generated. In the sample dataset, the sample audio is obtained, and sample time-spectral data of multiple audio tracks in the sample audio are obtained; For each audio track, the sample time-spectrum data of the audio track is sliced to obtain multiple sample slice data. Based on the feature extraction network to be trained, features are extracted from the multiple sample slice data to obtain multiple first sample feature data corresponding to the audio track. Based on the neighbor attention mechanism network to be trained, inter-track neighborhood attention processing is performed on all first sample feature data corresponding to the multiple audio tracks to obtain second sample feature data corresponding to each first sample feature data. Based on the classifier to be trained, the multiple first sample feature data and the corresponding multiple second sample feature data are classified and predicted to obtain the classification result of the sample audio. The classification result of the sample audio is used to indicate whether the sample audio is generated by an artificial intelligence (AI) model. Based on the classification results of the sample audio and the training labels corresponding to the sample audio, the feature extraction network, the neighbor attention mechanism network, and the classifier to be trained are trained to obtain the trained feature extraction network, the trained neighbor attention mechanism network, and the trained classifier.
[0020] On the other hand, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the program to implement the method described in any of the above.
[0021] On the other hand, a non-transitory computer-readable storage medium is provided, the non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method described in any of the preceding claims.
[0022] On the other hand, a computer program product is provided, including computer program instructions that, when run on a computer, cause the computer to perform the method described in any of the preceding claims.
[0023] The beneficial effects of the technical solution provided in this application are as follows: by separating the audio tracks of music and analyzing the features of multiple tracks, richer acoustic features can be extracted from independent tracks such as different instruments and vocals. This allows for the capture of subtle flaws that may exist in the cross-track coordination and temporal naturalness of AI-generated music. It avoids the problem that artifact detection technology relies on the features of specific generation models and has poor accuracy in detecting music generated by unknown generation models, thus significantly improving the accuracy of AI-generated music recognition. Attached Figure Description
[0024] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0025] Figure 1 This is a schematic diagram of an implementation environment provided in an embodiment of this application; Figure 2 This is a flowchart of the music detection method provided in the embodiments of this application; Figure 3 This is the principle of the music detection method provided in the embodiments of this application. Figure 1 ; Figure 4 This is the principle of the music detection method provided in the embodiments of this application. Figure 2 ; Figure 5 This is the principle of the music detection method provided in the embodiments of this application. Figure 3 ; Figure 6 This is a schematic diagram of the music detection device structure provided in the embodiments of this application. Figure 1 ; Figure 7 This is a schematic diagram of the music detection device structure provided in the embodiments of this application. Figure 2 ; Figure 8 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation
[0026] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.
[0027] This disclosure provides a music detection method. This method can be applied to a terminal. For example... Figure 1 As shown, the terminal may include a processor 110, a memory 120, and a communication component 130.
[0028] Processor 110 can be a central processing unit (CPU), graphics processing unit (GPU), microcontroller unit (MCU), accelerated processing unit (APU), neural processing unit (NPU), tensor processing unit (TPU), field-programmable gate array (FPGA), application-specific integrated circuit (ASIC), digital signal processor (DSP), etc. Processor 110 can be used to acquire time-spectrum data, run neighborhood attention mechanism networks, and so on.
[0029] The memory 120 may include volatile memory and / or non-volatile memory. Volatile memory may include random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), etc. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, non-volatile random access memory (NVRAM), etc. The memory 120 can be used to store music audio, time-frequency data, etc.
[0030] The communication component 130 can be a wireless communication module (WCM), a subscriber identity module (SIM), a universal subscriber identity module (USIM), an optical network unit (ONU), etc. The communication component 130 can be used for communication between the terminal and a server, or other terminals.
[0031] This application provides a music detection method, such as... Figure 2 As shown, in some embodiments, the method includes: S201. Obtain the time-spectrum data of multiple audio tracks in the music audio to be identified.
[0032] The time-spectral data is a Mel spectrogram, obtained by acquiring the music audio to be identified and then dividing the audio into tracks based on the Demucs (Deep Extractor for Music Sources) model (e.g.,...). Figure 3As shown, the audio is divided into Drum, Vocal, Bass, and Other tracks, resulting in multiple audio tracks. These are then processed using a short-time Fourier transform. A Mel spectrogram is a two-dimensional data representation that simultaneously displays the frequency components of an audio signal and their changes over time. In a Mel spectrogram, the X-axis represents time, the Y-axis represents frequency, and the color or brightness of a pixel represents the energy intensity (amplitude) at the corresponding time and frequency.
[0033] S202. For each audio track, the time-spectrum data of the audio track is sliced to obtain multiple slice data, and features are extracted from the multiple slice data based on the feature extraction network to obtain multiple first feature data corresponding to the audio track.
[0034] Temporal slicing divides the Mel spectrum into continuous or overlapping short time intervals along the time axis. For example, a track lasting several seconds can be divided into segments of hundreds of milliseconds. Frequency slicing divides the spectrum into different frequency bands along the frequency axis.
[0035] In specific implementation, such as Figure 4The Mel spectrogram of the audio track can be sliced in the time domain to obtain time-domain slices. These time-domain slices are then reduced in dimensionality using a time-domain tokenizer to obtain multiple time-domain slice data, i.e., multiple time-domain tokens, corresponding to the audio track. Similarly, the Mel spectrogram of the audio track can be sliced in the frequency domain to obtain frequency-domain slices. These frequency-domain slices are then reduced in dimensionality using a frequency-domain tokenizer to obtain multiple frequency-domain slice data, i.e., multiple frequency-domain tokens, corresponding to the audio track. Multiple slice data, i.e., time-frequency tokens, can be obtained by simultaneously slicing the time-spectrum data of the audio track in both the time and frequency domains, or by slicing only the time-spectrum data of the audio track in either the time or frequency domain (not shown in the figure). The feature extraction network can be a global attention mechanism network (e.g., a Transformer architecture). A global attention mechanism network can evaluate the importance of information at different positions in multiple time-domain slice data and multiple frequency-domain slice data, and capture long-range dependencies. For signals with strong temporal correlation, such as music, this is crucial. Therefore, before inputting multiple time-domain and frequency-domain slices into the global attention mechanism network, slices from the same audio track can be arranged in rows according to time order and frequency from low to high. Slices from different tracks at the same time or frequency position can be arranged in columns. To enable the global attention mechanism network to understand the positional relationship of multiple time-domain and frequency-domain slices in the Mel spectrogram, positional encoding is added to the multiple time-domain and frequency-domain slices. These position-encoded time-frequency tokens are then input into the global attention mechanism network. The network calculates and generates a corresponding feature vector containing rich contextual information for each input slice, i.e., the first feature data, such as... Figure 3 As shown, the time-spectral data of multiple audio tracks are processed by a word segmenter and a feature extraction network (global attention mechanism network) to obtain the first feature data of each audio track.
[0036] S203. Perform inter-track neighborhood attention processing on all the first feature data corresponding to the multiple audio tracks to obtain the second feature data corresponding to each first feature data.
[0037] The processing procedure for this step can be as follows: First, the first feature data corresponding to multiple audio tracks are used to construct a first feature data matrix.
[0038] Among them, such as Figure 5In the first feature data matrix, the first feature data of the same audio track belongs to the same row, and the first feature data of the same time or the same frequency in different audio tracks belongs to the same column; or, in the first feature data matrix, the first feature data of the same audio track belongs to the same column, and the first feature data of the same time or the same frequency in different audio tracks belongs to the same row (not shown in the figure). This embodiment uses the example of the first feature data of the same audio track belonging to the same row and the first feature data of the same time or the same frequency in different audio tracks belonging to the same column for explanation.
[0039] Then, the first feature data matrix is input into the neighborhood attention mechanism network to obtain the second feature data matrix output by the neighborhood attention mechanism network, wherein the second feature data matrix includes the second feature data corresponding to each first feature data.
[0040] See Figure 5 The working mechanism of the neighborhood attention mechanism network is as follows: The general idea is to perform neighborhood aggregation on each element in the feature matrix. That is, for each element in the input first feature data matrix (i.e., the first feature data), it is combined with the elements in its local neighborhood in the feature matrix (the elements in the local neighborhood are elements within a specified range centered on the element) to calculate the corresponding neighborhood aggregation element. Finally, all the neighborhood aggregation elements are combined to form the second feature data matrix, which is the final output result. The neighborhood attention mechanism network learns the correlation between features of different audio tracks at the same time or frequency (such as the harmony between drum tracks and bass tracks) or the evolution pattern of features of the same audio track in adjacent time periods, thereby identifying potential cross-track incoherence or temporal unnaturalness in AI-generated music. Finally, each first feature data is transformed into second feature data containing rich neighborhood context after this processing, resulting in the second feature data matrix, which provides a more discriminative input for the subsequent classifier.
[0041] S204. Based on the classification prediction of the plurality of first feature data and the corresponding plurality of second feature data, the classification result of the music audio to be identified is obtained, wherein the classification result is used to indicate whether the music audio to be identified is generated by an artificial intelligence AI model.
[0042] In specific implementation, such as Figure 3The classification prediction based on the multiple first feature data and corresponding multiple second feature data is achieved by fusing feature representations from different levels and perspectives to construct a highly discriminative decision function, thereby accurately distinguishing whether the music audio to be identified was generated by an AI model. The first feature data carries the basic acoustic patterns of local temporal and frequency domain slices of each track, while the second feature data incorporates cross-track contextual information through a neighborhood attention mechanism. The input of the classifier (e.g., it can be a support vector machine, a fully connected neural network, or multiple base classifiers combined through weighted averaging or other fusion learning methods) can usually be integrated into a composite feature vector through an early fusion strategy (such as feature concatenation). Based on the classifier, global inconsistencies that may exist between tracks in AI-generated music (such as logical deviations in rhythm or harmony between different instrument tracks) can be captured simultaneously, thereby obtaining the classification result of the music audio to be identified, i.e., whether the music audio to be identified was generated by an artificial intelligence AI model.
[0043] In this embodiment, multi-track time-frequency analysis, hierarchical feature extraction, and fusion decision-making mechanisms significantly improve the recognition accuracy and robustness of AI-generated music. This avoids the passive nature of artifact detection technology, which lags behind AI technology development due to its reliance on specific generative model features. Employing a multi-track processing flow based on the Demucs music source separation model, richer acoustic features can be extracted from independent tracks such as melody, harmony, and rhythm, thereby capturing subtle flaws that may exist in the cross-track coordination and temporal naturalness of AI-generated music. Through time-domain and frequency-domain slicing processing, combined with global attention and neighborhood attention mechanisms, multi-level feature mining from local acoustic patterns to cross-track contextual associations is achieved. This effectively integrates the discrimination capabilities of different features, improves the model's expressive power and discriminative ability, and thus significantly improves classification accuracy and enhances the model's robustness to noise and variation.
[0044] In some embodiments, the step of extracting features from the plurality of slice data based on a feature extraction network to obtain a plurality of first feature data corresponding to the audio track includes: Based on the temporal order of multiple time-domain slice data corresponding to the audio track and the frequency order of multiple frequency-domain slice data corresponding to the audio track, the multiple time-domain slice data and multiple frequency-domain slice data corresponding to the audio track are combined to obtain a slice data sequence; the slice data sequence is input into a feature extraction network to obtain the feature data sequence corresponding to the audio track output by the feature extraction network, wherein the feature data sequence includes multiple first feature data.
[0045] In practice, the time-domain slice data is arranged in chronological order, allowing the feature extraction network to capture the local temporal patterns of the music signal (such as the onset, duration, and decay of notes), while the frequency-domain slice data is arranged from low to high frequency to characterize the energy distribution in different frequency bands (such as the fundamental frequency and harmonic structure). By combining the time-domain slice data sequence and the frequency-domain slice data sequence according to preset rules, such as alternating between time-domain slice data and frequency-domain slice data, or concatenating the time-domain slice data and frequency-domain slice data end to end, that is, concatenating the frequency-domain slice data sequence after the time-domain slice data sequence, a slice data sequence is formed. After the slice data sequence is input into the feature extraction network, the network gradually learns the long-range dependencies between slices through multi-layer nonlinear transformations, and finally outputs a feature data sequence, where each first feature data is a high-dimensional abstract representation of the original slice, which significantly enhances the sensitivity of the feature extraction network to possible time-frequency anomalies (such as unnatural harmonic transitions or rhythmic breaks) in AI-generated music.
[0046] In this embodiment, time-domain and frequency-domain slice data are combined into a sequence according to preset rules and input into the feature extraction network. By fusing multi-dimensional features, a more discriminative audio representation space is constructed, thereby significantly improving the ability to capture subtle time-frequency anomalies in AI-generated music. The temporal slices are arranged in chronological order, enabling the network to learn the local temporal patterns of music (such as the transient characteristics of note initiation, energy stability during sustained phases, and the natural transition of decay processes), while the frequency-domain slices are arranged in frequency order, directly representing the energy distribution under different frequency bands (such as the rationality of the fundamental frequency and harmonic structure). By combining them into a sequence through alternating splicing or first-to-last splicing, each input unit simultaneously contains time-frequency correlation information, providing a richer context for the feature extraction network. Based on this, the feature extraction network learns the long-range dependencies between slices through multi-layer nonlinear transformations, and outputs a high-dimensional abstract representation of the original slices. This not only strengthens the model's ability to model the time-frequency consistency of normal music signals, but also makes it highly sensitive to artifacts common in AI-generated music (such as unnatural harmonic transitions, rhythmic breaks, or abnormal cross-track coordination). This is because artifacts often expose statistical differences in local slices in the time or frequency domains, and serialization can amplify the contribution weight of these differences through an attention mechanism, thereby achieving more accurate differentiation in subsequent classification.
[0047] In some embodiments, the method further includes: Obtain the original audio of the sample music, generate AI audio of the sample music based on the AI model, set the training labels of the original audio to be generated by the non-AI model, and set the training labels of the AI audio to be generated by the AI model.
[0048] In practice, the first step is to collect original human-generated music (i.e., sample music) covering different styles (such as classical, pop, etc.) and instrument configurations as raw audio to ensure the representativeness and breadth of the data. AI models are then used to learn from and recreate these raw audio samples, generating corresponding AI audio. For example, the MusicGen model generates music based on the textual descriptions of the sample music (such as structured lyrics and descriptions of musical style) or the raw audio. The raw audio is labeled "not generated by AI model," and the AI-generated audio is labeled "generated by AI model," thus providing clear criteria for supervised learning.
[0049] Based on the original audio, the training labels of the original audio, the AI audio, and the training labels of the AI audio, the feature extraction network, the neighborhood attention mechanism network, and the classifier to be trained are trained to obtain the trained feature extraction network, the trained neighborhood attention mechanism network, and the trained classifier.
[0050] In practice, preprocessed multi-track time-spectral data is input into a feature extraction network to be trained. This network is responsible for extracting basic features from time-domain and frequency-domain slices of each track, generating first feature data. Then, the first feature data of each track is input into a neighborhood attention mechanism network to be trained. This network captures subtle cross-track correlations by focusing on the local neighborhood relationships of features in time, frequency, or track dimensions, generating corresponding second feature data rich in contextual information for each first feature data. The first and second feature data are then fed together into a classifier (such as a fully connected neural network) to be trained. The classifier outputs a predicted probability, indicating the likelihood that the input audio is judged as AI-generated. By comparing the classifier's predicted output with the true label (non-AI generated or AI-generated), a pre-defined loss function (such as cross-entropy loss) is used to calculate the difference between the predicted and true values (loss value). Through a backpropagation algorithm, the loss value is backpropagated layer by layer to the feature extraction network, the neighborhood attention mechanism network, and the classifier, calculating the contribution (gradient) of each parameter (weight and bias) in the network to the final loss. The optimizer (such as the Adam optimizer) updates all parameters in the three training groups simultaneously based on the calculated gradient, following the principle of gradient descent (new weights = old weights - learning rate × gradient), so that the entire model can reduce the loss in the next prediction. This process is repeated until the model's loss value is reduced to a predetermined threshold or the performance stabilizes (e.g., accuracy > 95%). At this point, the feature extraction network, the neighborhood attention mechanism network, and the classifier are considered to have been trained.
[0051] In this embodiment, a training set is constructed based on clearly labeled original audio (non-AI generated) and AI-generated audio, providing the model with a clear learning objective. This enables the feature extraction network to effectively capture subtle artifacts in the time-frequency domain of AI music (such as unnatural harmonic transitions or rhythmic breaks). The neighborhood attention mechanism further enhances the recognition of cross-track incoherence patterns by analyzing the spatial dependencies between audio tracks. During training, the difference between the predicted value and the true label is calculated using a loss function, and the network parameters are iteratively updated using a gradient descent algorithm, allowing the model to gradually converge to the optimal state. This enhances the model's ability to analyze complex musical features, ultimately achieving efficient and robust AI music detection.
[0052] All of the above-mentioned optional technical solutions can be combined in any way to form optional embodiments of this disclosure, and will not be described in detail here.
[0053] Based on the same inventive concept, corresponding to the music detection method provided in the embodiments of this application, this application also provides a music detection device.
[0054] refer to Figure 6 The music detection device includes: The acquisition module 601 is configured to acquire time-spectrum data of multiple audio tracks in the music audio to be identified.
[0055] The extraction module 602 is configured to slice the time-spectrum data of each audio track to obtain multiple slice data, and extract features from the multiple slice data based on a feature extraction network to obtain multiple first feature data corresponding to the audio track.
[0056] The processing module 603 is configured to perform inter-track neighborhood attention processing on all first feature data corresponding to the plurality of audio tracks to obtain second feature data corresponding to each first feature data.
[0057] The classification module 604 is configured to perform classification prediction based on the plurality of first feature data and the corresponding plurality of second feature data to obtain the classification result of the music audio to be identified, wherein the classification result is used to indicate whether the music audio to be identified is generated by an artificial intelligence AI model.
[0058] In one possible implementation, the extraction module 602 is used for: The time-domain slicing process is performed on the time-spectrum data of the audio track to obtain multiple time-domain slice data corresponding to the audio track, and the frequency-domain slicing process is performed on the time-spectrum data of the audio track to obtain multiple frequency-domain slice data corresponding to the audio track.
[0059] In another possible implementation, the extraction module 602 is used for: Based on the temporal order of multiple time-domain slice data corresponding to the audio track and the frequency order of multiple frequency-domain slice data corresponding to the audio track, the multiple time-domain slice data and multiple frequency-domain slice data corresponding to the audio track are combined to obtain a slice data sequence. The sliced data sequence is input into a feature extraction network to obtain the feature data sequence corresponding to the audio track output by the feature extraction network, wherein the feature data sequence includes multiple first feature data.
[0060] In another possible implementation, the processing module 603 is used for: The first feature data corresponding to multiple audio tracks are used to construct a first feature data matrix; The first feature data matrix is input into the neighborhood attention mechanism network to obtain the second feature data matrix output by the neighborhood attention mechanism network, wherein the second feature data matrix includes the second feature data corresponding to each first feature data.
[0061] refer to Figure 7 The music detection device further includes: The generation module 701 is configured to acquire the original audio of the sample music and generate AI audio of the sample music based on the AI model. Training module 702 is configured to construct a sample dataset, the sample dataset including multiple sample audios and training labels corresponding to each sample audio, the multiple sample audios including the original audio and the AI audio, the training labels corresponding to the original audio are non-AI generated, and the training labels corresponding to the AI audio are AI generated; In the sample dataset, the sample audio is obtained, and sample time-spectral data of multiple audio tracks in the sample audio are obtained; For each audio track, the sample time-spectrum data of the audio track is sliced to obtain multiple sample slice data. Based on the feature extraction network to be trained, features are extracted from the multiple sample slice data to obtain multiple first sample feature data corresponding to the audio track. Based on the neighbor attention mechanism network to be trained, inter-track neighborhood attention processing is performed on all first sample feature data corresponding to the multiple audio tracks to obtain second sample feature data corresponding to each first sample feature data. Based on the classifier to be trained, the multiple first sample feature data and the corresponding multiple second sample feature data are classified and predicted to obtain the classification result of the sample audio. The classification result of the sample audio is used to indicate whether the sample audio is generated by an artificial intelligence (AI) model. Based on the classification results of the sample audio and the training labels corresponding to the sample audio, the feature extraction network, the neighbor attention mechanism network, and the classifier to be trained are trained to obtain the trained feature extraction network, the trained neighbor attention mechanism network, and the trained classifier.
[0062] It should be noted that the music detection device provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the music detection device and the music detection method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.
[0063] Based on the same inventive concept, corresponding to the music detection method provided in the embodiments of this application, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the music detection method described in the above embodiments.
[0064] Figure 8 This embodiment illustrates a more specific hardware structure of an electronic device, which may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.
[0065] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.
[0066] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.
[0067] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.
[0068] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).
[0069] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.
[0070] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.
[0071] The electronic device described above is used to implement the corresponding music detection method in the foregoing embodiments and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.
[0072] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including instructions that can be executed by a processor in a terminal to perform the music detection method described above. This computer-readable storage medium may be non-transitory. For example, the computer-readable storage medium may be ROM (Read-Only Memory), RAM (Random Access Memory), CD-ROM (Compact Disc Read-Only Memory), magnetic tape, floppy disk, and optical data storage devices, etc.
[0073] In an exemplary embodiment, a computer program product is also provided, including computer program instructions that, when executed on a computer, cause the computer to perform the music detection method described above.
[0074] It should be noted that the information (including but not limited to user equipment information, user personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals (including but not limited to signals transmitted between user terminals and other devices, etc.) involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.
[0075] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.
[0076] It should be understood that "multiple" as used herein refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. Furthermore, the step numbers described herein are merely illustrative of one possible execution order. In some other embodiments, the steps may not be executed in numerical order, such as two steps with different numbers being executed simultaneously, or two steps with different numbers being executed in the reverse order of the illustration. This application does not limit this.
[0077] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.
Claims
1. A music detection method characterized by, The method comprises: obtaining time-frequency spectrum data of a plurality of audio tracks in to-be-identified music audio; for each audio track, performing slicing processing on the time-frequency spectrum data of the audio track respectively to obtain a plurality of slice data, and performing feature extraction on the plurality of slice data based on a feature extraction network to obtain a plurality of first feature data corresponding to the audio track; performing inter-track neighborhood attention processing on all first feature data corresponding to the plurality of audio tracks to obtain second feature data corresponding to each first feature data; based on classification prediction on the plurality of first feature data and the corresponding plurality of second feature data, obtaining a classification result of the to-be-identified music audio, wherein the classification result is used to indicate whether the to-be-identified music audio is generated by an artificial intelligence (AI) model.
2. The music detection method of claim 1, wherein, The slicing processing on the time-frequency spectrum data of the audio track respectively to obtain a plurality of slice data comprises: performing time domain slicing processing on the time-frequency spectrum data of the audio track to obtain a plurality of time domain slice data corresponding to the audio track, and performing frequency domain slicing processing on the time-frequency spectrum data of the audio track to obtain a plurality of frequency domain slice data corresponding to the audio track.
3. The music detection method of claim 2, wherein, The feature extraction on the plurality of slice data based on the feature extraction network to obtain a plurality of first feature data corresponding to the audio track comprises: combining the plurality of time domain slice data and the plurality of frequency domain slice data corresponding to the audio track in a time sequence of the plurality of time domain slice data and a frequency sequence of the plurality of frequency domain slice data to obtain a slice data sequence; inputting the slice data sequence into the feature extraction network to obtain a feature data sequence output by the feature extraction network, wherein the feature data sequence comprises a plurality of first feature data.
4. The music detection method of claim 3, wherein, The feature extraction network is a global attention mechanism network.
5. The music detection method of claim 1, wherein, The inter-track neighborhood attention processing on all first feature data corresponding to the plurality of audio tracks to obtain second feature data corresponding to each first feature data comprises: constructing the first feature data corresponding to the plurality of audio tracks into a first feature data matrix; inputting the first feature data matrix into a neighborhood attention mechanism network to obtain a second feature data matrix output by the neighborhood attention mechanism network, wherein the second feature data matrix comprises second feature data corresponding to each first feature data.
6. The music detection method of claim 5, wherein, In the first feature data matrix, the first feature data of the same audio track belongs to the same row, and the first feature data of the same time or the same frequency in different audio tracks belongs to the same column; or in the first feature data matrix, the first feature data of the same audio track belongs to the same column, and the first feature data of the same time or the same frequency in different audio tracks belongs to the same row.
7. The music detection method according to any one of claims 1 to 6, characterized in that, The method further comprises: obtaining original audio of sample music, and generating AI audio of the sample music based on an AI model; constructing a sample data set, the sample data set comprising a plurality of sample audio and a training label corresponding to each sample audio, the plurality of sample audio comprising the original audio and the AI audio, the training label corresponding to the original audio being non-AI generated, and the training label corresponding to the AI audio being AI generated; In the sample data set, the sample audio is obtained, and sample time-frequency spectrum data of multiple audio tracks in the sample audio is obtained; For each audio track, the sample time-frequency spectrum data of the audio track is respectively sliced to obtain multiple sample slice data, and the multiple sample slice data is subjected to feature extraction based on a to-be-trained feature extraction network to obtain multiple first sample feature data corresponding to the audio track; Based on a to-be-trained neighbor attention mechanism network, all first sample feature data corresponding to the multiple audio tracks is subjected to inter-track neighborhood attention processing to obtain second sample feature data corresponding to each first sample feature data; Based on a to-be-trained classifier, the multiple first sample feature data and the corresponding multiple second sample feature data are subjected to classification prediction to obtain a classification result of the sample audio, wherein the classification result of the sample audio is used to indicate whether the sample audio is generated by an artificial intelligence AI model; Based on the classification result of the sample audio and a training label corresponding to the sample audio, the to-be-trained feature extraction network, the to-be-trained neighbor attention mechanism network and the to-be-trained classifier are trained to obtain a trained feature extraction network, a trained neighbor attention mechanism network and a trained classifier.
8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, The processor implements the method of any one of claims 1 to 7 when executing the program.
9. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to make the computer execute the method of any one of claims 1 to 7.
10. A computer program product comprising computer program instructions, characterised in that, When the computer program instructions run on the computer, the computer is caused to execute the method of any one of claims 1 to 7. When the computer program instructions run on the computer, the computer is caused to execute the method of any one of claims 1 to 7.