Music data identification method and device based on multiple parts, equipment and storage medium

Abstract

The invention discloses a music data identification method and device based on multiple sound parts, equipment and a storage medium. The method comprises the steps that music data is collected and then preprocessed, noise interference is removed, the music data is converted into a spectrogram through multiple kinds of time-frequency analysis, features are extracted through a deep learning algorithm fusing CNN, RNN and an attention mechanism, then sound parts are classified and labeled, adaptive noise elimination and a filter are adopted in preprocessing, and volume normalization is carried out. The device comprises a data acquisition module, a preprocessing module, a time-frequency analysis module, a feature extraction module and a classification labeling module, and the equipment comprises a processor and a memory to execute the method. The method is also realized when the program stored in the storage medium is executed; through adaptive denoising, a multi-analysis method, a composite deep learning model and a classification labeling technology, noise is effectively removed, sound features are clearly displayed, features are accurately extracted, sound functions are determined, processing efficiency is improved, large-scale real-time processing requirements are met, and digital development of the music industry is assisted.

Description

Technical Field

[0001] The present invention relates to the technical field of cross-border e-commerce security systems, and in particular to a method, device, equipment and storage medium for identifying polyphonic music data. Background Art

[0002] With the continuous development of the music industry and the acceleration of the digitalization process, the demand for the analysis and processing of music data is growing. As a rich and complex musical form, polyphonic music occupies an important position in music creation, performance and appreciation. However, traditional music data recognition methods face many challenges when processing polyphonic music.

[0003] On the one hand, the mutual interference between different parts in polyphonic music makes it difficult to accurately identify the note information and melody direction of each part. Traditional audio processing methods often have difficulty in effectively separating different parts, resulting in inaccurate recognition results. In addition, noise interference in polyphonic music can also seriously affect the recognition effect. Traditional denoising methods may not be able to completely remove complex noise, thus affecting subsequent analysis and processing.

[0004] On the other hand, traditional music data recognition methods are usually based on a single feature extraction method, which makes it difficult to fully capture the complex characteristics of polyphonic music. For example, some methods only focus on the time domain features of the audio, while ignoring the frequency domain features; other methods only consider the local features of the music, while ignoring the overall structure and contextual information. This makes it impossible to fully explore the potential information in the music data when processing polyphonic music, resulting in low recognition accuracy.

[0005] At the same time, the existing music data recognition technology has low computational efficiency when facing large-scale music data, and it is difficult to meet the needs of real-time processing. Moreover, different types of music may have different characteristics and styles. Traditional methods often lack flexibility and adaptability, and it is difficult to effectively identify different types of music. Summary of the invention

[0006] The present invention proposes a method, device, equipment and storage medium for multi-part music data recognition to solve the problems mentioned in the above-mentioned prior art.

[0007] In order to achieve the above object, the present invention adopts the following technical solution: a method for identifying music data based on polyphonic parts, comprising the following steps:

[0008] S1. Collect multi-part music data, pre-process the music data, remove noise interference, introduce adaptive noise elimination technology, adjust the filter parameters in real time according to the characteristics of the music data, remove various complex noises, and normalize the volume to ensure that the volume of different music samples is at the same level;

[0009] S2. Use time-frequency analysis methods to convert music data into spectrograms to intuitively display the frequency characteristics of different parts. Use short-time Fourier transform and wavelet transform time-frequency analysis tools to automatically select the most appropriate analysis method according to the characteristics of the music, enhance the spectrogram, and highlight the energy distribution of different frequency ranges;

[0010] S3. Use deep learning algorithms to extract features from the spectrogram and identify the note information and melody direction of each part. A composite model of convolutional neural network (CNN), recurrent neural network (RNN) and attention mechanism is used. CNN is used to extract local features of the spectrogram, RNN is used to capture the temporal features of music, and the attention mechanism dynamically focuses on the important parts of different parts.

[0011] S4. Classify and label different parts according to the extracted features, determine the role and function of each part in the music, introduce the clustering algorithm in machine learning, classify the parts with similar features into one category, and label the parts of different categories in combination with music theory knowledge, including the main melody part, accompaniment part, and harmony part. Use multimodal information fusion technology and combine the audio characteristics, rhythm characteristics, and melody characteristics of the music for comprehensive judgment.

[0012] Preferably, the S1 step includes filtering the music data with a filter and normalizing the volume. The filter adopts an adaptive filter that can automatically adjust the filtering parameters according to the real-time changes of the music data. The volume normalization adopts a standardized method to adjust the volume of the music data to a specific range.

[0013] Preferably, the deep learning algorithm is a model that combines a convolutional neural network with a recurrent neural network, and an attention mechanism is also introduced into the model to increase the focus on important features.

[0014] Preferably, a device based on a polyphonic music data recognition method comprises:

[0015] The data acquisition module is used to collect multi-part music data and has several data acquisition interfaces for receiving music data from different sources, including audio files and real-time audio streams;

[0016] A preprocessing module performs denoising, filtering and normalization on the music data. The preprocessing module includes an adaptive noise elimination unit, a filter unit and a volume normalization unit, which respectively perform corresponding preprocessing operations;

[0017] The time-frequency analysis module converts music data into a spectrogram, automatically selects the appropriate time-frequency analysis method according to the characteristics of the music, and enhances the spectrogram;

[0018] The feature extraction module uses a deep learning algorithm to extract music features. The feature extraction module adopts a deep learning model that integrates CNN, RNN and attention mechanism;

[0019] The classification and labeling module classifies and labels different parts, and uses clustering algorithm and multimodal information fusion technology to classify and label the parts.

[0020] Preferably, the feature extraction module adopts a deep learning model that combines a convolutional neural network with a recurrent neural network, and introduces an attention mechanism to improve the accuracy of feature extraction.

[0021] Preferably, a polyphonic music data recognition device comprises a processor and a memory, wherein the memory stores a computer program, and the processor implements a polyphonic music data recognition method when executing the computer program.

[0022] Preferably, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, a method for identifying multi-part music data is implemented.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] The present invention can more effectively remove complex noise interference in the music data preprocessing stage by introducing adaptive noise elimination technology and multi-filter combination, provide purer music data for subsequent analysis, and improve recognition accuracy.

[0025] By integrating multiple time-frequency analysis methods and enhancing the spectrum diagram, the present invention can more comprehensively and clearly display the frequency characteristics of different parts, making the characteristics of each part more prominent, which is helpful to accurately identify the note information and melody direction.

[0026] The present invention adopts a composite deep learning model that integrates convolutional neural networks, recurrent neural networks and attention mechanisms, which can fully explore the spatiotemporal characteristics and important information of polyphonic music, improve the accuracy and comprehensiveness of feature extraction, and thus achieve more accurate music data recognition.

[0027] The present invention classifies and labels different parts by introducing clustering algorithms and multimodal information fusion technology, which can accurately determine the role and function of each part in music and provide more valuable information for music creation, analysis and appreciation.

[0028] The present invention improves the processing efficiency of polyphonic music data by optimizing the coordination of algorithms and hardware devices, can meet the real-time processing requirements of large-scale music data, and provides powerful technical support for the digital development of the music industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A schematic block diagram of a polyphonic music data recognition method proposed by the present invention. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0032] In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined. In addition, the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, and it can be the internal connection of two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances. The present invention will be further described in detail below in conjunction with the accompanying drawings.

[0033] Reference Figure 1 :A method for identifying music data based on polyphonic parts, comprising the following steps:

[0034] S1. Collect multi-part music data, pre-process the music data, remove noise interference, introduce adaptive noise elimination technology, adjust the filter parameters in real time according to the characteristics of the music data, remove various complex noises, and normalize the volume to ensure that the volume of different music samples is at the same level;

[0035] S2. Use time-frequency analysis methods to convert music data into spectrograms to intuitively display the frequency characteristics of different parts. Use short-time Fourier transform and wavelet transform time-frequency analysis tools to automatically select the most appropriate analysis method according to the characteristics of the music, enhance the spectrogram, and highlight the energy distribution of different frequency ranges;

[0036] S3. Use deep learning algorithms to extract features from the spectrogram and identify the note information and melody direction of each part. A composite model of convolutional neural network (CNN), recurrent neural network (RNN) and attention mechanism is used. CNN is used to extract local features of the spectrogram, RNN is used to capture the temporal features of music, and the attention mechanism dynamically focuses on the important parts of different parts.

[0037] S4. Classify and label different parts according to the extracted features, determine the role and function of each part in the music, introduce the clustering algorithm in machine learning, classify the parts with similar features into one category, and label the parts of different categories in combination with music theory knowledge, including the main melody part, accompaniment part, and harmony part. Use multimodal information fusion technology and combine the audio characteristics, rhythm characteristics, and melody characteristics of the music for comprehensive judgment.

[0038] In the present invention, the S1 step includes using a filter to filter the music data and normalizing the volume. The filter uses an adaptive filter that can automatically adjust the filter parameters according to the real-time changes of the music data. The volume normalization process uses a standardized method to adjust the volume of the music data to a specific range.

[0039] In the present invention, the deep learning algorithm used has significant innovation and high efficiency. The deep learning algorithm is a model combining convolutional neural networks (CNN) and recurrent neural networks (RNN).

[0040] Convolutional neural networks are good at extracting local features of data. Through the combined operation of convolutional layers and pooling layers, they can effectively capture the spatial structure information in music data. Its mathematical expression can be expressed as: where x i represents the element in the input feature map, w i is the corresponding convolution kernel weight, b is the bias term, M j represents a local receptive field of the input feature map, f is the activation function, and y is the output feature. In music data processing, CNN can extract local features such as note combinations and rhythm patterns in the spectrum map, providing a basis for subsequent analysis.

[0041] Recurrent neural networks can process sequence data. For music data with time series characteristics, RNN can well capture the temporal characteristics and dynamic changes of music. Its mathematical expression is: t =f(Ux t +Wh t-1 ), where x t Indicates the input at the current moment, h t-1 is the hidden state at the previous moment, U and W are the input weight matrix and the hidden state weight matrix respectively, f is the activation function, h t is the hidden state at the current moment. Through RNN, it is possible to remember the development of music in time series and capture the ups and downs of melody and changes in rhythm.

[0042] In order to further improve the focus on important features, the model also introduces an attention mechanism. The attention mechanism allows the model to dynamically focus on the importance of different parts when processing music data and allocate computing resources in a targeted manner. Its mathematical expression is usually expressed as: Where Q, K, and V represent the query vector, key vector, and value vector, respectively. k is the dimension of the key vector. By calculating the similarity between Q and K, the attention weight is obtained after normalization by the softmax function, and then the weighted sum is performed to highlight the important features. In the recognition of polyphonic music data, the attention mechanism can pay more attention to the parts with prominent melody or the areas with obvious rhythm changes, thereby improving the accuracy and reliability of feature extraction.

[0043] In the present invention, the data acquisition module plays a vital role in the entire device. It is equipped with several data acquisition interfaces and can widely receive music data from different channels, including audio files and real-time audio streams. For files in various audio formats, the data acquisition module can accurately read and collect. For real-time audio streams, it can efficiently connect with various audio input devices to obtain the music data being played or performed in real time, ensuring the timeliness and authenticity of the data.

[0044] The preprocessing module processes the music data in detail to improve its quality. Among them, the adaptive noise elimination unit adopts advanced signal processing algorithms, which can monitor the noise components in the music data in real time, and automatically adjust the elimination parameters according to the noise characteristics, effectively remove various noises, and provide a clean music signal for subsequent processing. The filter unit contains various types of filters, such as low-pass filters, high-pass filters, band-pass filters, etc., which can accurately filter according to the frequency characteristics of the music, remove unnecessary frequency components, and highlight useful music information. The volume normalization unit adjusts the volume of the music data through an intelligent algorithm, and adjusts the volume of music data from different sources to a uniform level to avoid the accuracy of subsequent processing affected by excessive volume differences.

[0045] The time-frequency analysis module plays a key role in music data processing. It can automatically select the appropriate time-frequency analysis method according to the specific characteristics of the music. For music with a brisk rhythm and large frequency changes, short-time Fourier transform may be selected to better capture the changes in frequency over time. For music with complex harmonic structures, wavelet transform is selected to more finely analyze the distribution of different frequency components. At the same time, the time-frequency analysis module also enhances the spectrum graph, and makes the frequency characteristics of different parts more clearly visible by adjusting parameters such as color and contrast.

[0046] The feature extraction module is one of the core parts of the whole device. It uses a deep learning model that combines convolutional neural network (CNN), recurrent neural network (RNN) and attention mechanism to fully explore the potential features of music data. CNN can extract local features of the spectrum and capture micro information such as notes and rhythm in music. RNN can process the temporal characteristics of music data and understand the melody direction and dynamic changes of music. The attention mechanism can dynamically allocate computing resources according to the importance of music and improve the accuracy of feature extraction.

[0047] The classification and annotation module is responsible for accurately classifying and annotating different parts. It uses clustering algorithms and multimodal information fusion technology. The clustering algorithm automatically groups different parts by calculating the feature similarity of music data. Multimodal information fusion technology combines multiple aspects of information such as audio features, rhythm features, and melody features of music to make a comprehensive judgment, more accurately determine the role and function of the parts in music, and annotate them accordingly.

[0048] In the present invention, the feature extraction module plays a vital role. The module adopts an innovative deep learning model, i.e., a combination of a convolutional neural network and a recurrent neural network. Convolutional neural networks are good at extracting local features of data and can effectively capture local patterns and texture information in music spectrograms. Recurrent neural networks can process sequence data, and for music, which has time series characteristics, it can well capture the temporal characteristics and dynamic changes of music. By combining these two networks, their respective advantages can be fully utilized to more comprehensively extract the features of polyphonic music data.

[0049] In addition, in order to further improve the accuracy of feature extraction, the feature extraction module also introduces an attention mechanism. The attention mechanism allows the model to dynamically pay attention to the importance of different parts when processing music data, and allocate computing resources in a targeted manner. For key parts in polyphonic music, such as parts with prominent melodies or areas with obvious rhythm changes, the attention mechanism can pay more attention to them, thereby more accurately extracting the features of these important parts. This can prevent the model from being disturbed by irrelevant information when processing complex polyphonic music data, improve the accuracy and reliability of feature extraction, and provide a more accurate basis for subsequent music data recognition and analysis.

[0050] In the present invention, a device for identifying music data based on polyphonic parts plays a key role. The device is mainly composed of a processor and a memory. The memory is a core component for data storage, in which a specially designed computer program is stored. This computer program is carefully compiled and covers various key steps and algorithms for identifying music data based on polyphonic parts.

[0051] When the processor executes the computer program in the memory, it can efficiently implement the polyphonic music data recognition method. The processor uses its powerful computing power to quickly process the input polyphonic music data. First, it will pre-process the music data according to the instructions in the program to remove various noise interferences and ensure the purity of the data. Then, the processor will call the time-frequency analysis method to convert the music data into an intuitive spectrogram to better display the frequency characteristics of different parts. Then, the deep learning algorithm is used to perform in-depth feature extraction on the spectrogram to accurately identify the note information and melody direction of each part. Finally, the different parts are classified and labeled according to the extracted features to determine the specific role and function of each part in the music.

[0052] This music data recognition device is highly flexible and adaptable, and can be widely used in many fields such as music creation, music analysis, and music education, providing strong technical support for the development of the music industry.

[0053] In the present invention, a computer-readable storage medium plays a vital role. The storage medium stores a well-designed computer program. When a processor executes the computer program, a method for identifying multi-voice-based music data can be efficiently implemented.

[0054] This storage medium can be in various forms such as hard disk, solid-state disk, USB flash drive, etc., with stable and reliable storage performance. The computer program is carefully written and optimized, and can accurately guide the processor to perform a series of complex processing operations on the input polyphonic music data. From data collection and preprocessing to remove noise interference, to the use of advanced time-frequency analysis methods to convert it into a spectrum diagram, and then using deep learning algorithms to extract features and classify and label different parts, the entire process is carried out in an orderly manner under the control of the computer program, providing a solid technical guarantee for the accurate identification of polyphonic music data.

[0055] The above are only preferred specific implementation modes of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical solutions and inventive concepts of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for identifying multi-voice music data, characterized in that: The following steps are involved: S1. Collect multi-part music data, pre-process the music data, remove noise interference, introduce adaptive noise elimination technology, adjust the filter parameters in real time according to the characteristics of the music data, remove various complex noises, and normalize the volume to ensure that the volume of different music samples is at the same level; S2. Use time-frequency analysis methods to convert music data into spectrograms to intuitively display the frequency characteristics of different parts. Use short-time Fourier transform and wavelet transform time-frequency analysis tools to automatically select the most appropriate analysis method according to the characteristics of the music, enhance the spectrogram, and highlight the energy distribution of different frequency ranges; S3. Use deep learning algorithms to extract features from the spectrogram and identify the note information and melody direction of each part. A composite model of convolutional neural network (CNN), recurrent neural network (RNN) and attention mechanism is used. CNN is used to extract local features of the spectrogram, RNN is used to capture the temporal features of music, and the attention mechanism dynamically focuses on the important parts of different parts. S4. Classify and label different parts according to the extracted features, determine the role and function of each part in the music, introduce the clustering algorithm in machine learning, classify the parts with similar features into one category, and label the parts of different categories in combination with music theory knowledge, including the main melody part, accompaniment part, and harmony part. Use multimodal information fusion technology and combine the audio characteristics, rhythm characteristics, and melody characteristics of the music for comprehensive judgment.

2. The method for identifying polyphonic music data according to claim 1, characterized in that: The S1 step includes filtering the music data with a filter and normalizing the volume. The filter adopts an adaptive filter, which can automatically adjust the filtering parameters according to the real-time changes of the music data. The volume normalization adopts a standardized method to adjust the volume of the music data to a specific range.

3. The method for identifying polyphonic music data according to claim 1, characterized in that: The deep learning algorithm is a model that combines a convolutional neural network with a recurrent neural network. The model also introduces an attention mechanism to increase the focus on important features.

4. A device for implementing the method for multi-part music data recognition as described in any one of claims 1 to 3, characterized in that: include: The data acquisition module is used to collect multi-part music data and has several data acquisition interfaces for receiving music data from different sources, including audio files and real-time audio streams; A preprocessing module performs denoising, filtering and normalization on the music data. The preprocessing module includes an adaptive noise elimination unit, a filter unit and a volume normalization unit, which respectively perform corresponding preprocessing operations; The time-frequency analysis module converts music data into a spectrogram, automatically selects the appropriate time-frequency analysis method according to the characteristics of the music, and enhances the spectrogram; The feature extraction module uses a deep learning algorithm to extract music features. The feature extraction module adopts a deep learning model that integrates CNN, RNN and attention mechanism; The classification and labeling module classifies and labels different parts, and uses clustering algorithm and multimodal information fusion technology to classify and label the parts.

5. The multi-part music data recognition device according to claim 4 is characterized in that: The feature extraction module adopts a deep learning model that combines a convolutional neural network with a recurrent neural network and introduces an attention mechanism to improve the accuracy of feature extraction.

6. A multi-voice music data recognition device, characterized in that: It comprises a processor and a memory, wherein the memory stores a computer program, and when the processor executes the computer program, the multi-part music data recognition method according to any one of claims 1 to 3 is implemented.

7. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method for identifying multi-part music data according to any one of claims 1 to 3 is implemented.

Images