Melody recognition and model training method, program product, device and storage medium
By using conditional random field models and multiple feature extraction methods in the melody recognition model, combining rhythm and treble annotation, the melody recognition model is optimized, and the problem of low accuracy of melody recognition in the existing technology is solved, which significantly improves the accuracy of melody recognition sex.
Patent Information
- Application Number
- CN202411297120.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-18
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-09-18
AI Technical Summary
The existing melody recognition methods may easily lead to the loss of melody rhythm information or the identification errors due to improper selection of window size, resulting in low accuracy of melody recognition results.
The conditional random field model is used for melody recognition model training. Feature extraction and feature optimization are performed on the training audio data, and the internal parameters of the melody recognition model are optimized to improve the accuracy of melody recognition.
Through the combination of conditional random field models and multiple feature extraction methods, the dependence between different feature data is effectively considered, the accuracy of the melody recognition model is improved, and the loss of melody rhythm information or identification errors caused by improper window size selection is avoided. Improve the accuracy of melody recognition results.
Smart Images

Figure CN118824216B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of audio processing technology, and in particular to melody recognition and its model training method, program product, device and storage medium. Background Art
[0002] Melody recognition is an audio processing technology that allows users to identify the name of a song, singer information, etc. by humming or playing a music melody. This technology is usually used in music recognition software or applications to help users identify the name of a song and singer information through the music melody playing around them, so that they can easily know the information of the song being played.
[0003] Most existing melody recognition methods first divide the continuous audio signal into short time frames through window division, and then further analyze and process each frame. Usually a fixed-length window is used to divide the signal by sliding the window on the audio signal. If the window size is not selected properly, it is easy to cause the melody rhythm information to be lost or the recognition error, which leads to low accuracy of the melody recognition result. Summary of the invention
[0004] In view of this, the purpose of the embodiments of the present application is to provide a melody recognition and model training method, program product, device and storage medium, so as to solve the technical problem that the melody recognition results obtained based on the existing melody recognition method have low accuracy.
[0005] In a first aspect, an embodiment of the present application provides a melody recognition model training method, the method comprising:
[0006] Perform feature extraction on the audio data to be trained to obtain feature data to be trained;
[0007] Inputting the feature data to be trained into the melody recognition model to be trained to obtain a model recognition result output by the melody recognition model to be trained; wherein the melody recognition model to be trained includes a conditional random field model;
[0008] According to the model recognition result and the melody labeling result of the audio data to be trained, the internal parameters of the melody recognition model to be trained are optimized to obtain a trained melody recognition model.
[0009] In the above implementation process, the melody recognition model training method extracts features from the audio data to be trained to obtain feature data to be trained; inputs the feature data to be trained into the melody recognition model to be trained to obtain the model recognition result output by the melody recognition model to be trained; and optimizes the internal parameters of the melody recognition model to be trained according to the model recognition result and the melody labeling result of the audio data to be trained to obtain a trained melody recognition model. Since the melody recognition model to be trained includes a conditional random field model, the conditional random field model can effectively take into account the dependencies between different feature data to be trained, thereby improving the accuracy of the model recognition result. The trained melody recognition model obtained based on the melody recognition model training method can perform melody recognition on the audio data to be recognized, and obtain a melody recognition result with higher accuracy. The technical problem of low accuracy of the melody recognition result obtained based on the existing melody recognition method is solved.
[0010] Optionally, in an embodiment of the present application, the feature extraction of the audio data to be trained to obtain the feature data to be trained includes: performing feature extraction on the audio data to be trained based on at least two feature extraction methods among short-time Fourier transform, Mel-frequency cepstral coefficients, constant Q transform or Chroma to obtain the feature data to be trained.
[0011] In the above implementation process, since different feature extraction methods can capture feature data of different aspects of the audio data to be trained, feature extraction of the audio data to be trained based on multiple feature extraction methods can improve the diversity of the feature data to be trained, thereby improving the melody recognition precision and accuracy of the trained melody recognition model.
[0012] Optionally, in an embodiment of the present application, before extracting features from the audio data to be trained to obtain the feature data to be trained, the method further includes: performing speed normalization processing on the audio data to be trained to obtain normalized data; extracting features from the audio data to be trained to obtain the feature data to be trained includes: extracting features from the normalized data to obtain the feature data to be trained.
[0013] In the above implementation process, by performing speed normalization processing on the audio data to be trained to ensure the speed consistency of the model training data, the melody recognition precision and accuracy of the melody recognition model obtained through training can be improved.
[0014] Optionally, in an embodiment of the present application, the inputting the melody feature data to be trained into the melody recognition model to be trained to obtain the model recognition result output by the melody recognition model to be trained includes: rhythmically marking and / or high-pitched marking on the melody feature data to be trained based on a preset marking system to obtain marked feature data; inputting the marked feature data into the melody recognition model to be trained to obtain the model recognition result output by the melody recognition model to be trained.
[0015] In the above implementation process, by marking the rhythm and / or high pitch of the melody feature data to be trained, the melody recognition model to be trained can more accurately capture the rhythm and high pitch features in the training data, thereby improving the melody recognition accuracy of the trained melody recognition model. Rhythm marking and / or high pitch marking based on a preset marking system can avoid the loss of melody rhythm information or recognition errors caused by improper window size selection, thereby improving the accuracy of the melody recognition result.
[0016] Optionally, in an embodiment of the present application, the preset marking system includes a BIES mark for implementing the rhythm marking and / or a MIDI treble mark for implementing the treble marking; wherein the BIES mark includes a blank note mark and a start mark, a middle mark and an end mark of each note; and the MIDI treble mark includes the treble value of each note.
[0017] Optionally, in an embodiment of the present application, the intermediate mark includes a plurality of different intermediate marks that are sequentially adjacent to the start mark.
[0018] In the above implementation process, since the middle mark includes multiple different middle marks adjacent to the start mark in sequence, the diversity of rhythm labeling results can be further improved, more rhythm feature data can be obtained, and the melody recognition accuracy of the trained melody recognition model can be improved.
[0019] In a second aspect, an embodiment of the present application provides a melody recognition method, the method comprising:
[0020] Extract features from the audio data to be identified to obtain the features of the melody to be identified;
[0021] The melody feature to be recognized is input into a trained melody recognition model to obtain a melody recognition result output by the trained melody recognition model; wherein the melody recognition model is trained based on the melody recognition model training method as described in any one of the first aspects above.
[0022] In the above implementation process, since the melody recognition model is obtained by training based on the melody recognition model training method as described in any one of the first aspects, as described above, the trained melody recognition model obtained based on the melody recognition model training method can perform melody recognition on the audio data to be recognized, and obtain a melody recognition result with higher accuracy. The technical problem of low accuracy of the melody recognition result obtained based on the existing melody recognition method is solved.
[0023] In a third aspect, an embodiment of the present application further provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the method described in any one of the first aspect or the second aspect.
[0024] In a fourth aspect, an embodiment of the present application further provides an electronic device; the electronic device includes:
[0025] Memory;
[0026] processor;
[0027] The memory stores a computer program executable by the processor, and when the computer program is executed by the processor, the method described in any one of the first aspect or the second aspect is performed.
[0028] In a fifth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described in any one of the first aspect or the second aspect is executed.
[0029] The beneficial effects of the present application include at least: extracting features from audio data to be trained to obtain feature data to be trained; inputting the feature data to be trained into a melody recognition model to be trained to obtain a model recognition result output by the melody recognition model to be trained; and optimizing the internal parameters of the melody recognition model to be trained based on the model recognition result and the melody labeling result of the audio data to be trained to obtain a trained melody recognition model. Since the melody recognition model to be trained includes a conditional random field model, the conditional random field model can effectively take into account the dependencies between different feature data to be trained, thereby improving the accuracy of the model recognition result. The trained melody recognition model obtained based on the melody recognition model training method can perform melody recognition on the audio data to be recognized, thereby obtaining a melody recognition result with higher accuracy. The technical problem of low accuracy of melody recognition results obtained based on existing melody recognition methods is solved. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0031] Figure 1 A flowchart of a melody recognition model training method provided in an embodiment of the present application;
[0032] Figure 2 A schematic diagram of a melody recognition method provided in an embodiment of the present application;
[0033] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0034] The following embodiments of the technical solution of the present application are described in detail in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present application, and are therefore only used as examples, and cannot be used to limit the scope of protection of the present application.
[0035] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field to which this application belongs; the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit this application.
[0036] In the description of the embodiments of the present application, the technical terms "first", "second", etc. are only used to distinguish different objects, and cannot be understood as indicating or implying relative importance or implicitly indicating the number, specific order or primary and secondary relationship of the indicated technical features. In the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.
[0037] See also Figure 1 The flowchart of a melody recognition model training method provided by an embodiment of the present application is shown. The melody recognition model training method may include the following steps:
[0038] S101, extracting features from the audio data to be trained to obtain feature data to be trained;
[0039] S102, inputting the feature data to be trained into the melody recognition model to be trained, and obtaining a model recognition result output by the melody recognition model to be trained; wherein the melody recognition model to be trained includes a conditional random field model;
[0040] S103: Optimize internal parameters of the melody recognition model to be trained according to the model recognition result and the melody labeling result of the audio data to be trained, so as to obtain a trained melody recognition model.
[0041] In step S101, the audio data to be trained may be vocal data with melody or musical instrument sound data with melody; or it may be vocal data separated from audio data or video data based on an existing sound source separation method (e.g., a vocal extraction function in an existing audio processing software or video editing software). Feature extraction methods such as short-time Fourier transform (STFT), Mel Frequency Cepstrum Coefficient (MFCC), Constant Q Transformation (CQT) or Chroma may be used to extract features from the audio data to be trained to obtain feature data to be trained. Chroma is a feature extraction method for describing audio pitch and harmony, which extracts features based on the idea of representing musical notes as colors on a two-dimensional plane.
[0042] Wherein, in step S102, the melody recognition model to be trained includes a conditional random field model. Since the conditional random field model can fully consider the dependency between different feature data to be trained, the accuracy of the model recognition result is improved through global constraints. The conditional random field model can be specifically set in the last layer of the melody recognition model to be trained, and is used to be combined with other neural network models (for example, convolutional neural network, recurrent neural network or deep learning model based on self-attention mechanism, etc.) to jointly predict the model recognition result of the corresponding audio data. Traditional neural network models such as convolutional neural network, recurrent neural network or deep learning model based on self-attention mechanism are prone to ignore the dependency between different feature data, which makes the model fall into a local optimal solution and cause error accumulation. After obtaining the feature data to be trained, a standardization processing method such as zero mean normalization or amplitude normalization can be used to standardize the feature data to be trained, and then the standardized feature data is input into the melody recognition model to be trained for model training; so as to improve the stability of model training and melody recognition performance.
[0043] Wherein, in step S103, the corresponding loss function value can be calculated according to the model recognition result and the melody labeling result of the audio data to be trained, and the internal parameters of the melody recognition model to be trained can be optimized based on the loss function value. The melody labeling result can be obtained by artificially labeling the melody of the audio data to be trained. Exemplarily, the loss function value between the model recognition result and the melody labeling result can be calculated based on distance calculation methods such as Euclidean distance, cosine distance or Manhattan distance. In the case where the conditional random field model and the neural network model are combined to jointly predict the model recognition result of the corresponding audio data, the conditional random field model can be trained end-to-end with the neural network model to achieve the optimization of the loss function of the conditional random field model and the model recognition result, thereby better realizing the optimization of the internal parameters of the melody recognition model to be trained. The performance of the trained melody recognition model can be effectively improved through end-to-end joint training, and the accuracy of the model recognition result can be improved.
[0044] It can be seen that the melody recognition model training method provided in the embodiment of the present application obtains feature data to be trained by extracting features from the audio data to be trained; inputs the feature data to be trained into the melody recognition model to be trained to obtain the model recognition result output by the melody recognition model to be trained; and optimizes the internal parameters of the melody recognition model to be trained according to the model recognition result and the melody labeling result of the audio data to be trained to obtain a trained melody recognition model. Since the melody recognition model to be trained includes a conditional random field model, the conditional random field model can effectively take into account the dependencies between different feature data to be trained, thereby improving the accuracy of the model recognition result. The trained melody recognition model obtained based on the melody recognition model training method can perform melody recognition on the audio data to be recognized, and obtain a melody recognition result with higher accuracy. The technical problem of low accuracy of the melody recognition result obtained based on the existing melody recognition method is solved.
[0045] In some optional embodiments, S101, performing feature extraction on the audio data to be trained to obtain the feature data to be trained includes: performing feature extraction on the audio data to be trained based on at least two feature extraction methods including short-time Fourier transform, Mel-frequency cepstral coefficients, constant Q transform or Chroma to obtain the feature data to be trained.
[0046] Among them, feature extraction can be performed on the training audio data based on two or three feature extraction methods of short-time Fourier transform, Mel cepstral coefficients, constant Q transform or Chroma. Exemplarily, feature extraction can be performed on the training audio data based on the two feature extraction methods of short-time Fourier transform and Mel cepstral coefficients, or based on the three feature extraction methods of Mel cepstral coefficients, constant Q transform and Chroma to obtain feature data to be trained. Feature extraction can also be performed on the training audio data based on the four feature extraction methods of short-time Fourier transform, Mel cepstral coefficients, constant Q transform and Chroma. It should be noted that when feature extraction is performed on the training audio data based on at least two feature extraction methods of short-time Fourier transform, Mel cepstral coefficients, constant Q transform or Chroma, other feature extraction methods can also be combined at the same time to extract features from the training audio data to obtain more diverse feature data. Since different feature extraction methods can capture feature data of different aspects of the audio data to be trained, extracting features from the audio data to be trained based on multiple feature extraction methods can improve the diversity of the feature data to be trained, thereby improving the melody recognition accuracy and precision of the melody recognition model obtained through training.
[0047] In addition, the conditional random field model can also handle complex and changeable audio rhythms and beat patterns, making full use of the feature data to be trained extracted based on at least two different feature extraction methods, so that the trained melody recognition model can handle more diverse audio data.
[0048] In some optional embodiments, before S101, extracting features from the audio data to be trained to obtain the feature data to be trained, the method further includes: performing speed normalization on the audio data to be trained to obtain normalized data; extracting features from the audio data to be trained to obtain the feature data to be trained includes: extracting features from the normalized data to obtain the feature data to be trained.
[0049] Among them, the speed normalization processing can be performed on the audio data to be trained based on the audio bpm of the audio data to be trained (that is, the number of beats per minute of the audio data). Specifically, the bpm detection can be performed on the audio data to be trained to obtain the bpm of the audio data to be trained, and the audio speed of the audio data to be trained can be accelerated or reduced according to the difference between the bpm of the audio data to be trained and the standard bpm (which can be adjusted according to the actual application situation). It should be noted that during the speed normalization processing, the pitch of the audio data to be trained is not changed. By performing speed normalization processing on the audio data to be trained, the speed consistency of the model training data is ensured to improve the melody recognition precision and accuracy of the melody recognition model obtained by training.
[0050] Before normalizing the speed of the training audio data, or before extracting features from the training audio data in S101, the training audio data may be preprocessed by noise reduction, silent segment removal, and other preprocessing operations.
[0051] In some optional embodiments, S102, inputting the melody feature data to be trained into the melody recognition model to be trained to obtain the model recognition result output by the melody recognition model to be trained, including: rhythm marking and / or high-pitched marking on the melody feature data to be trained based on a preset marking system to obtain marked feature data; inputting the marked feature data into the melody recognition model to be trained to obtain the model recognition result output by the melody recognition model to be trained.
[0052] Among them, the marked feature data may include rhythm marked feature data and / or treble marked feature data. In the case where the melody recognition model is mainly used to identify audio data with large rhythm changes, the marked feature data may include rhythm marked feature data; in the case where the melody recognition model is mainly used to identify audio data with a high frequency of treble data, the marked feature data may include treble marked feature data. Exemplarily, the rhythm of the melody feature data to be trained can be marked based on standardized rhythm units (for example, quarter notes, sixteenth notes, thirty-second notes, etc.). By marking the rhythm and / or treble of the melody feature data to be trained, it is convenient for the melody recognition model to be trained to more accurately capture the rhythm and treble features in the training data, thereby improving the melody recognition accuracy of the melody recognition model obtained by training. It is also possible to obtain the beat information of the melody feature data to be trained based on the method of accent detection, and mark the rhythm of the melody feature data to be trained according to the beat information.
[0053] In addition, the present application uses a preset annotation system to perform rhythm annotation and / or high-pitched annotation on the melody feature data to be trained, that is, to achieve the division of audio data through the standardization of the melody itself. Compared with the traditional window division method, the feature annotation method provided by the present application can reduce labor costs and errors (mainly reducing the manual adjustment costs and errors of dividing windows) while ensuring the accuracy of model recognition results.
[0054] In some optional embodiments, the preset marking system includes a BIES mark for implementing the rhythm marking and / or a MIDI treble mark for implementing the treble marking; wherein the BIES mark includes a blank note mark and a start mark, a middle mark and an end mark of each note; and the MIDI treble mark includes the treble value of each note.
[0055] Among them, the BIES mark includes the start mark B, the middle mark I and the end mark E of each note, as well as the empty note mark S. MIDI treble marking mainly involves the treble marking method in the MIDI track, which is a way to mark the treble by specific values, which correspond to the note names and pitches in music theory. MIDI treble marking can specifically include 128 MIDI values. The MIDI treble marking can correspond the treble in the audio data to the 128 MIDI values, and the 128 MIDI values are used to realize the treble marking of the melody feature data to be trained. The recognition accuracy of melody and rhythm can be improved by refining the time signature and using more refined units (such as 1 / 64 beats). Taking the 1a note with a duration of an eighth note (the marking unit is 1 / 64 beats) as an example, its rhythm marking result can be: B, I, I, I, I, I, I, E; its treble marking result can be: 69, 69, 69, 69, 69, 69, 69, 69 (where 69 is the MIDI value corresponding to 1a).
[0056] In some optional embodiments, the intermediate mark includes a plurality of different intermediate marks sequentially adjacent to the start mark.
[0057] Among them, taking the 1a note (marking unit is 1 / 64 beat) with a duration of eighth note as an example, its rhythm marking result can be: B, I1, I1, I2, I2, I3, I3, E; or, B, I1, I2, I3, I4, I5, I6, E. Since the middle mark includes a plurality of different middle marks that are sequentially adjacent to the start mark, the diversity of the rhythm marking results can be further improved, more rhythm feature data can be obtained, and the melody recognition accuracy of the trained melody recognition model can be improved.
[0058] Please refer to Figure 2 , Figure 2 A schematic diagram of a melody recognition method provided in an embodiment of the present application. The melody recognition method includes:
[0059] S201, extracting features from the audio data to be identified, and obtaining features of the melody to be identified;
[0060] S202. Input the melody feature to be recognized into a trained melody recognition model to obtain a melody recognition result output by the trained melody recognition model; wherein the melody recognition model is trained based on the melody recognition model training method as described in any one of the first aspects above.
[0061] Among them, in step S201, feature extraction methods such as short-time Fourier transform, Mel cepstral coefficients, constant Q transform or Chroma can be used to extract features of the audio data to be identified to obtain the melody features to be identified. Feature extraction can also be performed on the audio data to be identified based on at least two feature extraction methods among short-time Fourier transform, Mel cepstral coefficients, constant Q transform or Chroma feature extraction to obtain the melody features to be identified. Before extracting features from the audio data to be identified, the audio data to be identified can also be firstly subjected to speed normalization processing to obtain a normalized identified melody; and then feature extraction can be performed on the normalized identified melody to obtain the melody features to be identified.
[0062] Among them, in step S202, the rhythm and / or high pitch marking of the melody feature to be identified can be first performed based on a preset marking system to obtain marked identification feature data; the marked identification feature data is input into a trained melody recognition model to obtain a melody recognition result output by the trained melody recognition model. The preset marking system includes a BIES mark for implementing rhythm marking and / or a MIDI high pitch mark for implementing high pitch marking. The middle mark can specifically include a plurality of different middle marks that are sequentially adjacent to the start mark.
[0063] It should be noted that the specific implementation of step S201 and step S202 may correspond to the specific implementation of the above-mentioned melody recognition model training method. Exemplarily, in S101, feature extraction is performed on the audio data to be trained to obtain the feature data to be trained, including: based on at least two feature extraction methods among short-time Fourier transform, Mel cepstral coefficients, constant Q transform or Chroma, feature extraction is performed on the audio data to be trained to obtain the feature data to be trained. In this case, S201, feature extraction is performed on the audio data to be recognized to obtain the melody features to be recognized, which may include: based on at least two feature extraction methods among short-time Fourier transform, Mel cepstral coefficients, constant Q transform or Chroma, feature extraction is performed on the audio data to be recognized to obtain the melody features to be recognized. And the multiple feature extraction methods specifically adopted in S201 may be the same as the multiple feature extraction methods adopted in S101. In S102, inputting the melody feature data to be trained into the melody recognition model to be trained to obtain the model recognition result output by the melody recognition model to be trained, including: rhythm marking and / or high pitch marking of the melody feature data to be trained based on a preset marking system to obtain marked feature data; inputting the marked feature data into the melody recognition model to be trained to obtain the model recognition result output by the melody recognition model to be trained, in the case of S202, inputting the melody feature to be recognized into the trained melody recognition model to obtain the melody recognition result output by the trained melody recognition model, may include: rhythm marking and / or high pitch marking of the melody feature to be recognized based on a preset marking system to obtain marked melody feature data; inputting the marked melody feature data into the trained melody recognition model to obtain the melody recognition result output by the trained melody recognition model. Since the melody recognition model is obtained by training based on the melody recognition model training method as described in any one of the first aspects above, as described above, the trained melody recognition model obtained based on the melody recognition model training method can perform melody recognition on the audio data to be recognized and obtain a melody recognition result with higher accuracy. The invention solves the technical problem that the melody recognition result obtained based on the existing melody recognition method has low accuracy.
[0064] An embodiment of the present application also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the melody recognition model training method as described in any one of the first aspect or the melody recognition method as described in the second aspect.
[0065] Please refer to Figure 3 , Figure 3The electronic device 300 is a schematic diagram of a structure of an electronic device 300 provided in an embodiment of the present application. The electronic device 300 comprises: a memory 302 and a processor 301; the memory 302 stores a computer program executable by the processor 301, and when the computer program is executed by the processor 301, the method described in any one of the first aspect or the second aspect is executed.
[0066] The memory 302 and the processor 301 may be interconnected and communicate with each other via a communication bus 303 and / or other forms of connection mechanisms (not shown). The memory 302 stores a computer program executable by the processor 301. When the computer program is executed by the processor 301, the method described in the first aspect or the second aspect above is executed.
[0067] The embodiment of the present application further provides a computer-readable storage medium, on which computer program instructions are stored. When the computer program instructions are executed by the processor 301, the method described in the first aspect or the second aspect above is executed.
[0068] Among them, the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable red-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.
[0069] In several embodiments provided by the embodiments of the present application, it should be understood that the disclosed device / system and method can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to the multiple embodiments of the embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and a part of the module, program segment or code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.
[0070] In addition, the functional modules in each embodiment of the present application can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part.
[0071] The above description is only an optional implementation manner of the embodiments of the present application, but the protection scope of the embodiments of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or replacements within the technical scope disclosed in the embodiments of the present application, which should be covered within the protection scope of the embodiments of the present application.
Claims
1. A melody recognition model training method, characterized in that: The method comprises: Perform feature extraction on the audio data to be trained to obtain feature data to be trained; Inputting the feature data to be trained into the melody recognition model to be trained to obtain a model recognition result output by the melody recognition model to be trained; wherein the melody recognition model to be trained includes a conditional random field model; According to the model recognition result and the melody labeling result of the audio data to be trained, optimizing the internal parameters of the melody recognition model to be trained to obtain a trained melody recognition model; The step of inputting the feature data to be trained into the melody recognition model to be trained to obtain the model recognition result output by the melody recognition model to be trained comprises: rhythm labeling and / or pitch labeling of the feature data to be trained based on a preset labeling system to obtain labeled feature data; inputting the labeled feature data into the melody recognition model to be trained to obtain the model recognition result output by the melody recognition model to be trained; Wherein, the preset marking system includes BIES marks for implementing the rhythm marking and / or MIDI pitch marks for implementing the pitch marking; wherein, the BIES marks include empty note marks and the start mark, middle mark and end mark of each note; the MIDI pitch mark includes the pitch value of each note; the middle mark includes multiple different middle marks adjacent to the start mark in sequence.
2. The method according to claim 1, characterized in that The extracting features of the audio data to be trained to obtain the feature data to be trained comprises: Based on at least two feature extraction methods among short-time Fourier transform, Mel-frequency cepstral coefficient, constant Q transform or Chroma, feature extraction is performed on the audio data to be trained to obtain the feature data to be trained.
3. The method according to claim 1, characterized in that Before extracting features from the audio data to be trained to obtain feature data to be trained, the method further includes: performing speed normalization processing on the audio data to be trained to obtain normalized data; The extracting features from the audio data to be trained to obtain the feature data to be trained includes: extracting features from the normalized data to obtain the feature data to be trained.
4. A melody recognition method, characterized in that: The method comprises: Extract features from the audio data to be identified to obtain the features of the melody to be identified; The melody feature to be recognized is input into a trained melody recognition model to obtain a melody recognition result output by the trained melody recognition model; wherein the melody recognition model is trained based on the melody recognition model training method as described in any one of claims 1 to 3 above.
5. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 4 is implemented.
6. An electronic device, characterized in that: The electronic device comprises: Memory; processor; The memory stores a computer program executable by the processor, and when the computer program is executed by the processor, the method according to any one of claims 1 to 4 is performed.
7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the method according to any one of claims 1 to 4 is executed.
Citation Information
Patent Citations
Humming type rhythm identification method based on hidden Markov model
CN101504834A
