Harmony recognition and model training method, program product, equipment and storage medium
Through the harmonic recognition method based on beat data, the problem of inaccurate harmonic recognition in the prior art is solved, and the harmony recognition results with higher accuracy are achieved, and the calculation amount and labor cost are reduced.
Patent Information
- Application Number
- CN202411442252.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-16
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-10-16
AI Technical Summary
The existing harmony recognition method uses a fixed-length window to segment the audio signal, resulting in inaccurate position of the chord change, which in turn leads to blurred and inaccurate recognition results.
The segmentation method based on beat data is adopted, and the audio data is segmented through beat data, feature data is extracted, and input into the harmony recognition model for training and recognition, and the model parameters are optimized to improve the accuracy of harmony recognition.
Improve the accuracy of harmony recognition, reduce the amount of calculation, reduce labor costs, and eliminate the need to manually adjust the window length, achieving more accurate positioning of chord changes.
Smart Images

Figure CN118969012A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of audio processing technology, and in particular to harmony recognition and its model training method, program product, device and storage medium. Background Art
[0002] Harmony is the coordination of multiple voices in music, which can enrich the layering and depth of music. Harmony recognition technology is used to identify different chords in music. Most existing harmony recognition methods first divide the continuous audio signal into short time frames through window division, and then further analyze and process each frame.
[0003] Usually, a fixed-length window is used to segment the signal by sliding the window on the audio signal, and harmony recognition is performed on the multiple audio signals obtained by segmentation. Using a fixed-length window cannot accurately locate the chord change position, and the sliding window may cross the chord change point, which can easily lead to fuzzy harmony recognition and inaccurate harmony recognition results. Summary of the invention
[0004] In view of this, the purpose of the embodiments of the present application is to provide a harmony recognition and its model training method, program product, device and storage medium, so as to solve the technical problem that the harmony recognition results obtained based on the existing harmony recognition method are not accurate enough.
[0005] In a first aspect, an embodiment of the present application provides a harmony recognition model training method, the method comprising: Segmenting the audio data to be trained based on the beat data of the audio data to be trained to obtain a plurality of audio segmentation segments; Extracting features from the audio segmentation fragments to obtain audio feature data to be trained; Inputting the audio feature data to be trained into the harmony recognition model to be trained, and obtaining a model recognition result output by the harmony recognition model to be trained; According to the model recognition result and the harmony labeling result of the audio data to be trained, the internal parameters of the harmony recognition model to be trained are optimized to obtain a trained harmony recognition model.
[0006] In the above implementation process, the harmony recognition model training method segments the audio data to be trained based on the beat data of the audio data to be trained to obtain multiple audio segmentation segments; extracts features from the audio segmentation segments to obtain audio feature data to be trained; inputs the audio feature data to be trained into the harmony recognition model to be trained to obtain the model recognition result output by the harmony recognition model to be trained; optimizes the internal parameters of the harmony recognition model to be trained according to the model recognition result and the harmony labeling result of the audio data to be trained to obtain a trained harmony recognition model. Since the harmony recognition model segments the audio data to be trained based on the beat data of the audio data to be trained to obtain multiple audio segmentation segments, compared with the existing sliding window segmentation method, the beat data can better characterize the chord change position of the audio data to be trained, and the segmentation and positioning of the audio data to be trained can be more accurately achieved based on the beat data, thereby improving the harmony recognition accuracy of the trained harmony recognition model. Based on the harmony recognition model obtained by the harmony recognition model training method, harmony recognition is performed on the audio data to be recognized, and a more accurate harmony recognition result can be obtained. The invention solves the technical problem that the harmony recognition result obtained based on the existing harmony recognition method is not accurate enough.
[0007] In addition, the method of using a fixed-length window to segment the audio signal has overlapping windows to ensure the continuity of the audio segment signals obtained by multiple segmentations, which will lead to a large amount of redundant calculations. However, this application uses a method based on beat data to achieve segmentation and positioning of the audio data to be trained, and divides the audio data to be trained into multiple continuous and independent audio segmentation segments, which can improve the accuracy of audio segmentation and positioning while reducing the amount of calculation in the harmony recognition process.
[0008] Optionally, in an embodiment of the present application, the beat data of the audio data to be trained includes the stress beat position; before segmenting the audio data to be trained based on the beat data of the audio data to be trained to obtain multiple audio segmentation segments, the method also includes: performing rhythm feature recognition on the audio data to be trained to obtain the rhythm feature data of the audio data to be trained; and determining the stress beat position based on the rhythm feature data.
[0009] Optionally, in an embodiment of the present application, determining the stress beat position based on the rhythm feature data includes: using a period estimation method to determine the beat period of the audio data to be trained; determining the stress beat position based on the rhythm feature data and the beat period.
[0010] In the above implementation process, the stress beat position of the audio data to be trained can be determined more comprehensively and accurately through the beat cycle and rhythm feature data.
[0011] In addition, when using a fixed-length window to segment the audio signal, it is necessary to adjust the window length based on the music with different rhythmic characteristics, which will increase the difficulty of recognizing the harmony and the labor cost. However, the present application can accurately determine the stress beat position based on the obtained rhythm feature data and beat period through rhythm feature recognition and period estimation, and then implement the segmentation of the training audio data based on the stress beat position. The above method provided by the present application does not require human participation in the process of implementing the segmentation of the training audio data, and can improve the accuracy of the obtained audio segmentation segments while reducing labor costs.
[0012] Optionally, in an embodiment of the present application, after determining the stress beat position based on the rhythm feature data, the method further includes: calculating the average beat frame of the audio data to be trained based on the stress beat position; supplementing the stress beat position based on the average beat frame to obtain the supplemented beat position; segmenting the audio data to be trained based on the beat data of the audio data to be trained to obtain multiple audio segmentation segments, including: segmenting the audio data to be trained based on the supplemented beat position to obtain multiple audio segmentation segments.
[0013] In the above implementation process, by supplementing the accented beat position through averaging the beat frames, a more complete supplemented beat position can be obtained, thereby improving the harmony recognition precision and accuracy of the harmony recognition model obtained by training based on audio segmentation fragments.
[0014] Optionally, in an embodiment of the present application, the audio data to be trained is segmented based on the beat data of the audio data to be trained to obtain multiple audio segmentation segments, including: determining the number of beats of the audio data to be trained according to the accent beat position; determining the beat speed of the audio data to be trained based on the audio length of the audio data to be trained and the number of beats; performing speed normalization processing on the audio data to be trained according to the beat speed and the standard beat speed to obtain standardized audio data; and segmenting the standardized audio data based on the beat data to obtain multiple audio segmentation segments.
[0015] In the above implementation process, by performing speed normalization processing on the audio data to be trained, the sizes of multiple audio feature data to be trained obtained by feature extraction of the audio segmentation fragments can be made consistent, so as to improve the harmony recognition precision and accuracy of the harmony recognition model trained based on the audio feature data to be trained.
[0016] Optionally, in an embodiment of the present application, the standardized audio data is segmented based on the beat data to obtain a plurality of the audio segmentation segments, including: segmenting the standardized audio data based on the beat data to obtain a plurality of initial segmentation segments; filling or truncating the initial segmentation segments based on the average beat frame to obtain the processed audio segmentation segments.
[0017] In the above implementation process, the lengths of multiple initial segmented segments obtained by segmenting the standardized audio data based on the beat data are not necessarily exactly the same. By padding or truncating the initial segmented segments, the slight length differences between different initial segmented segments can be eliminated, and multiple audio segmented segments with consistent lengths can be obtained, thereby improving the harmony recognition precision and accuracy of the harmony recognition model obtained by training based on the audio segmented segments.
[0018] In a second aspect, an embodiment of the present application provides a harmony recognition method, the method comprising: Segmenting the audio data to be recognized based on the beat data of the audio data to be recognized to obtain a plurality of audio recognition segments; Extracting features from the audio recognition segment to obtain audio feature data to be recognized; The audio feature data to be recognized is input into a trained harmony recognition model to obtain a harmony recognition result output by the trained harmony recognition model; wherein the harmony recognition model is trained based on the harmony recognition model training method as described in any one of the first aspects above.
[0019] In the above implementation process, the harmony recognition model adopted by the harmony recognition method is to segment the audio data to be trained through the beat data of the audio data to be trained to obtain multiple audio segmentation segments. Compared with the existing sliding window segmentation method, the beat data can better characterize the chord change position of the audio data to be trained. Based on the beat data, the segmentation and positioning of the audio data to be trained can be more accurately achieved, thereby improving the harmony recognition accuracy of the trained harmony recognition model. Based on the harmony recognition method, harmony recognition is performed on the audio data to be recognized, and a more accurate harmony recognition result can be obtained. The technical problem that the harmony recognition result obtained based on the existing harmony recognition method is not accurate enough is solved.
[0020] In a third aspect, an embodiment of the present application further provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the method described in any one of the first aspect or the second aspect.
[0021] In a fourth aspect, an embodiment of the present application further provides an electronic device; the electronic device includes: Memory; processor; The memory stores a computer program executable by the processor, and when the computer program is executed by the processor, the method described in any one of the first aspect or the second aspect is performed.
[0022] In a fifth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described in any one of the first aspect or the second aspect is executed.
[0023] The beneficial effects of the present application include at least: the harmony recognition model training method segments the audio data to be trained based on the beat data of the audio data to be trained to obtain multiple audio segmentation segments; extracts features from the audio segmentation segments to obtain audio feature data to be trained; inputs the audio feature data to be trained into the harmony recognition model to be trained to obtain the model recognition result output by the harmony recognition model to be trained; optimizes the internal parameters of the harmony recognition model to be trained according to the model recognition result and the harmony labeling result of the audio data to be trained to obtain a trained harmony recognition model. Since the harmony recognition model segments the audio data to be trained based on the beat data of the audio data to be trained to obtain multiple audio segmentation segments, compared with the existing sliding window segmentation method, the beat data can better characterize the chord change position of the audio data to be trained, and the segmentation and positioning of the audio data to be trained can be more accurately achieved based on the beat data, thereby improving the harmony recognition accuracy of the trained harmony recognition model. Based on the harmony recognition model obtained by the harmony recognition model training method, harmony recognition can be performed on the audio data to be recognized, and a more accurate harmony recognition result can be obtained. The invention solves the technical problem that the harmony recognition result obtained based on the existing harmony recognition method is not accurate enough.
[0024] Furthermore, the method of segmenting audio signals using a fixed-length window has overlapping windows to ensure the continuity of the audio segment signals obtained by multiple segmentations, which will lead to a large amount of redundant calculations. However, the present application adopts a method of segmenting and locating the audio data to be trained based on beat data, and divides the audio data to be trained into multiple continuous and independent audio segmentation segments, which can improve the accuracy of audio segmentation and positioning while reducing the amount of calculation in the harmony recognition process. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0026] Figure 1 A flowchart of a harmony recognition model training method provided in an embodiment of the present application; Figure 2 A schematic diagram of the position of the stress beats of the audio data to be trained provided in an embodiment of the present application; Figure 3 A flowchart of an audio data segmentation method provided in an embodiment of the present application; Figure 4 A flowchart of a harmony recognition method provided in an embodiment of the present application; Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0027] The following embodiments of the technical solution of the present application are described in detail in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present application, and are therefore only used as examples, and cannot be used to limit the scope of protection of the present application.
[0028] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field to which this application belongs; the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit this application.
[0029] In the description of the embodiments of the present application, the technical terms "first", "second", etc. are only used to distinguish different objects, and cannot be understood as indicating or implying relative importance or implicitly indicating the number, specific order or primary and secondary relationship of the indicated technical features. In the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.
[0030] See also Figure 1 The flowchart of a harmony recognition model training method provided by an embodiment of the present application is shown. The harmony recognition model training method may include the following steps: S101, segmenting the audio data to be trained based on the beat data of the audio data to be trained to obtain a plurality of audio segmentation segments; S102, extracting features from the audio segmentation segments to obtain audio feature data to be trained; S103, inputting the audio feature data to be trained into the harmony recognition model to be trained, and obtaining a model recognition result output by the harmony recognition model to be trained; S104: Optimize internal parameters of the harmony recognition model to be trained according to the model recognition result and the harmony labeling result of the audio data to be trained, to obtain a trained harmony recognition model.
[0031] Wherein, in step S101, the beat data may include a beat cycle or an accent beat position, or may include both a beat cycle and an accent beat position. The accent beat position may be represented by a time series of accents in different beats. The audio data to be trained may be segmented into a plurality of audio segmentation segments according to the beat cycle or the accent beat position. The number of audio segmentation segments may be 20 segments, or 16 segments or other reasonable values; the specific number of audio segmentation segments is related to the actual application data such as the audio length and beat cycle of the audio data to be trained.
[0032] Among them, in step S102, feature extraction methods such as fast Fourier transform (FFT), Mel Frequency Cepstrum Coefficient (MFCC), Constant Q Transformation (CQT) or Mel transform can be used to extract features from the audio segmentation fragments. Among them, Mel transform simulates the perception characteristics of the human ear to sound and nonlinearly maps the frequency axis to the Mel scale, so as to better realize the feature processing of the audio signal. It is also possible to choose to use a variety of feature extraction methods such as FFT, MFCC, CQT and Mel transform to jointly extract features from the audio segmentation fragments. This application does not make specific restrictions on this.
[0033] Among them, in step S103, after obtaining the audio feature data to be trained, a standardization processing method such as zero mean normalization or amplitude normalization can be used to standardize the audio feature data to be trained, and then the standardized feature data is input into the harmony recognition model to be trained to perform model training; so as to improve the stability of model training and harmony recognition performance.
[0034] Wherein, in step S104, the corresponding loss function value can be calculated according to the model recognition result and the harmony labeling result of the audio data to be trained, and the internal parameters of the harmony recognition model to be trained can be optimized based on the loss function value. Exemplarily, the loss function value between the model recognition result and the harmony labeling result can be calculated based on distance calculation methods such as Euclidean distance, cosine distance or Manhattan distance. The harmony labeling result can be obtained by manually performing harmony labeling on the training audio data.
[0035] It can be seen that the harmony recognition model training method provided by the embodiment of the present application is to segment the audio data to be trained based on the beat data of the audio data to be trained to obtain multiple audio segmentation segments; extract features from the audio segmentation segments to obtain audio feature data to be trained; input the audio feature data to be trained into the harmony recognition model to be trained to obtain the model recognition result output by the harmony recognition model to be trained; according to the model recognition result and the harmony labeling result of the audio data to be trained, the internal parameters of the harmony recognition model to be trained are optimized to obtain a trained harmony recognition model. Since the harmony recognition model is to segment the audio data to be trained based on the beat data of the audio data to be trained to obtain multiple audio segmentation segments, compared with the existing sliding window segmentation method, the beat data can better characterize the chord change position of the audio data to be trained, and the segmentation and positioning of the audio data to be trained can be more accurately achieved based on the beat data, thereby improving the harmony recognition accuracy of the trained harmony recognition model. Based on the harmony recognition model obtained by the harmony recognition model training method, harmony recognition is performed on the audio data to be recognized, and a more accurate harmony recognition result can be obtained. The invention solves the technical problem that the harmony recognition result obtained based on the existing harmony recognition method is not accurate enough.
[0036] In some optional embodiments, the beat data of the audio data to be trained includes the stress beat position; before S101, segmenting the audio data to be trained based on the beat data of the audio data to be trained to obtain multiple audio segmentation segments, the method also includes: performing rhythm feature recognition on the audio data to be trained to obtain the rhythm feature data of the audio data to be trained; and determining the stress beat position based on the rhythm feature data.
[0037] Among them, the rhythm feature recognition of the audio data to be trained can be performed based on short-time Fourier transform or energy envelope, and the rhythm feature data of the audio data to be trained can be obtained. Among them, the energy envelope can extract the intensity change of the audio signal by analyzing the energy distribution of the audio data to be trained, and the rhythm feature data of the audio data to be trained can be obtained based on the intensity change of the audio signal. Short-time Fourier transform can divide the audio data to be trained into multiple short time periods (frames) and perform Fourier transform on each frame to obtain the frequency characteristics of the audio data to be trained at each time point; based on the frequency characteristics of the audio data to be trained, its rhythm feature data can be obtained. Rhythm feature recognition of the audio data to be trained can also be performed based on short-time Fourier transform and energy envelope. The rhythm feature data can specifically include the measure, beat, position of the stress and cycle of the audio data to be trained. According to the identified rhythm feature data, the stress beat position of the audio data to be trained can be determined.
[0038] In some optional embodiments, determining the stress beat position based on the rhythm feature data includes: using a period estimation method to determine the beat period of the audio data to be trained; determining the stress beat position based on the rhythm feature data and the beat period.
[0039] Among them, the period estimation method such as autocorrelation or spectral domain can be used to estimate the beat period of the audio data to be trained, and then the stress beat position can be determined based on the rhythm feature data and the beat period. The stress beat position can be represented by the stress time series of different beats. Please refer to Figure 2 , Figure 2 A schematic diagram of the position of the stress beat of the audio data to be trained provided in an embodiment of the present application. Figure 2 The stress beat position Tb corresponding to the audio data to be trained can be expressed as Tb= It should be noted that Figure 2 It is only exemplarily shown that part of the stress beat position of the audio data to be trained, t0 represents the starting position of the audio data to be trained, and tn represents the position of the nth stress beat. The value of tn can be determined according to the time when the nth stress beat appears. Since the stress does not necessarily appear in every beat, the stress beat position of the audio data to be trained can be determined more comprehensively and accurately through the beat cycle and rhythm feature data.
[0040] In some optional embodiments, after determining the stress beat position based on the rhythm feature data, the method further includes: calculating an average beat frame of the audio data to be trained according to the stress beat position; supplementing the stress beat position according to the average beat frame to obtain a supplemented beat position; S101, segmenting the audio data to be trained based on the beat data of the audio data to be trained to obtain a plurality of audio segmentation segments, including: segmenting the audio data to be trained based on the supplemented beat positions to obtain a plurality of the audio segmentation segments.
[0041] The average beat frame can be represented by the average time difference between every two accent beats; accordingly, Calculate the average beat frame; where, represents the average beat frame, Indicates the position of the ith accent beat (specifically, it can be represented by the time when the ith accent beat occurs). According to the average beat frame, the accent can be supplemented where the distance between the accent beats is too large. The accent beat distance refers to the distance between two adjacent accent beat positions (that is, the time difference between the two adjacent accent beats marked). The accent beat position determined based on the rhythm feature data may be incomplete (there may be missed beats). By supplementing the accent beat position with the average beat frame, a more complete supplemented beat position can be obtained; thereby improving the harmony recognition accuracy and precision of the harmony recognition model obtained based on the audio segmentation segment training.
[0042] Please refer to Figure 3 , Figure 3 A flowchart of an audio data segmentation method provided in an embodiment of the present application.
[0043] In some optional embodiments, S101, segmenting the audio data to be trained based on the beat data of the audio data to be trained to obtain multiple audio segmentation segments, including: S1011, determining the number of beats of the audio data to be trained according to the accent beat position; S1012, determining the beat speed of the audio data to be trained based on the audio length of the audio data to be trained and the number of beats; S1013, performing speed normalization processing on the audio data to be trained according to the beat speed and the standard beat speed to obtain standardized audio data; S1014, segmenting the standardized audio data based on the beat data to obtain multiple audio segmentation segments.
[0044] The number of beats m of the audio data to be trained can be determined according to the position of the stress beat or the position of the stress beat after the stress is supplemented. The beat speed of the audio data to be trained can be represented by bpm (i.e., the number of beats per minute of the audio data). Calculate the tempo of the audio data to be trained; (Unit: seconds) represents the audio length of the audio data to be trained. The standard beat speed can be 100, 120 or other reasonable values. When the beat speed is lower than the standard beat speed, the audio data to be trained needs to be accelerated; when the beat speed is higher than the standard beat speed, the audio data to be trained needs to be slowed down to obtain standardized audio data. By normalizing the speed of the audio data to be trained, the sizes of multiple audio feature data to be trained obtained by feature extraction of the audio segmentation segments can be made consistent, so as to improve the harmony recognition precision and accuracy of the harmony recognition model trained based on the audio feature data to be trained.
[0045] In some optional embodiments, S1014, segmenting the standardized audio data based on the beat data to obtain a plurality of the audio segmentation segments, includes: segmenting the standardized audio data based on the beat data to obtain a plurality of initial segmentation segments; filling or truncating the initial segmentation segments based on the average beat frame to obtain the processed audio segmentation segments.
[0046] The length of the processed audio segment can be made consistent with the length of the average beat frame by padding or truncating the initial segment. The lengths of multiple initial segment obtained by segmenting the standardized audio data based on the beat data are not necessarily exactly the same. By padding or truncating the initial segment, the slight length differences between different initial segment can be eliminated, and multiple audio segment with the same length can be obtained, so as to improve the harmony recognition precision and accuracy of the harmony recognition model obtained by training based on the audio segment.
[0047] Please refer to Figure 4 , Figure 4 A flow chart of a harmony recognition method provided in an embodiment of the present application. The harmony recognition method includes: S201, segmenting the audio data to be recognized based on the beat data of the audio data to be recognized, to obtain a plurality of audio recognition segments; S202, extracting features from the audio recognition segment to obtain audio feature data to be recognized; S203. Input the audio feature data to be recognized into a trained harmony recognition model to obtain a harmony recognition result output by the trained harmony recognition model; wherein the harmony recognition model is trained based on the harmony recognition model training method as described in any one of the first aspects above.
[0048] Wherein, in step S201, the beat data may include a beat cycle or an accent beat position, or may include both a beat cycle and an accent beat position. The accent beat position may be represented by a time series of accents in different beats. The audio data to be identified may be segmented into a plurality of audio recognition segments according to the beat cycle or the accent beat position. The number of audio recognition segments may be 20 segments, or 16 segments or other reasonable values; the specific number of audio recognition segments is related to the audio length, beat cycle and other data of the audio data to be identified.
[0049] In step S202, feature extraction methods such as FFT, MFCC, CQT or Mel transform can be used to extract features from the audio recognition segment. It is also possible to select to use multiple feature extraction methods among FFT, MFCC, CQT and Mel transform to jointly extract features from the audio recognition segment.
[0050] Among them, in step S203, after obtaining the audio feature data to be identified, a standardization processing method such as zero mean normalization or amplitude normalization can be used to standardize the audio feature data to be identified, and then the standardized feature data is input into the trained harmony recognition model to obtain the harmony recognition result.
[0051] It should be noted that the specific implementation of steps S201-S203 may correspond to the specific implementation of the above-mentioned harmony recognition model training method. Exemplarily, in the case where step S101 specifically includes the above-mentioned steps S1011-S1014, S201, based on the beat data of the audio data to be recognized, the audio data to be recognized is segmented to obtain multiple audio recognition segments, which may specifically include: determining the number of beats of the audio data to be recognized based on the position of the accented beat; determining the beat speed of the audio data to be recognized based on the audio length and the number of beats of the audio data to be recognized; performing speed normalization processing on the audio data to be recognized based on the beat speed and the standard beat speed to obtain normalized audio data; segmenting the normalized audio data based on the beat data to obtain multiple audio recognition segments. And the feature extraction method adopted in S202 may be the same as the feature extraction method adopted in S101. Since the harmony recognition model adopted by the harmony recognition method obtains multiple audio segmentation segments by segmenting the audio data to be trained through the beat data of the audio data to be trained, compared with the existing sliding window segmentation method, the beat data can better characterize the chord change position of the audio data to be trained, and the segmentation and positioning of the audio data to be trained can be more accurately achieved based on the beat data, thereby improving the harmony recognition accuracy of the trained harmony recognition model. Based on the harmony recognition method, harmony recognition of the audio data to be recognized can obtain a more accurate harmony recognition result. The technical problem that the harmony recognition results obtained based on the existing harmony recognition method are not accurate enough is solved.
[0052] An embodiment of the present application also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the harmony recognition model training method as described in any one of the first aspect or the harmony recognition method as described in the second aspect.
[0053] Please refer to Figure 5 , Figure 5 The electronic device 300 is a schematic diagram of a structure of an electronic device 300 provided in an embodiment of the present application. The electronic device 300 comprises: a memory 302 and a processor 301; the memory 302 stores a computer program executable by the processor 301, and when the computer program is executed by the processor 301, the method described in any one of the first aspect or the second aspect is executed.
[0054] The memory 302 and the processor 301 may be interconnected and communicate with each other via a communication bus 303 and / or other forms of connection mechanisms (not shown). The memory 302 stores a computer program executable by the processor 301. When the computer program is executed by the processor 301, the method described in the first aspect or the second aspect above is executed.
[0055] The embodiment of the present application further provides a computer-readable storage medium, on which computer program instructions are stored. When the computer program instructions are executed by the processor 301, the method described in the first aspect or the second aspect above is executed.
[0056] Among them, the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable red-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.
[0057] In several embodiments provided by the embodiments of the present application, it should be understood that the disclosed device / system and method can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to the multiple embodiments of the embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and a part of the module, program segment or code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.
[0058] In addition, the functional modules in each embodiment of the present application can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part.
[0059] The above description is only an optional implementation manner of the embodiments of the present application, but the protection scope of the embodiments of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or replacements within the technical scope disclosed in the embodiments of the present application, which should be covered within the protection scope of the embodiments of the present application.
Claims
1. A harmony recognition model training method, characterized in that: The method comprises: Segmenting the audio data to be trained based on the beat data of the audio data to be trained to obtain a plurality of audio segmentation segments; Extracting features from the audio segmentation fragments to obtain audio feature data to be trained; Inputting the audio feature data to be trained into the harmony recognition model to be trained, and obtaining a model recognition result output by the harmony recognition model to be trained; According to the model recognition result and the harmony labeling result of the audio data to be trained, the internal parameters of the harmony recognition model to be trained are optimized to obtain a trained harmony recognition model.
2. The method according to claim 1, characterized in that The beat data of the audio data to be trained includes the stress beat position; Before segmenting the audio data to be trained based on the beat data of the audio data to be trained to obtain a plurality of audio segmentation segments, the method further includes: Performing rhythm feature recognition on the audio data to be trained to obtain rhythm feature data of the audio data to be trained; Based on the rhythm feature data, the accent beat position is determined.
3. The method according to claim 2, characterized in that The step of determining the stress beat position based on the rhythm feature data comprises: Using a cycle estimation method to determine the beat cycle of the audio data to be trained; The accent beat position is determined based on the rhythm feature data and the beat period.
4. The method according to claim 2, characterized in that: in, After determining the stress beat position based on the rhythm feature data, the method further includes: Calculating the average beat frame of the audio data to be trained according to the stress beat position; Supplementing the accent beat position according to the average beat frame to obtain a supplemented beat position; The step of segmenting the audio data to be trained based on the beat data of the audio data to be trained to obtain a plurality of audio segmentation segments includes: segmenting the audio data to be trained based on the supplemented beat positions to obtain a plurality of audio segmentation segments.
5. The method according to claim 4, characterized in that The step of segmenting the audio data to be trained based on the beat data of the audio data to be trained to obtain a plurality of audio segmentation segments includes: Determine the number of beats of the audio data to be trained according to the accent beat position; Determining the beat speed of the audio data to be trained based on the audio length of the audio data to be trained and the number of beats; Performing speed normalization processing on the audio data to be trained according to the beat speed and the standard beat speed to obtain standardized audio data; The standardized audio data is segmented based on the beat data to obtain a plurality of audio segmentation segments.
6. The method according to claim 5, characterized in that The step of segmenting the standardized audio data based on the beat data to obtain a plurality of audio segmentation segments includes: Segmenting the standardized audio data based on the beat data to obtain a plurality of initial segmented segments; The initial segmented segments are padded or truncated based on the average beat frame to obtain the processed audio segmented segments.
7. A harmony recognition method, characterized in that: The method comprises: Segmenting the audio data to be recognized based on the beat data of the audio data to be recognized to obtain a plurality of audio recognition segments; Extracting features from the audio recognition segment to obtain audio feature data to be recognized; The audio feature data to be identified is input into a trained harmony recognition model to obtain a harmony recognition result output by the trained harmony recognition model; wherein the harmony recognition model is trained based on the harmony recognition model training method as described in any one of claims 1 to 6 above.
8. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.
9. An electronic device, characterized in that: The electronic device comprises: Memory; processor; The memory stores a computer program executable by the processor, and when the computer program is executed by the processor, the method according to any one of claims 1 to 7 is performed.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is executed.
Citation Information
Patent Citations
Chord recognition method combining SVM with enhanced PCP
CN103714806A
Song climax fragment identification method and device
CN111081272A
Training method of beat re-shooting joint detection model and beat re-shooting joint detection method
CN114154574A
Musical instrument music signal and chord identification method
CN115083373A
Information reproducing device
JP1995153190A
