A music structure analysis method and device based on multi-task learning

Abstract

The application relates to the technical field of audio processing, in particular to a music structure analysis method and device based on multi-task learning. The method comprises the following steps: obtaining audio data and preprocessing to obtain target audio data; extracting target audio features; calling a target neural network model, inputting the target audio features, and outputting boundary probability values and label probability values of an audio paragraph; determining target boundary values and target label values among all the boundary probability values and the label probability values of the audio paragraph based on preset determination rules, and determining target boundaries and target labels of the audio paragraph according to the target boundary values and the target label values; and through multi-task learning recognition of paragraph boundaries and paragraph labels of music, the method solves the limitation and dependence of existing schemes on artificial experience features and other media feature data, reduces the workload and the requirement for prior knowledge in engineering, and can improve the precision of music structure paragraph segmentation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of audio processing, in particular to a music structure analysis method and device based on multi-task learning. BACKGROUND

[0002] The difference between music and random sound source is that music has complex hierarchical structure, and there are differences in tone, pitch, duration and pronunciation for a single note. Higher level has the theme, section and chapter of music, and there are more hierarchical structures. For example, popular music is composed of prelude, chorus and main song. The main purpose of applying computer to music structure analysis is to distinguish the corresponding parts of a song, and some parts may appear many times. Among the existing numerous music structure analysis methods, some are based on predetermined audio features to determine the positions of music chorus and other sections, some are based on text semantic similarity to determine different sections, and some are based on music audio data and corresponding text data to predict the boundary points of music sections after convex non-negative matrix decomposition. The existing music structure analysis based on audio, text or their combination has great limitations and dependence on artificial experience features and other media feature data, and has high requirements for prior knowledge and analysis engineering workload, and the precision of music structure section segmentation is low. SUMMARY

[0003] In view of the problems, the present application is proposed to provide a music structure analysis method and device based on multi-task learning to overcome the problems or at least partially solve the problems, comprising:

[0004] A music structure analysis method based on multi-task learning, comprising:

[0005] Obtaining audio data of music to be analyzed, and pre-processing the audio data to obtain target audio data;

[0006] Extracting target audio features in the target audio data;

[0007] Calling a preset target neural network model, and outputting boundary probability values of audio sections and label probability values corresponding to the audio sections through the target neural network model according to the target audio features as input; wherein the audio data contains at least one audio section;

[0008] Based on a preset determination rule, determining target boundary values and target label values among all the boundary probability values and the label probability values of the audio section, and determining the target boundary and the target label of the audio section according to the target boundary values and the target label values.

[0009] Preferably, the audio data of the music to be analyzed is acquired, and the audio data is preprocessed to obtain target audio data, comprising:

[0010] A target link address input by a user is acquired, and the audio data is acquired according to the target link address;

[0011] A preset audio transcoding program is called, and the audio data is transcoded according to a preset target sampling rate by the audio transcoding program to obtain the target audio data.

[0012] Preferably, the target audio features in the target audio data are extracted, comprising:

[0013] A preset audio feature extraction program is called, and the target audio features are extracted according to a preset target extraction strategy by the audio feature extraction program to obtain the target audio features.

[0014] Preferably, before the preset target neural network model is called, comprising:

[0015] An audio sample training set is acquired;

[0016] Based on a preset neural network framework, the audio sample training set is input to train a deep learning network module in the neural network framework; wherein the deep learning network module comprises a spectrum enhancement layer, a two-dimensional convolution layer, a batch normalization layer, a nonlinear activation layer, an average pooling layer, a self-attention layer, a Dropout layer, an ADD layer, a LogSoftmax, and a Sigmiod layer connected in sequence;

[0017] According to the trained deep learning network module, boundary probability values and label probability values corresponding to the audio sample training set are output.

[0018] Preferably, before the preset neural network framework is input to train the deep learning network module in the neural network framework, it further comprises:

[0019] A preset labeling program is called to determine the audio paragraphs contained in each audio sample data in the audio sample training set and the corresponding labels.

[0020] Preferably, based on a preset determination rule, target boundary values and target label values are determined from all the boundary probability values and the label probability values in the audio paragraph, comprising:

[0021] A boundary reference value is determined according to the determination rule;

[0022] If the boundary reference value is greater than any boundary probability value in the audio paragraph, the boundary reference value is taken as the target boundary value of the audio paragraph.

[0023] If one of the boundary probability values in the audio paragraph is greater than the boundary reference value, the boundary probability value is taken as the target boundary value of the audio paragraph.

[0024] Preferably, the target boundary and the target label of the audio paragraph are determined according to the target boundary value and the target label value, comprising:

[0025] All boundary probability values contained in the target boundary value of the audio paragraph and corresponding label probability values are determined.

[0026] The average probability value of each label is calculated according to all corresponding label probability values.

[0027] The label with the maximum average probability value is determined as the target label.

[0028] A music structure analysis device based on multi-task learning is also provided, comprising:

[0029] An acquisition module is configured to acquire audio data of music to be analyzed, and pre-process the audio data to obtain target audio data.

[0030] An extraction module is configured to extract target audio features from the target audio data.

[0031] A processing module is configured to call a preset target neural network model, and output boundary probability values of an audio paragraph and label probability values corresponding to the audio paragraph by taking the target audio features as input of the target neural network model; wherein the audio data contains at least one audio paragraph.

[0032] A determination module is configured to determine a target boundary value and a target label value based on a preset determination rule among all boundary probability values and label probability values of the audio paragraph, and determine a target boundary and a target label of the audio paragraph according to the target boundary value and the target label value.

[0033] To achieve the present application further includes an electronic device, comprising a processor, a memory and a computer program stored on the memory and capable of running on the processor, the computer program is executed by the processor to realize the steps of the above method.

[0034] To achieve the present application a computer readable storage medium, the computer readable storage medium stores a computer program, the computer program is executed by the processor to realize the steps of the above method.

[0035] The present application has the following advantages:

[0036] In the embodiments of the present application, by acquiring audio data of music to be analyzed, and preprocessing the audio data to obtain target audio data, target audio features in the target audio data are extracted; a preset target neural network model is called, and the target neural network model outputs boundary probability values of audio paragraphs and label probability values corresponding to the audio paragraphs according to the target audio features as input; wherein the audio data contains at least one audio paragraph; based on a preset determination rule, target boundary values and target label values are determined among all the boundary probability values and the label probability values of the audio paragraph, and the target boundary and the target label of the audio paragraph are determined according to the target boundary values and the target label values. Through the deep network model and post-processing, the restriction and dependence of the existing scheme on artificial experience features and other media feature data are solved, the workload and the requirement for priori knowledge in engineering are reduced, and the accuracy of music structure paragraph segmentation is improved. BRIEF DESCRIPTION OF DRAWINGS

[0037] In order to more clearly illustrate the technical solutions of the present application, the drawings needed to be used in the description of the present application will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0038] Figure 1 is a step flow chart of a music structure analysis method based on multi-task learning provided by an embodiment of the present application;

[0039] Figure 2 is a structural schematic diagram of a neural network model of a music structure analysis method based on multi-task learning provided by an embodiment of the present application;

[0040] Figure 3 is a structural schematic diagram of a music structure analysis device based on multi-task learning provided by an embodiment of the present application;

[0041] Figure 4 is a structural schematic diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0042] In order to make the purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below in combination with the drawings and specific embodiments. Obviously, the described embodiments are some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0043] Referring to Figure 1 , a step flow chart of a music structure analysis method based on multi-task learning provided by an embodiment of the present application is shown, and specifically includes the following steps:

[0044] S110, audio data of music to be analyzed is obtained, and the audio data is preprocessed to obtain target audio data;

[0045] S120, target audio features in the target audio data are extracted;

[0046] S130, a preset target neural network model is called, and the target neural network model outputs boundary probability values of audio paragraphs and label probability values corresponding to the audio paragraphs according to the target audio features as input; wherein the audio data contains at least one audio paragraph;

[0047] S140, based on a preset determination rule, target boundary values and target label values are determined among all the boundary probability values and the label probability values of the audio paragraph, and a target boundary and a target label of the audio paragraph are determined according to the target boundary values and the target label values.

[0048] By obtaining audio data of music to be analyzed, and preprocessing the audio data to obtain target audio data; extracting target audio features in the target audio data; calling a preset target neural network model, and outputting boundary probability values of audio paragraphs and label probability values corresponding to the audio paragraphs according to the target audio features as input by the target neural network model; wherein the audio data contains at least one audio paragraph; based on a preset determination rule, target boundary values and target label values are determined among all the boundary probability values and the label probability values of the audio paragraph, and a target boundary and a target label of the audio paragraph are determined according to the target boundary values and the target label values, through multi-task learning recognition of the paragraph boundary and the paragraph label of the music, the limitation and dependence of the prior art on artificial experience features and other media feature data are solved, the workload in engineering and the requirement for prior knowledge are reduced, and the precision of music structure paragraph segmentation is improved.

[0049] In the following, the above-mentioned music structure analysis method based on multi-task learning will be further described through the following embodiments.

[0050] As described in step S110, audio data of music to be analyzed is obtained, and the audio data is preprocessed to obtain target audio data.

[0051] In an embodiment of the present application, the specific process of "obtaining audio data of music to be analyzed, and pre-processing the audio data to obtain target audio data" in step S110 can be further illustrated in combination with the following description.

[0052] As described in the following steps, a target link address input by a user is obtained, and the audio data is obtained according to the target link address; a preset audio transcoding program is called, and the audio data is transcoded according to a preset target sampling rate by the audio transcoding program to obtain the target audio data.

[0053] As an example, according to the target link address input by the user, it is determined whether the audio data is obtained from the ontology file or http; an application program for audio transcoding, such as FFmpeg, can be called to transcode the audio in the audio data into audio data of single-shaped numbers with a sampling rate of 22050 Hz, which is convenient for subsequent feature extraction.

[0054] As described in step S120, target audio features in the target audio data are extracted.

[0055] In an embodiment of the present application, the specific process of "extracting target audio features in the target audio data" in step S120 can be further illustrated in combination with the following description.

[0056] As described in the following steps, a preset audio feature extraction program is called, and the target audio features are extracted according to a preset target extraction strategy by the audio feature extraction program to obtain the target audio features.

[0057] As an example, an audio feature extraction program, such as librosa, can be called, and the extraction strategy can be to extract 128-dimensional Fbank features.

[0058] It should be noted that according to the above sampling rate of 22050 Hz, the sampling value of each frame in the target audio data can be set to 2048, and the step length is 512.

[0059] As described in step S130, a preset target neural network model is called, and the target neural network model outputs a boundary probability value of an audio paragraph and a label probability value corresponding to the audio paragraph according to the target audio features as input; wherein the audio data contains at least one audio paragraph.

[0060] In an embodiment of the present application, the specific process of "before calling the preset target neural network model" in step S130 can be further illustrated in combination with the following description.

[0061] An audio sample training set is obtained according to the following steps; a deep learning network module in a preset neural network framework is trained based on the audio sample training set; wherein the deep learning network module comprises a spectrum enhancement layer, a two-dimensional convolution layer, a batch normalization layer, a nonlinear activation layer, an average pooling layer, a self-attention layer, a Dropout layer, an ADD layer, a LogSoftmax layer, and a Sigmiod layer connected in sequence; and the boundary probability value and the label probability value corresponding to the audio sample training set are output by the deep learning network module obtained by training.

[0062] As an example, as shown in (a) is a neural network framework, and (b) is a convolutional network sub-module in the deep learning network framework. Figure 2

[0063] The neural network framework comprises: an Audio module for inputting an audio with a sampling rate of 22050 Hz; a FeaturePre-Processing module for feature preprocessing, extracting Fbank features from the audio; a DNN module representing a deep learning network module; a boundary probability representing a boundary probability value output by the DNN module, and a segment probability representing a label probability value output by the DNN module; and a Post-Processing module for processing the boundary probability value and the label probability value output by the former to obtain the final boundary and label of each paragraph in the audio data.

[0064] In the DNN module, “x4” represents that the same structure is stacked 4 times; the SpecAugment layer represents a spectrum enhancement layer for randomly masking the Fbank features in the time and frequency dimensions to enhance the performance of the model; the Conv2D represents a two-dimensional convolution layer; the Conv1D represents a one-dimensional convolution layer; the BatchNorm represents a batch normalization layer; the ELU represents a nonlinear activation layer for increasing the nonlinear expression ability of the model; the AveragePooling represents an average pooling; the Self-Attention represents a self-attention layer for enabling the model to learn the global correlation weight of the input data to obtain the importance of each local information and to focus more attention on important data parts; the Dropout layer is used for input data, and some positions of the data are set to 0 at a random probability to reduce the overfitting risk of the model; the ADD layer is used for adding two input data; the LogSoftmax layer is used for performing Softmax and Log operations on the input data in sequence; and the Sigmiod layer is used for performing Sigmiod operation on the input data.

[0065] ​It should be noted that each audio paragraph represents two adjacent boundary probability values, and the audio paragraph defined by the boundary values determines the corresponding label probability. For example, the starting time node of the audio paragraph is determined according to the preceding boundary probability value of the audio paragraph, and the ending time node of the audio paragraph is determined according to the following boundary probability value, and then the label probability value of the audio paragraph between the starting time node and the ending time node is corresponded, such as the probability value of the intro and the probability value of the chorus; of course, through model calculation, we only output the maximum label probability value.

[0066] In the embodiment of the present application, the preset neural network framework is used to train the deep learning network module in the neural network framework based on the audio sample training set, and the method further comprises the following steps:

[0067] A preset labeling program is called to determine the audio paragraph contained in each audio sample data in the audio sample training set and the corresponding label.

[0068] It should be noted that the Praat application can be used to label the audio sample training set to obtain the audio paragraph contained in each audio sample data and the corresponding label. Then, each audio data can be cut into several data segments for training according to a moving step of 1s.

[0069] As described in step S140, the preset determination rule is used to determine a target boundary value and a target label value from all the boundary probability values and the label probability values of the audio paragraph, and determine a target boundary and a target label of the audio paragraph according to the target boundary value and the target label value.

[0070] In an embodiment of the present application, the specific process before calling the preset target neural network model in step S130 can be further described in combination with the following description.

[0071] As described in the following steps, the boundary reference value is determined according to the determination rule; if the boundary reference value is greater than any boundary probability value in the audio paragraph, the boundary reference value is taken as the target boundary value of the audio paragraph; if one of the boundary probability values in the audio paragraph is greater than the boundary reference value, the boundary probability value is taken as the target boundary value of the audio paragraph.

[0072] It should be noted that the best threshold of the boundary output is calculated by calculating the average F1 index in all possible threshold ranges [0, 1] in the verification set data, and the boundary threshold corresponding to the highest average F1 index value is selected as the best boundary threshold, and the result greater than the best boundary threshold is taken as the reference boundary, and then the application rule of the reference boundary is applied: if each candidate boundary remains the maximum value within 10 seconds after the candidate boundary, the boundary is taken as the final boundary result, and N segments will be formed. The F1 index is a statistical index used to measure the accuracy of a binary classification (or multi-task binary classification) model.

[0073] In an embodiment of the present application, the target boundary and the target label of the audio segment are determined according to the target boundary value and the target label value, comprising: determining all boundary probability values and corresponding label probability values contained in the target boundary value of the audio segment; calculating the average probability value of each label according to all corresponding label probability values; and determining the label with the maximum average probability value as the target label.

[0074] It should be noted that after obtaining N segments, the average probability value of each label is calculated in each segment, and the maximum label is selected as the final label of the segment.

[0075] The present application can automatically complete the music structure analysis task, and the processing procedure is simple, effective and fast. The deep network model and post-processing solve the limitation and dependence of existing solutions on artificial experience features and other media feature data, reduce the engineering workload and the requirement for prior knowledge, and improve the accuracy of music structure segment division.

[0076] For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the related parts can be referred to the part of the method embodiment.

[0077] Referring to Figure 3 , a structure schematic diagram of a music structure analysis device based on multi-task learning is shown, and the device comprises the following modules:

[0078] The acquisition module 110 is configured to acquire audio data of music to be analyzed, and pre-process the audio data to obtain target audio data.

[0079] The extraction module 120 is configured to extract target audio features in the target audio data.

[0080] The processing module 130 is configured to call a preset target neural network model, and output boundary probability values of audio segments and label probability values corresponding to the audio segments by taking the target audio features as input according to the target neural network model; wherein the audio data comprises at least one audio segment.

[0081] The determining module 140 is configured to determine a target boundary value and a target label value from all the boundary probability values and the label probability values of the audio paragraph based on a preset determination rule, and determine a target boundary and a target label of the audio paragraph according to the target boundary value and the target label value.

[0082] Referring to Figure 4 , a computer device of a music structure analysis method based on multi-task learning is shown, which can specifically include the following:

[0083] The computer device 12 is in the form of a general-purpose computing device, and the components of the computer device 12 can include but are not limited to one or more processors or processing units 16, a memory 28, and a bus 18 connecting different system components including the memory 28 and the processing unit 16.

[0084] The bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor or a local bus using any of a variety of bus architectures. For example, these architectures include but are not limited to an industry standard architecture (ISA) bus, a microchannel architecture (MAC) bus, an enhanced ISA bus, a video electronics standards association (VESA) local bus, and a peripheral component interconnect (PCI) bus.

[0085] The computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 12 and includes both volatile and nonvolatile media, removable and non-removable media.

[0086] The memory 28 can include computer system readable media in the form of volatile memory, such as random access memory 30 and / or cache memory 32. The computer device 12 can further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, a storage system 34 can be provided for reading from and writing to non-removable, non-volatile magnetic media (typically called a "hard drive"). Figure 4 Although not shown in the computer device 12, a magnetic disk drive for reading from and writing to a removable, non-volatile magnetic data storage disk (such as a "floppy disk"), and an optical disk drive for reading from or writing to a removable, non-volatile optical disk (such as a CD-ROM, DVD-ROM or other optical media) can be provided. In such instances, each drive can be connected to the bus 18 by one or more data media interfaces. The memory can include at least one program product having a set (e.g., at least one) of program modules 42 configured to carry out the functions of embodiments of the application.

[0087] Program / utility 40 having a set of program modules 42 can be stored in memory 26, for example, including an operating system, one or more application programs, other program modules 42, and program data, each or some combination thereof, which implement the embodiments described herein. Program modules 42 generally carry out the functions and / or methodologies of embodiments of the application as described herein.

[0088] Computer device 12 can also communicate with one or more external devices 14 such as a keyboard or pointing device, a display 24, camera, etc. and also with one or more devices that enable a human user to interact with computer device 12 and / or with any devices (e.g., network cards, modems, etc.) that enable computer device 12 to communicate with one or more other computing devices. Such communication can be via I / O interface 22. Furthermore, computer device 12 can communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or the Internet) through network adapter 20. As Figure 4 illustrated, network adapter 20 communicates with the other components of computer device 12 via bus 18. It should be understood that although not shown, other hardware and / or software modules could be used in conjunction with computer device 12. Examples, include, but are not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems 34, etc. Figure 4 Processing unit 16 performs various functions and data processing by running programs stored in memory 28, such as implementing a music structure analysis method based on multi-task learning provided by embodiments of the present application.

[0089] That is, the above processing unit 16 implements the following when executing the above program: obtaining audio data of music to be analyzed, and pre-processing the audio data to obtain target audio data; extracting target audio features in the target audio data; calling a preset target neural network model, and outputting boundary probability values of audio paragraphs and label probability values corresponding to the audio paragraphs by the target neural network model according to the target audio features as input; wherein the audio data contains at least one audio paragraph; determining target boundary values and target label values among all the boundary probability values and the label probability values of the audio paragraph based on a preset determination rule, and determining a target boundary and a target label of the audio paragraph according to the target boundary values and the target label values.

[0090]

[0091] ​In the embodiments of the present application, the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the music structure analysis method based on multi-task learning provided by all the embodiments of the present application.

[0092] That is, the computer program is executed by the processor to implement the following: obtaining audio data of music to be analyzed, and pre-processing the audio data to obtain target audio data; extracting target audio features in the target audio data; calling a preset target neural network model, and outputting boundary probability values of audio paragraphs and label probability values corresponding to the audio paragraphs by the target neural network model according to the target audio features as input; wherein the audio data contains at least one audio paragraph; determining target boundary values and target label values among all the boundary probability values and the label probability values of the audio paragraph based on a preset determination rule, and determining a target boundary and a target label of the audio paragraph according to the target boundary values and the target label values.

[0093] Any combination of one or more computer readable medium can be employed. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the above. More specific examples (a non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer readable storage medium can be any tangible medium that contains or stores a program used by or in connection with an instruction execution system, apparatus, or device.

[0094] The computer readable signal medium can include a data signal propagated in a baseband or propagated as a carrier wave in a propagated data signal, in which the computer readable program code is embodied. Such a propagated data signal can take a variety of forms, including, but not limited to, an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium can also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

[0095] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). The embodiments of the present application described above are implemented in a manner as follows, and each of the embodiments focuses on the difference from other embodiments. The same or similar parts among the embodiments are cross-referenced.

[0096] While the preferred embodiments of the application have been described above, it should be understood that many modifications and adaptations to the described embodiments will occur to those skilled in the art once given this teaching. The foregoing description is considered as illustrative only of the principles of the application. Further, since numerous modifications and changes will readily occur to those skilled in the art, it is not desired to limit the application to the exact construction and practice described. Accordingly, the scope should be limited only by the claims appended hereto and equivalents thereof.

[0097] Finally, it is also necessary to point out that, in this text, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or terminal device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or terminal device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or terminal device including the element.

[0098] The above provides a music structure analysis method and device based on multi-task learning. The principles and implementation manners of the present application are described by specific examples. The above embodiment description is only used to help understand the method and core idea of the present application. Meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manner and application range will be changed. The above description should not be understood as a limitation of the present application.

Claims

1. A music structure analysis method based on multi-task learning, characterized in that, include: Acquire the audio data of the music to be analyzed, and preprocess the audio data to obtain the target audio data; Extract the target audio features from the target audio data; A preset target neural network model is invoked. The target neural network model takes the target audio features as input and outputs boundary probability values ​​for each time position in the audio data as the boundary of the audio segment, as well as label probability values ​​corresponding to the audio segments defined by the boundary probability values; wherein, the audio data contains at least one of the audio segments. Based on preset determination rules, target boundary value and target tag value are determined from all the boundary probability values ​​and tag probability values ​​of the audio segment, and the target boundary and target tag of the audio segment are determined based on the target boundary value and target tag value.

2. The method according to claim 1, characterized in that, The process of acquiring the audio data of the music to be analyzed and preprocessing the audio data to obtain the target audio data includes: Obtain the target link address input by the user, and obtain the audio data based on the target link address; A preset audio transcoding program is invoked, and the audio data is transcoded according to a preset target sampling rate to obtain the target audio data.

3. The method according to claim 1, characterized in that, The extraction of target audio features from the target audio data includes: A preset audio feature extraction program is invoked, and the target audio features are extracted by the audio feature extraction program according to a preset target extraction strategy to obtain the target audio features.

4. The method according to claim 1, characterized in that, Before invoking the preset target neural network model, the following steps are included: Obtain the audio sample training set; Based on a preset neural network framework, the deep learning network module in the neural network framework is trained by inputting the audio sample training set; wherein, the deep learning network module includes a spectral enhancement layer, a two-dimensional convolutional layer, a batch normalization layer, a non-linear activation layer, an average pooling layer, a self-attention layer, a Dropout layer, an ADD layer, a LogSoftmax layer, and a Sigmoid layer connected in sequence. The deep learning network module outputs boundary probability values ​​and label probability values ​​corresponding to the audio sample training set based on the training results.

5. The method according to claim 4, characterized in that, The process of training the deep learning network module within the preset neural network framework using the audio sample training set, prior to which includes: A preset annotation program is invoked to determine the audio segments and corresponding labels contained in each audio sample data in the audio sample training set.

6. The method according to claim 1, characterized in that, Based on preset determination rules, target boundary values ​​and target label values ​​are determined from all the boundary probability values ​​and label probability values ​​in the audio segment, including: Determine the boundary reference value according to the aforementioned determination rules; If the boundary reference value is greater than any boundary probability value in the audio segment, then the boundary reference value is used as the target boundary value of the audio segment. If one of the boundary probability values ​​in an audio segment is greater than the boundary reference value, that boundary probability value is used as the target boundary value for that audio segment.

7. The method according to claim 6, characterized in that, Determining the target boundary and target label of the audio segment based on the target boundary value and target label value includes: Determine all the boundary probability values ​​and corresponding label probability values ​​contained in the target boundary values ​​of the audio segment; Calculate the average probability value of each label based on all the corresponding label probability values; The label with the highest average probability value is determined as the target label.

8. A music structure analysis device based on multi-task learning, characterized in that, include: The acquisition module is used to acquire the audio data of the music to be analyzed and to preprocess the audio data to obtain the target audio data. The extraction module is used to extract target audio features from the target audio data; The processing module is used to call a preset target neural network model, and output the boundary probability value of each time position in the audio data as the boundary of the audio segment, and the label probability value corresponding to the audio segment defined by the boundary probability value, based on the target audio features as input; wherein the audio data contains at least one of the audio segments; The determination module is used to determine the target boundary value and the target label value among all the boundary probability values ​​and the label probability values ​​of the audio segment based on preset determination rules, and to determine the target boundary and the target label of the audio segment based on the target boundary value and the target label value.

9. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the steps of the music structure analysis method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the steps of the music structure analysis method as described in any one of claims 1 to 7.

Images