Music processing method and apparatus

Abstract

The embodiment of the present application provides a music processing method, the method comprises the following steps: performing sound source separation on target music to obtain at least two media resources, at least one of the media resources corresponds to one music component; extracting a first feature of each of the media resources; determining a chorus segment of the target music according to the first feature. The music processing method provided by the embodiment of the present application can improve the efficiency of chorus recognition and the accuracy of chorus recognition.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a music processing method, apparatus, computer device, and storage medium. Background Technology

[0002] In popular songs, the chorus often appears multiple times and usually features rich instrumentation and complex rhythmic patterns. The chorus is generally the most infectious and captivating part of the music. In short video background music production, to attract users within a limited time, chorus segments are often chosen as background music to quickly grab their attention.

[0003] Chorus detection is a challenging problem in music signal processing. Currently, the methods for selecting the chorus generally fall into the following two categories:

[0004] (1) Manual selection of the chorus based on visual and auditory judgment. Although this method has a high accuracy rate, it requires a lot of manpower and is inefficient.

[0005] (2) Using computer equipment to select the chorus. Most of these methods are based on pre-assumptions about the loudness, frequency and rhythm density of the sound in the music. Although these methods are more efficient than manual methods, such pre-assumptions limit the generalization ability of the method, resulting in low accuracy of chorus recognition. Summary of the Invention

[0006] The purpose of this application is to provide a music processing method, apparatus, computer equipment, and storage medium to solve the technical problem of low efficiency and accuracy in current chorus recognition.

[0007] One aspect of this application provides a music processing method, comprising: separating the sound sources of target music to obtain at least two media resources, at least one of the media resources corresponding to a musical component; extracting a first feature of each of the media resources; and determining the chorus segment of the target music based on the first feature.

[0008] Optionally, the step of separating the target music into sound sources to obtain at least two media resources includes: separating the target music into sound sources to obtain media resources corresponding to the overall target music and media resources corresponding to the vocal component, drum component, and bass component, respectively.

[0009] Optionally, the step of extracting a first feature of each media resource and determining the chorus segment of the target music based on the first feature includes: extracting frequency domain features of each media resource; determining a self-similarity matrix of each media resource based on the frequency domain features; inputting the self-similarity matrix into a recognition model to determine the probability that each frame of the target music is a chorus through the recognition model; and determining the chorus segment of the target music based on the probability.

[0010] Optionally, extracting the frequency domain features of each of the media resources includes: extracting the frequency domain features of each of the media resources in the Mel spectrum or CQT spectrum.

[0011] Optionally, the recognition model is trained by a convolutional neural network.

[0012] Optionally, the recognition model is trained using a convolutional neural network and Self Attention.

[0013] Optionally, determining the chorus segment of the target music based on the probability includes: determining whether the probability of each frame in the target music exceeds a preset threshold; if the probability of the current frame in the target music is greater than or equal to the preset threshold, determining that the current frame belongs to the chorus; and determining the chorus segment of the target music based on whether some or all frames in the target music belong to the chorus.

[0014] One aspect of this application provides a music processing apparatus, comprising: a separation module for separating the sound source of target music to obtain at least two media resources, at least one of the media resources corresponding to a musical component; an extraction module for extracting a first feature from each of the media resources; and a determination module for determining a chorus segment of the target music based on the first feature.

[0015] One aspect of this application provides a computer device, the computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the music processing method described above.

[0016] Another aspect of this application provides a computer-readable storage medium storing a computer program that can be executed by at least one processor to perform the steps of the music processing method described above.

[0017] The music processing method, apparatus, computer equipment, and storage medium provided in this application have the following advantages:

[0018] By separating the sound source of the target music to obtain at least two media resources, at least one media resource corresponds to a musical component, and a first feature is extracted from each media resource. The chorus segment of the target music is determined based on the first feature. Since the chorus segment of the target music can be automatically identified using computer equipment, it is more efficient than manual methods. Furthermore, since the chorus of the target music is identified based on the features of media resources containing at least one musical component, different judgments can be made for different types of music. This method has a higher generalization ability than methods that rely on pre-assumptions, resulting in higher accuracy. Attached Figure Description

[0019] Figure 1 An environmental architecture diagram illustrating an embodiment of this application is shown schematically;

[0020] Figure 2 A flowchart illustrating the music processing method of Embodiment 1 of this application is shown schematically;

[0021] Figure 3 for Figure 2 Flowcharts of the sub-steps of steps S420 and S430;

[0022] Figure 4 Example diagram of the network structure for identifying the model;

[0023] Figure 5 for Figure 3 Flowchart of the sub-steps of step S540;

[0024] Figure 6 This is a schematic diagram of the binarization process;

[0025] Figure 7 A specific example diagram of music processing methods;

[0026] Figure 8 A block diagram of the music processing apparatus according to Embodiment 2 of this application is shown schematically;

[0027] Figure 9 The schematic diagram illustrates the hardware architecture of the computer device according to Embodiment 3 of this application. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0029] It should be noted that the descriptions involving "first," "second," etc., in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0030] In the description of this application, it should be understood that the numerical labels before the steps do not indicate the order of the steps, but are only used to facilitate the description of this application and to distinguish each step, and therefore should not be construed as a limitation of this application.

[0031] The following is an explanation of the terms used in this application:

[0032] A refrain (also known as a chorus) is a repeated lyric or musical section in a song. It usually appears between several verses (also called the main verses), that is, after the first verse reaches the refrain, it connects to the second verse and then returns to the refrain, and so on.

[0033] Musical components: These refer to the constituent parts of music, such as the sounds of human voices and instruments (drums, bass, guitar), which make up a piece of music.

[0034] Figure 1 An environmental architecture diagram illustrating an embodiment of this application is shown in the figure:

[0035] Client 300 connects to server 100 via network 200. There can be multiple clients 300. Server 100 receives music uploaded by clients 300 via network 200. After receiving the uploaded music, it performs sound source separation to obtain at least two media resources, with at least one media resource corresponding to a musical component. Then, server 100 extracts the features of each media resource and determines the chorus segment of the current music based on the extracted features, thereby completing the identification of the chorus part in the music.

[0036] In an exemplary embodiment, server 100 may refer to a data center, such as a single house, or distributed across different geographical locations (e.g., across several houses). Server 100 may provide services through one or more networks 200.

[0037] Network 200 includes various network devices, such as routers, switches, multiplexers, hubs, modems, bridges, repeaters, firewalls, agent devices, and / or similar devices. Network 200 may include physical links, such as coaxial cable links, twisted-pair cable links, fiber optic links, combinations thereof, and / or similar combinations. Network 200 may include wireless links, such as cellular links, satellite links, Wi-Fi links, and / or similar combinations.

[0038] Client 300 may include devices such as mobile devices, tablets, laptops, smart devices (e.g., smart clothing, smartwatches, smart glasses), virtual reality headsets, gaming devices, set-top boxes, digital streaming devices, robots, in-vehicle terminals, smart TVs, TV boxes, or e-book readers.

[0039] In related technologies, manually determining the chorus in music is inefficient; while determining the chorus by making pre-assumptions about explicit features in music has weak generalization ability and low recognition accuracy.

[0040] The music processing scheme of this application embodiment can improve recognition efficiency while improving recognition accuracy.

[0041] The music processing scheme of this application will be described below through several embodiments. For ease of understanding, the following will use examples... Figure 1 The server 100 in the example is described as the execution entity.

[0042] Example 1

[0043] Figure 2 The schematic diagram illustrates a music processing method according to Embodiment 1 of this application, including steps S410 to S430, which are described in detail below:

[0044] Step S410: Separate the sound sources of the target music to obtain at least two media resources, with at least one media resource corresponding to a musical component.

[0045] For example, source separation of target music can be achieved by inputting the target music into a deep source separation network, thereby separating the target music into several musical components. Alternatively, the target music can be input into an audio track separation tool for source separation, such as Spleeter, which is a separation tool based on the U-Net (a convolutional neural network method) network structure.

[0046] At least two media resources are required. For example, it could be a media resource consisting of vocals and a media resource consisting of drums; or a media resource consisting of vocals, a media resource consisting of drums, and a media resource consisting of bass. In particular, media resources can also include the entire resource of the target music, that is, one of the at least two media resources can be the target music itself.

[0047] In an exemplary embodiment, step S410 may include: separating the target music into sound sources to obtain media resources corresponding to the overall target music and media resources corresponding to the vocal component, drum component, and bass component, respectively. That is, the target music is divided into four media resources: vocal component, drum component, bass component, and the overall target music. Since the vocal, drum, bass, and overall music are different, their corresponding features naturally differ. Therefore, dividing the target music into these four media resources allows for the comprehensive determination of the chorus section using the different features of the four media resources, which is beneficial for improving the chorus recognition efficiency. In particular, the drum and bass components, because their variations in the main and chorus sections (such as changes in striking force and frequency) are relatively obvious, can be used to determine the chorus segment of the target music, thus significantly improving the accuracy of chorus recognition. Furthermore, since most music choruses contain vocals, drums, and bass, analyzing the target music by dividing it into these four media resources allows for effective identification of the chorus. Even if the chorus doesn't contain one of these components (e.g., drums or bass), the other media resources can still be used for effective identification. Optionally, different component extraction configurations can be applied based on different music styles (genres). For example, for rock music, the configuration can extract vocals, drums, and bass; for jazz, it can extract saxophone, violin, and drums, and so on.

[0048] Step S420: Extract the first feature of each media resource.

[0049] Since media resources are in the time domain, to facilitate their analysis, they can be converted from the time domain to the frequency domain, and the frequency domain features can be extracted as the primary features of each media resource. For example, a Fourier transform can be used to convert media resources from the time domain to the frequency domain, and then the frequency domain features can be extracted.

[0050] Step S430: Determine the chorus segment of the target music based on the first feature.

[0051] For example, server 100 can input the first feature into a pre-trained detection model to determine the chorus segment of the target music. Specifically, server 100 can prepare a certain number of music samples in advance, perform sound source separation on the music samples to obtain several media resources; extract the first feature from each media resource, and then train the pre-built detection model using the first feature. After the detection model reaches a certain detection accuracy, the pre-trained detection model is obtained. Alternatively, the detection model can determine the probability that a frame in the media resource is the chorus. After obtaining the probability that some frames or all frames are the chorus, the chorus segment of the target music can be obtained accordingly.

[0052] In related technologies, there are methods that determine the chorus by extracting the overall audio information of the music. However, because the overall audio information is quite complex, this method cannot clearly distinguish the segments in the music, resulting in low accuracy in chorus recognition. The music processing method of this application, however, obtains at least one media resource corresponding to a musical component after separating the target music. Therefore, it can acquire finer-grained features from the target music and, based on these features, clearly distinguish the segments in the music, thus improving the accuracy of chorus recognition.

[0053] The music processing method provided in this application separates the sound source of the target music to obtain at least two media resources, with at least one media resource corresponding to a musical component. A first feature is extracted from each media resource, and the chorus segment of the target music is determined based on the first feature. Since the chorus segment of the target music can be automatically identified using computer equipment, it is more efficient than manual methods. Furthermore, since the chorus of the target music is identified based on the features of media resources containing at least one musical component, different judgments can be made for different types of music. This method has a higher generalization ability and thus a higher accuracy rate compared to methods that rely on pre-assumptions for judgment.

[0054] In an exemplary embodiment, steps S420 and S430 involve extracting a first feature from each media resource and determining the chorus segment of the target music based on the first feature, such as... Figure 3 As shown, it may include steps S510 to S540, as detailed below:

[0055] Step S510: Extract the frequency domain features of each media resource.

[0056] In an exemplary embodiment, extracting the frequency domain features of each media resource may include: extracting the frequency domain features of each media resource in the Mel spectrum or the CQT (Constant Q Transform) spectrum. Preferably, the frequency domain features of each media resource in the CQT spectrum are extracted. In music, all notes are composed of several octaves of equal temperament, which corresponds to the twelve semitones in an octave on a piano, with a frequency ratio of 21 / 12 between adjacent semitones. Clearly, for two octaves of the same pitch, the higher octave has twice the frequency of the lower octave. Therefore, in music, sound is exponentially distributed. However, the audio spectrum obtained by the commonly used Fourier transform is linearly distributed, and the frequency points of the two cannot correspond one-to-one, leading to errors in the estimation of certain pitch frequencies. CQT, on the other hand, is a filter bank with a center frequency distributed exponentially, different filtering bandwidths, but a constant center frequency to bandwidth ratio Q. Unlike the Fourier transform, the horizontal axis frequency of the CQT spectrum is not linear but exponentially distributed. Since the CQT spectrum has the same frequency distribution as the musical scale, the amplitude value of the musical signal at each note frequency can be directly obtained by calculating the CQT spectrum of the musical signal, which is beneficial for the processing of musical signals.

[0057] Step S520: Determine the self-similarity matrix of each media resource based on the frequency domain characteristics.

[0058] Specifically, the self-similarity matrix of each media resource can be calculated using the following formula:

[0059] SSM = X·X T ;

[0060] Where X is the feature sequence of the frequency domain features of each media resource, X T This is the transpose of the feature sequence.

[0061] For example, the target music can be divided into four media resources: vocals, drums, bass, and the whole music, and then the self-similarity matrix of the frequency domain features of each media resource can be calculated separately.

[0062] Optionally, after calculating the self-similarity matrix, an enhanced self-similarity matrix can be obtained through feature smoothing, path smoothing, and setting a threshold, thereby making the recognition results of the subsequent recognition model more accurate.

[0063] Step S530: Input the self-similarity matrix of all media resources into the recognition model to determine the probability that each frame in the target music is the chorus.

[0064] The recognition model can be trained using a pre-built neural network model. Optionally, the recognition model can be trained using a convolutional neural network (CNN). Since CNNs perform better in speech recognition compared to other deep learning structures, and chorus recognition falls under speech recognition, using a recognition model trained with a CNN can effectively improve the accuracy of both the recognition model and the chorus recognition.

[0065] In an exemplary embodiment, the recognition model can also be trained using a convolutional neural network and Self-Attention. Since music contains long-range interdependent features, and Self-Attention is better at capturing these features in sentences, training the recognition model using a convolutional neural network and Self-Attention can improve the accuracy of the recognition results. Optionally, the recognition model can also improve its generalization ability through Batch Normalization, preventing overfitting and accelerating model convergence.

[0066] Please refer to Figure 4 This is an example diagram of the network structure of the recognition model. In the convolutional neural network, the convolutional kernel is used to extract local features. Batch Norm (BN in the figure) can improve the generalization ability of the model, prevent overfitting, and accelerate the convergence of the model. After introducing Self Attention, it is easier to capture long-distance interdependent features in the sentence. The input of the model is the SSM matrix of human voice, drum sound, bass and overall audio. The output is the time series of chorus probability, which is a value between 0 and 1.

[0067] In one implementation, step S530 may involve inputting the self-similarity matrix of each media resource into different recognition models, determining the probability that each frame in each media resource is a chorus through different recognition models, and then finally determining the probability that each frame in the target music is a chorus based on the pre-set weights for each media resource. In another implementation, step S530 may involve inputting the self-similarity matrices of all media resources into one recognition model, and determining the probability that each frame in the target music is a chorus through the recognition model. In both implementations, since both methods incorporate the features of all media resources in determining the probability that each frame in the target music is a chorus, there will not be a significant difference in recognition accuracy. Generally speaking, in the first implementation, since the recognition model can focus on the self-similarity matrix of one media resource, the recognition accuracy of the model will be better than in the second implementation, but it is more complex to implement because it requires multiple recognition models. In addition, since the first implementation requires manually setting weights, and manually set weights can have a large error, the final recognition accuracy will not be significantly different between the two implementations.

[0068] Step S540: Determine the chorus segment of the target music based on probability.

[0069] In an exemplary embodiment, such as Figure 5 As shown, step S540 may also include steps S541 to S543, as detailed below:

[0070] Step S541: Determine whether the probability of each frame in the target music exceeds a preset threshold.

[0071] The preset threshold can be set according to the actual situation; there are no restrictions here.

[0072] Step S542: If the probability of the current frame in the target music is greater than or equal to a preset threshold, determine that the current frame belongs to the chorus.

[0073] For example, if the probability of the current frame in the target music is less than a preset threshold, the current frame is determined to belong to the main verse (verse part).

[0074] Step S543: Determine the chorus segment of the target music based on whether some or all frames in the target music belong to the chorus.

[0075] Since verses and choruses alternate in a piece of music, a chorus segment of the target music can be determined by whether a frame of a certain section belongs to the chorus, or all chorus segments of the target music can be determined by counting whether all frames belong to the chorus.

[0076] After obtaining the probability of each frame belonging to the chorus using a recognition model, the resulting probability curve will exhibit some jitter. By setting a preset threshold to determine whether each frame belongs to the chorus, a binarized result can be obtained, effectively enabling the identification of the chorus portion. In practical applications, a median filter can first be used to smooth the probability curve, and then a threshold (e.g., 0.5 times the curve amplitude) can be set to finally obtain the binarized result. A schematic diagram is shown below. Figure 6 As shown.

[0077] In this embodiment, by extracting the frequency domain features of each media resource, determining the self-similarity matrix of each media resource based on the frequency domain features, and inputting the self-similarity matrix into the recognition model, the recognition model determines the probability that each frame in the target music is the chorus, and determines the chorus segment of the target music based on the probability. This can effectively combine the features of each media resource to determine the chorus segment and improve the accuracy of chorus recognition.

[0078] To more clearly illustrate the music processing method of this application embodiment, a specific example is given below.

[0079] Please refer to Figure 7 The diagram illustrates a specific method for music processing, and its general flow is as follows:

[0080] 1. Separate the sound sources of the target music to obtain four media resources: vocals, drums, bass, and overall audio.

[0081] 2. Extract CQT features for each media resource;

[0082] 3. Calculate the self-similarity matrix of each media resource based on its CQT characteristics;

[0083] 4. Input the self-similarity matrix of all media resources into the recognition model to determine the probability that each frame in the target music is the chorus;

[0084] 5. Smooth and binarize the obtained probability curve to determine the chorus segment of the target music.

[0085] Example 2

[0086] Figure 8 The diagram schematically illustrates a music processing apparatus 600 according to Embodiment 2 of this application. This music processing apparatus 600 can be divided into one or more program modules. One or more program modules are stored in a storage medium and executed by one or more processors to complete the embodiments of this application. The program modules referred to in the embodiments of this application are a series of computer program instruction segments capable of performing specific functions. The following description will specifically introduce the functions of each program module in this embodiment.

[0087] like Figure 8 As shown, the music processing device 600 may include a separation module 610, an extraction module 620, and a determination module 630.

[0088] The separation module 610 is used to separate the sound source of the target music to obtain at least two media resources, at least one of the media resources corresponding to a music component;

[0089] Extraction module 620 is used to extract a first feature of each of the media resources;

[0090] The determining module 630 is used to determine the chorus segment of the target music based on the first feature.

[0091] In an exemplary embodiment, the separation module 610 is further configured to: perform sound source separation on the target music to obtain media resources corresponding to the overall target music and media resources corresponding to the vocal component, drum component and bass component respectively.

[0092] In an exemplary embodiment, the extraction module 620 is further configured to: extract frequency domain features of each of the media resources; the determination module 630 is further configured to: determine a self-similarity matrix of each of the media resources based on the frequency domain features; input the self-similarity matrix into a recognition model to determine the probability that each frame of the target music is a chorus through the recognition model; and determine the chorus segment of the target music based on the probability.

[0093] In an exemplary embodiment, the extraction module 620 is further configured to: extract the frequency domain features of each of the media resources in the Mel spectrum or CQT spectrum.

[0094] In an exemplary embodiment, the recognition model is trained by a convolutional neural network.

[0095] In an exemplary embodiment, the recognition model is trained using a convolutional neural network and Self Attention.

[0096] In an exemplary embodiment, the determining module 630 is further configured to: determine whether the probability of each frame in the target music exceeds a preset threshold; if the probability of the current frame in the target music is greater than or equal to the preset threshold, determine that the current frame belongs to the chorus; and determine the chorus segment of the target music based on the result of whether some or all frames in the target music belong to the chorus.

[0097] Example 3

[0098] Figure 9 The diagram schematically illustrates the hardware architecture of a computer device 700 suitable for a music processing method according to Embodiment 3 of this application. The computer device 700 can be a device capable of automatically performing numerical calculations and / or data processing according to pre-set or stored instructions. For example, it can be a rack server, blade server, tower server, or cabinet server (including standalone servers or server clusters composed of multiple servers), gateway, etc. Figure 9 As shown, the computer device 700 includes, but is not limited to, a memory 710, a processor 720, and a network interface 730 that can communicate with each other via a system bus. Wherein:

[0099] The memory 710 includes at least one type of computer-readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 710 may be an internal storage module of the computer device 700, such as the hard disk or memory of the computer device 700. In other embodiments, the memory 710 may also be an external storage device of the computer device 700, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. Of course, the memory 710 may include both the internal storage module and the external storage device of the computer device 700. In this embodiment, the memory 710 is typically used to store the operating system and various application software installed on the computer device 700, such as program code for music processing methods. In addition, the memory 710 can also be used to temporarily store various types of data that have been output or will be output.

[0100] In some embodiments, processor 720 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. Processor 720 is typically used to control the overall operation of computer device 700, such as performing control and processing related to data interaction or communication with computer device 700. In this embodiment, processor 720 is used to run program code stored in memory 710 or process data.

[0101] Network interface 730 may include a wireless network interface or a wired network interface, which is typically used to establish a communication link between computer device 700 and other computer devices. For example, network interface 730 is used to connect computer device 700 to an external terminal via a network, establishing a data transmission channel and communication link between computer device 700 and the external terminal. The network may be an intranet, the Internet, Global System for Mobile Communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi, or other wireless or wired networks.

[0102] It should be pointed out that, Figure 9 Only computer devices with components 710-730 are shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0103] In this embodiment, the music processing method stored in the memory 710 can be further divided into one or more program modules and executed by one or more processors (processor 720 in this embodiment) to complete the embodiment of this application.

[0104] Example 4

[0105] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the music processing method in the embodiments.

[0106] In this embodiment, the computer-readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the computer-readable storage medium can be an internal storage unit of a computer device, such as the hard disk or memory of the computer device. In other embodiments, the computer-readable storage medium can also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device. Of course, the computer-readable storage medium can also include both the internal storage unit and the external storage device of the computer device. In this embodiment, the computer-readable storage medium is typically used to store the operating system and various application software installed on the computer device, such as the program code of the music processing method in this embodiment. In addition, the computer-readable storage medium can also be used to temporarily store various types of data that have been output or will be output.

[0107] Obviously, those skilled in the art should understand that the modules or steps of the embodiments of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the embodiments of this application are not limited to any particular combination of hardware and software.

[0108] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A music processing method, characterized in that, include: The target music is separated into sound sources to obtain at least two media resources, at least one of which corresponds to a musical component; Extract the first feature of each of the media resources; The chorus segment of the target music is determined based on the first feature; The step of extracting a first feature from each of the media resources and determining the chorus segment of the target music based on the first feature includes: Extract the frequency domain features of each of the aforementioned media resources; The self-similarity matrix of each media resource is determined based on the frequency domain characteristics; The self-similarity matrix is ​​input into the recognition model to determine the probability that each frame in the target music is the chorus. The chorus segment of the target music is determined based on the probability stated therein.

2. The music processing method according to claim 1, characterized in that, The process of separating the target music into sound sources to obtain at least two media resources includes: The target music is separated into sound sources to obtain media resources corresponding to the overall target music and media resources corresponding to the vocal components, drum components, and bass components respectively.

3. The music processing method according to claim 1, characterized in that, The extraction of frequency domain features for each of the media resources includes: Extract the frequency domain features of each media resource in the Mel spectrum or CQT spectrum.

4. The music processing method according to claim 1, characterized in that, The recognition model is trained by a convolutional neural network.

5. The music processing method according to claim 1, characterized in that, The recognition model is trained using a convolutional neural network and Self Attention.

6. The music processing method according to any one of claims 3-5, characterized in that, The step of determining the chorus segment of the target music based on the probability includes: Determine whether the probability of each frame in the target music exceeds a preset threshold; If the probability of the current frame in the target music is greater than or equal to the preset threshold, the current frame is determined to belong to the chorus. Based on whether some or all frames in the target music belong to the chorus section, determine the chorus segment of the target music.

7. A music processing device, characterized in that, include: A separation module is used to separate the sound sources of the target music to obtain at least two media resources, at least one of the media resources corresponding to a musical component; The extraction module is used to extract the first feature of each of the media resources; A determining module is used to determine the chorus segment of the target music based on the first feature; The extraction module is also used for: Extract the frequency domain features of each of the aforementioned media resources; The determination module is also used for: The self-similarity matrix of each media resource is determined based on the frequency domain characteristics; The self-similarity matrix is ​​input into the recognition model to determine the probability that each frame in the target music is the chorus. The chorus segment of the target music is determined based on the probability stated therein.

8. A computer device, the computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the music processing method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that can be executed by at least one processor to cause the at least one processor to perform the steps of the music processing method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the music processing method according to any one of claims 1 to 6.

Images