Music processing method, computing device, and readable storage medium

Abstract

The application discloses a music processing method, a computing device and a readable storage medium. The method comprises the following steps: identifying a human voice end time point of a first music file and a human voice start time point of a second music file through a human voice recognition model, and automatically playing the second music file after the first music file; identifying a category of the first music file or the second music file; and inserting a predetermined audio corresponding to the category between the human voice end time point and the human voice start time point. The method solves the problem of abrupt connection between two pieces of music and improves user experience.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a music processing method, a computing device, and a readable storage medium. Background Technology

[0002] Currently, music players typically switch between songs directly after one song finishes playing. This method of switching music, especially when switching between different categories of music, such as different genres, results in abrupt transitions and a poor user experience.

[0003] Existing music switching methods use fixed playback intervals and mute music for a fixed period after each track finishes playing, creating a smooth transition between tracks and providing a break for the user's ears to relax. However, this method cannot eliminate the silent transitions between tracks, resulting in excessively long intervals after adding the mute. For some genres, such as upbeat and energetic music, adding the mute as a transition feels abrupt, leading to a poor user experience.

[0004] Therefore, the present invention provides a music processing solution to solve the problems existing in the prior art. Summary of the Invention

[0005] Therefore, the present invention provides a music processing method, a computing device, and a readable storage medium to solve or at least alleviate the problems mentioned above.

[0006] According to a first aspect of the present invention, a music processing method is provided, the method comprising: identifying, by means of a voice recognition model, a voice end time point of a first music file and a voice start time point of a second music file, wherein the second music file is automatically played after the first music file; identifying the category of the first music file or the second music file; and inserting a predetermined audio corresponding to the category between the voice end time point and the voice start time point.

[0007] Optionally, in the music processing method according to the present invention, the voice recognition model is obtained by: acquiring multiple third music files; preprocessing the multiple third music files to obtain multiple spectrograms corresponding to the multiple third music files; labeling the multiple spectrograms corresponding to the multiple third music files according to whether the spectrograms include human voices; and training based on the multiple spectrograms corresponding to the multiple third music files and their labels to obtain the voice recognition model.

[0008] Optionally, in the music processing method according to the present invention, identifying the end time point of the human voice in the first music file by means of a human voice recognition model includes: preprocessing the first music file to obtain a spectrogram corresponding to the first music file; identifying the spectrogram in the order of audio time from back to front based on the human voice recognition model; and taking the time point when the human voice is first identified as the end time point of the human voice.

[0009] Optionally, in the music processing method according to the present invention, identifying the start time point of the human voice in the second music file by means of a human voice recognition model includes: preprocessing the second music file to obtain a spectrogram corresponding to the second music file; identifying the spectrogram in the order of audio time from front to back based on the human voice recognition model; and taking the time point at which the human voice is first identified as the start time point of the human voice.

[0010] Optionally, in the music processing method according to the present invention, preprocessing the music file includes: converting the music file into a mono audio file; dividing the mono audio file of the music file into multiple audio segments according to the stage of the music; extracting feature data of each audio segment in the multiple audio segments corresponding to the music file, and generating multiple spectrograms corresponding to the music file based on the feature data; and the step of identifying the spectrograms includes: identifying the multiple spectrograms.

[0011] Optionally, in the music processing method according to the present invention, the step of identifying multiple spectrograms further includes: determining the type of audio based on multiple spectrograms corresponding to multiple audio segments of a music file, the type including instrumental music and remixes; if the detected spectrogram corresponds to an audio type of instrumental music, then the detection of that spectrogram is skipped.

[0012] Optionally, in the music processing method according to the present invention, the step of identifying the category of the first music file or the second music file includes: identifying the category of the first music file or the second music file through a category recognition model; the category recognition model is obtained by: acquiring multiple fourth music files including multiple categories of music; converting each of the multiple fourth music files into a mono audio file; extracting the audio data of the mono audio file of the fourth music file to generate a spectrogram of the fourth music file; labeling the spectrogram of the fourth music file according to the category of the fourth music file; and training based on the spectrograms and labels corresponding to the multiple fourth music files to obtain the category recognition model.

[0013] Optionally, in the music processing method according to the present invention, dividing the mono audio file of the music file into multiple audio segments includes: dividing the mono audio file of the music file into multiple audio segments including an introduction, narrative, climax, transition and / or coda.

[0014] Optionally, the music processing method according to the present invention further includes: clearing the audio and interval time between the end time point and the start time point of the human voice.

[0015] According to a second aspect of the present invention, a computing device is provided, comprising: at least one processor; and a memory storing program instructions, wherein the program instructions are configured to be executed by the at least one processor, the program instructions including instructions for performing the methods described above.

[0016] According to a third aspect of the invention, a readable storage medium storing program instructions is provided, which, when read and executed by a computing device, causes the computing device to perform the method described above.

[0017] According to the technical solution of the present invention, a voice recognition model is used to identify the end time of the voice in a first music file and the start time of the voice in a second music file. A category recognition model is used to identify the category of either the first or second music file. A predetermined audio clip corresponding to the category is inserted between the end and start times of the voice for a transition. Inserting interval music of the same category between two consecutive music files resolves the problem of abrupt transitions between the two pieces of music, thus improving the user experience.

[0018] Furthermore, by using deep learning techniques to train voice recognition and category recognition models, the accuracy and recognition efficiency are high, thus improving the efficiency of music processing.

[0019] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0020] To achieve the foregoing and related objectives, certain illustrative aspects are described herein in conjunction with the following description and accompanying drawings. These aspects indicate various ways in which the principles disclosed herein may be practiced, and all aspects and their equivalents are intended to fall within the scope of the claimed subject matter. The foregoing and other objectives, features, and advantages of this disclosure will become more apparent from the following detailed description, taken in conjunction with the accompanying drawings. Throughout this disclosure, the same reference numerals generally refer to the same parts or elements.

[0021] Figure 1 A block diagram of the physical components of the computing device 100 is shown;

[0022] Figure 2 A flowchart of a music processing method 200 according to an embodiment of the present invention is shown;

[0023] Figure 3 A schematic diagram illustrating a training method for a voice recognition model according to an embodiment of the present invention is shown;

[0024] Figure 4 A schematic diagram illustrating the identification of the end time point and the start time point of human voice according to an embodiment of the present invention is shown;

[0025] Figure 5 A schematic diagram illustrating a training method for a category recognition model according to an embodiment of the present invention is shown;

[0026] Figure 6 A schematic diagram illustrating the insertion of transitional music between the end point and the beginning point of human voice according to an embodiment of the present invention is shown.

[0027] Figure 7 A schematic flowchart of a music processing method 700 according to another embodiment of the present invention is shown. Detailed Implementation

[0028] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0029] Figure 1 A block diagram of the physical components (i.e., hardware) of a computing device 100 is shown. In a basic configuration, the computing device 100 includes at least one processing unit 102 and a system memory 104. According to one aspect, depending on the configuration and type of the computing device, the processing unit 102 may be implemented as a processor. The system memory 104 includes, but is not limited to, volatile memory (e.g., random access memory), non-volatile memory (e.g., read-only memory), flash memory, or any combination of such memory. According to one aspect, the system memory 104 includes an operating system 105 and a program module 106, the program module 106 including instructions 120 for performing the music processing method of the present invention.

[0030] According to one aspect, operating system 105 is, for example, suitable for controlling the operation of computing device 100. Furthermore, examples are practiced in conjunction with graphics libraries, other operating systems, or any other applications, and are not limited to any particular application or system. Figure 1The basic configuration is illustrated by the components within the dashed lines 108. According to one aspect, the computing device 100 has additional features or functions. For example, according to one aspect, the computing device 100 includes additional data storage devices (removable and / or non-removable), such as disks, optical discs, or magnetic tapes. This additional storage... Figure 1 The middle part is shown by removable storage device 109 and non-removable storage device 110.

[0031] As stated above, according to one aspect, a program module 106 is stored in system memory 104. According to one aspect, program module 106 may include one or more applications. The present invention does not limit the type of application; for example, applications may include: email and contact applications, word processing applications, spreadsheet applications, database applications, slideshow applications, drawing or computer-aided applications, web browser applications, etc.

[0032] According to one aspect, examples can be practiced on circuits including discrete electronic components, packaged or integrated electronic chips containing logic gates, circuits utilizing microprocessors, or on a single chip containing electronic components or a microprocessor. For example, it can be practiced via wherein... Figure 1 Each or many of the components shown can be implemented as an example by integrating a System-on-a-Chip (SOC) on a single integrated circuit. According to one aspect, such an SOC device may include one or more processing units, graphics units, communication units, system virtualization units, and various application functions, all integrated (or “burned in”) as a single integrated circuit onto a chip substrate. When operating via the SOC, the functions described herein can be operated via dedicated logic integrated on a single integrated circuit (chip) with other components of the computing device 100. Embodiments of the invention can also be implemented using other techniques capable of performing logical operations (e.g., AND, OR, and NOT), including but not limited to mechanical, optical, fluid, and quantum technologies. Additionally, embodiments of the invention can be implemented within a general-purpose computer or in any other circuit or system.

[0033] According to one aspect, computing device 100 may also have one or more input devices 112, such as a keyboard, mouse, pen, voice input device, touch input device, etc. It may also include output devices 114, such as a display, speaker, printer, etc. The foregoing devices are examples and other devices may also be used. Computing device 100 may include one or more communication connections 116 that allow communication with other computing devices 118. Examples of suitable communication connections 116 include, but are not limited to: RF transmitter, receiver and / or transceiver circuitry; Universal Serial Bus (USB), parallel and / or serial ports.

[0034] As used herein, the term computer-readable medium includes computer storage medium. Computer storage medium can include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information (e.g., computer-readable instructions, data structures, or program modules). System memory 104, removable storage device 109, and non-removable storage device 110 are examples of computer storage media (i.e., memory storage). Computer storage media can include random access memory (RAM), read-only memory (ROM), electrically erasable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape, magnetic tape, disk storage or other magnetic storage devices, or any other article of manufacture that can be used to store information and is accessible by computer device 100. According to one aspect, any such computer storage medium can be part of computing device 100. Computer storage media does not include carrier waves or other transmitted data signals.

[0035] According to one aspect, a communication medium is implemented by computer-readable instructions, data structures, program modules, or other data in a modulated data signal (e.g., a carrier wave or other transmission mechanism), and includes any information transmission medium. According to one aspect, the term "modulated data signal" describes a signal having one or more sets of characteristics or altered in a manner that encodes information in the signal. By way of example and not limitation, a communication medium includes wired media such as wired networks or direct wired connections, and wireless media such as acoustic, radio frequency (RF), infrared, and other wireless media.

[0036] In one embodiment of the present invention, a computing device 100 includes one or more processors and one or more readable storage media storing program instructions. When the program instructions are configured to be executed by one or more processors, the computing device causes to perform the music processing method of the present invention.

[0037] Figure 2 A flowchart of a music processing method 200 according to an embodiment of the present invention is shown. Method 200 can be executed in a computing device (e.g., the aforementioned computing device 100). Figure 2 As shown, method 200 begins at 210.

[0038] 210. Using a voice recognition model, identify the end time of the vocals in the first music file and the start time of the vocals in the second music file. The second music file plays automatically after the first music file. Optionally, the second music file plays immediately after the first music file, meaning the playback order is that the second music file plays immediately after the first music file finishes playing.

[0039] Specifically, the first music file is preprocessed to obtain its corresponding spectrogram. Based on a voice recognition model, the spectrogram is analyzed in reverse chronological order. The time point at which the first voice is detected is taken as the end time point of the voice.

[0040] According to one embodiment of the present invention, a first music file is converted into a mono audio file, feature data of the mono audio file is extracted, and a spectrogram corresponding to the first music file is generated based on the feature data. Then, the spectrogram corresponding to the first music file is input into a voice recognition model, and the spectrogram is recognized in reverse chronological order. The time point at which the first voice is recognized is taken as the end time point of the voice.

[0041] According to another embodiment of the present invention, firstly, a first music file is converted into a mono audio file. Then, based on the stage of the music, the mono audio file of the first music file is segmented into multiple audio segments. Optionally, the stage of the music may include an introduction, narrative, climax, transition, and / or coda, and the mono audio file of the first music file is segmented into multiple audio segments including an introduction, narrative, climax, transition, and / or coda. Optionally, the stage of the music may include an intro, interlude, and coda, and the mono audio file of the first music file is segmented into multiple audio segments including an intro, interlude, and coda. Finally, feature data of each audio segment in the multiple audio segments corresponding to the first music file is extracted, and multiple spectrograms corresponding to the first music file are generated based on the feature data. Specifically, one spectrogram is generated based on the feature data of each audio segment, and multiple spectrograms are generated based on the feature data of multiple audio segments. Optionally, Mel-frequency cepstral coefficients (MFCC) features of each audio segment in the multiple audio segments are extracted. Mel-frequency cepstral coefficients can be used for speech data feature extraction and reduction of computational dimensionality. The MFCC feature extraction process includes: pre-emphasis, framing, windowing, Fast Fourier Transform (FFT), Mel filter bank, and Discrete Cosine Transform (DCT). After obtaining multiple spectrograms corresponding to the first music file, these spectrograms are identified and input into the voice recognition model. The model identifies the multiple spectrograms corresponding to the first music file in reverse chronological order. The time point at which the voice is first detected is taken as the end time point of the voice. Based on the multiple spectrograms corresponding to the various audio segments of the first music file, the audio type is determined, including instrumental and remixes. If the detected spectrogram corresponds to instrumental music, the detection of that spectrogram is skipped. For example, the audio segments corresponding to the multiple spectrograms in the first music file might be in the time periods 00:00-00:20, 00:20-01:50, 01:50-03:20, 03:20-04:10, and 04:10-04:20, corresponding to the introduction, narrative, climax, transition, and coda stages, respectively. Following the audio time sequence from back to front, the spectrogram of the ending phase is first input into the voice recognition model to determine the audio type. If the recognition result is that the spectrogram of the ending phase does not include human voices (i.e., instrumental music), the detection of that spectrogram is skipped. Then, the spectrogram of the transition phase is input into the voice recognition model, and so on. If the recognition result is that the spectrogram of the transition phase includes human voices (i.e., mixed audio), then the time point when the first human voice appears in the time period of 03:20-04:10 is identified as the end time point of the human voice in the first music file, following the sequence from back to front.

[0042] Optionally, if multiple spectrograms corresponding to the first music file are identified in reverse chronological order according to the audio time sequence corresponding to the spectrograms, and none of the identification results include human voices, then the end position of the audio in the first music file is taken as the end time point of the human voice. In step 210, the human voice recognition model also identifies the start time point of the human voice in the second music file. Specifically, the second music file is preprocessed to obtain its corresponding spectrogram. Based on the human voice recognition model, the spectrograms are identified in reverse chronological order according to the audio time. The time point at which the human voice is first identified is taken as the start time point of the human voice.

[0043] According to one embodiment of the present invention, a second music file is converted into a mono audio file, feature data of the mono audio file is extracted, and a spectrogram corresponding to the second music file is generated based on the feature data. Then, the spectrogram corresponding to the second music file is input into a voice recognition model, and the spectrogram is recognized in chronological order of audio time. The time point at which the first voice is recognized is taken as the voice start time point.

[0044] According to another embodiment of the present invention, firstly, the second music file is converted into a mono audio file. Then, based on the stage of the music, the mono audio file of the second music file is segmented into multiple audio segments. Optionally, the stage of the music may include an introduction, narrative, climax, transition, and / or coda, and the mono audio file of the second music file is segmented into multiple audio segments including an introduction, narrative, climax, transition, and / or coda. Optionally, the stage of the music may include an intro, interlude, and coda, and the mono audio file of the second music file is segmented into multiple audio segments including an intro, interlude, and coda. Finally, feature data of each audio segment in the multiple audio segments corresponding to the second music file is extracted, and multiple spectrograms corresponding to the second music file are generated based on the feature data. Specifically, one spectrogram is generated based on the feature data of each audio segment, and multiple spectrograms are generated based on the feature data of multiple audio segments. Optionally, Mel-frequency cepstral coefficients (MFCC) features of each audio segment in the multiple audio segments are extracted. Mel-frequency cepstral coefficients can be used for speech data feature extraction and reduction of computational dimensionality. The MFCC feature extraction process includes: pre-emphasis, framing, windowing, Fast Fourier Transform (FFT), Mel filter bank, and Discrete Cosine Transform (DCT). After obtaining multiple spectrograms corresponding to the second music file, these spectrograms are identified and input into the voice recognition model. The model identifies the multiple spectrograms corresponding to the second music file in chronological order of audio time. The time point at which the voice is first detected is taken as the start time point. Based on the multiple spectrograms corresponding to the various audio segments of the second music file, the audio type is determined, including instrumental and remixes. If the detected spectrogram corresponds to instrumental music, the detection of that spectrogram is skipped. For example, the audio segments corresponding to the multiple spectrograms in the second music file might be in the time periods 00:00-00:20, 00:20-01:50, 01:50-03:20, 03:20-04:10, and 04:10-04:20, corresponding to the introduction, narrative, climax, transition, and coda, respectively. Following the audio time sequence from front to back, the spectrogram of the intro phase is first input into the voice recognition model to determine the audio type. If the recognition result is that the spectrogram of the intro phase does not include human voices (i.e., instrumental music), the detection of that spectrogram is skipped. Then, the spectrogram of the narrative phase is input into the voice recognition model, and so on. If the recognition result is that the spectrogram of the narrative phase includes human voices (i.e., mixed audio), then the time point when the first human voice appears in the time period of 00:20-01:50 is identified as the start and end point of the human voice in the second music file, following the sequence from front to back.

[0045] Optionally, if multiple spectrograms corresponding to the first music file are identified in the order of audio time corresponding to the spectrograms from front to back, and none of the identification results include human voices, then the beginning position of the audio in the second music file is taken as the end point of the human voice. Figure 4 A schematic diagram illustrating the identification of the end time point and the start time point of a human voice according to an embodiment of the present invention is shown. Figure 4 As shown, the end time of the vocals in the first music file is identified in reverse order, while the start time of the vocals in the second music file is identified in reverse order.

[0046] According to an embodiment of the present invention, Figure 3 A schematic diagram illustrating the training method of a voice recognition model according to an embodiment of the present invention is shown. Figure 3 As shown in section 310, obtain multiple third-party music files. Specifically, collect music files of various categories (i.e., third-party music files). These categories can be categorized using any method, such as multiple styles or genres, but are not limited to these.

[0047] 320. Preprocess multiple third-party music files to obtain multiple spectrograms corresponding to the multiple third-party music files.

[0048] According to an embodiment of the present invention, firstly, multiple third-channel music files are converted into mono audio files. Then, based on the stage of the music, the mono audio files of the third-channel music files are segmented into multiple audio segments. Optionally, the stage of the music may include an introduction, narrative, climax, transition, and / or coda, and the mono audio files of the third-channel music files are segmented into multiple audio segments including an introduction, narrative, climax, transition, and / or coda. Optionally, the stage of the music may include an intro, interlude, and coda, and the mono audio files of the third-channel music files are segmented into three audio segments including an intro, interlude, and coda. Finally, feature data of each audio segment in the multiple audio segments corresponding to the third-channel music file is extracted, and multiple spectrograms corresponding to the third-channel music file are generated based on the feature data. Specifically, one spectrogram is generated based on the feature data of each audio segment, and multiple spectrograms are generated based on the feature data of multiple audio segments. Optionally, Mel-frequency cepstral coefficients (MFCC) features of each audio segment in the multiple audio segments are extracted. Mel-frequency cepstral coefficients can be used for speech data feature extraction and reduction of computational dimensionality. The MFCC feature extraction process includes: pre-emphasis, framing, windowing, Fast Fourier Transform (FFT), Mel filter bank, and Discrete Cosine Transform (DCT).

[0049] 330. Label multiple spectrograms corresponding to multiple third-party music files based on whether or not they contain vocals. Specifically, if the audio corresponding to a spectrogram contains vocals, such as in a mix (i.e., a combination of music and vocals), then the spectrogram is labeled as containing vocals. If the audio corresponding to a spectrogram does not contain vocals, such as in instrumental music, then the spectrogram is labeled as not containing vocals.

[0050] 340. A voice recognition model is obtained by training multiple spectrograms and their labels corresponding to multiple third-party music files. Optionally, the voice recognition model is constructed using a Residual Network (ResNet) network model. ResNet is a convolutional neural network model that is easy to optimize and can improve accuracy by increasing network depth. The residual blocks inside the network use skip connections, which can alleviate the gradient vanishing problem caused by increasing the depth in deep neural networks.

[0051] After the voice recognition model is trained, it can be used to identify whether any music file contains human voices. Furthermore, it can determine the start and end times of human voices by identifying whether they are present at a specific point in time, including whether a first or second music file contains human voices, and the start and end times of the human voices. Optionally, by dividing the spectrogram to be recognized into multiple parts, such as dividing it into multiple parts per second, and inputting each part into the voice recognition model, it can be determined whether the spectrogram for that unit of time contains human voices.

[0052] 220. Identify the category of the first or second music file.

[0053] According to an embodiment of the present invention, the category of a first music file or a second music file is identified by a category recognition model. Figure 5 A schematic diagram illustrating the training method of a category recognition model according to an embodiment of the present invention is shown. Figure 5 As shown in Figure 510, obtain multiple fourth music files including various categories of music. Specifically, obtain multiple fourth music files, including various categories of music, such as music of different genres and styles, that is, collect music files of various genres.

[0054] 520. Convert each of the multiple fourth music files into a mono audio file.

[0055] 530. Extract the audio data from the mono audio file of the fourth music file and generate the spectrogram of the fourth music file.

[0056] According to an embodiment of the present invention, the audio data of the mono audio file of the fourth music file is extracted using Fast Fourier Transform (FFT) to generate the spectrogram of the fourth music file. Optionally, the spectrogram of the fourth music file is generated using Discrete Fourier Transform (DFT) or Mel-frequency cepstral coefficients (MFCC).

[0057] 540. Label the spectrogram of the fourth music file according to its category.

[0058] According to an embodiment of the present invention, the spectrum of the fourth music file is labeled according to the music genre or style or other classification attributes to which the fourth music file belongs. For example, if a fourth music file belongs to the rock music genre, it is labeled as pop; if another fourth music file belongs to the blues music genre, it is labeled as blues.

[0059] 550. A category recognition model is obtained by training based on the spectrograms and labels corresponding to multiple fourth music files. Optionally, a human voice recognition model is constructed using a Residual Network (ResNet) network model. ResNet is a convolutional neural network model that is easy to optimize and can improve accuracy by increasing network depth. The residual blocks inside the network use skip connections, which can alleviate the gradient vanishing problem caused by increasing the depth in deep neural networks.

[0060] Once the category recognition model is trained, it can be used to identify the category of any music file, including the category of the first or second music file.

[0061] 230. Insert a pre-defined audio clip corresponding to the category between the end time point and the start time point of the human voice to create a transition.

[0062] Specifically, each music category has a pre-set audio track as a transitional track. For example, blues music has a corresponding transitional track within the same category, rock music has a corresponding transitional track within the same category, and so on. Besides selecting music from the same category as the transition, the pre-set audio track can also be other suitable transitional music corresponding to the identified category, or it can be blank audio that serves as a mute effect.

[0063] According to one embodiment of the present invention, before inserting predetermined audio for transition, the audio and interval time between the end time of the vocals of the first music file and the start time of the vocals of the second music file are cleared. Figure 6 A schematic diagram illustrating the insertion of transitional music between the end and beginning times of vocals, according to an embodiment of the present invention, is shown. Figure 6As shown, 610 represents the end time of the vocals in the first music file, and 620 represents the start time of the vocals in the second music file. After clearing the audio and interval time between the end time of the vocals in the first music file and the start time of the vocals in the second music file, the predetermined audio corresponding to the category identified in 220 is inserted between the end time of the vocals in the first music file and the start time of the vocals in the second music file to achieve a natural music switching method, improve the user's listening experience, and form a better music transition method.

[0064] According to another embodiment of the present invention, the audio and interval time between the end time of the vocals in the first music file and the start time of the vocals in the second music file are not cleared, but the predetermined audio corresponding to the category identified in 220 is directly inserted for mixed playback. This can preserve the original music content and also produce a good transition between the two music files.

[0065] To more clearly illustrate the music processing method of the present invention, the following description, in conjunction with specific flowcharts, explains the music processing method of the present invention. Figure 7 A flowchart illustrating a music processing method 700 according to another embodiment of the present invention is shown. Figure 7 As shown, method 700 begins with 701.

[0066] 701. Play the first music file.

[0067] 702. Determine whether to use interval playback, i.e., whether to switch between two music files with intervals. If yes, proceed to 703; otherwise, proceed to 712 and play the music in the normal or default mode.

[0068] 703. Using a voice recognition model, identify the end time of the vocals in the first music file.

[0069] 704. Determine the method of interval playback. If the interval playback is performed in a silent manner, proceed to 705. If the transition is performed using a predetermined audio of the same category as the first music file, proceed to 706. Determine if a category recognition model exists.

[0070] 705. Use a blank audio clip of a certain duration as transitional music. Then, proceed to 709.

[0071] 706. Determine if a category recognition model exists. If a category recognition model exists, proceed to 707; otherwise, proceed to 708.

[0072] 707. Using a category recognition model, identify the category of the first music file and use the predefined audio corresponding to the identified category as transition music. Then, proceed to 709.

[0073] 708. Train the category recognition model. After the category recognition model is trained, proceed to 707. Use the category recognition model to identify the category of the first music file and use the predetermined audio corresponding to the identified category as the transition music.

[0074] 709. Play transitional music.

[0075] 710. Using a voice recognition model, identify the start time of the voice in the next music file.

[0076] 711. Play the content of the second music file starting from the point when the vocals begin in the second music file.

[0077] It should be noted that the specific implementation methods in Method 700, such as the training schemes for the voice recognition model and the category recognition model, are described in Method 200 and will not be repeated here.

[0078] According to the technical solution of the present invention, a voice recognition model is used to identify the end time of the voice in a first music file and the start time of the voice in a second music file. A category recognition model is used to identify the category of either the first or second music file. A predetermined audio clip corresponding to the category is inserted between the end and start times of the voice for a transition. Inserting interval music of the same category between two consecutive music files resolves the problem of abrupt transitions between the two pieces of music, thus improving the user experience.

[0079] Furthermore, by using deep learning techniques to train voice recognition and category recognition models, the accuracy and recognition efficiency are high, thus improving the efficiency of music processing.

[0080] The various techniques described herein can be implemented in combination with hardware or software, or a combination thereof. Thus, the methods and apparatus of the present invention, or certain aspects or portions thereof, can take the form of program code (i.e., instructions) embedded in a tangible medium, such as a removable hard disk, USB flash drive, floppy disk, CD-ROM, or any other machine-readable storage medium, wherein when the program is loaded into and executed by a machine such as a computer, the machine becomes an apparatus for practicing the present invention.

[0081] When the program code is executed on a programmable computer, the mobile terminal generally includes a processor, a processor-readable storage medium (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. The memory is configured to store program code; the processor is configured to execute the music processing method of the present invention according to instructions in the program code stored in the memory.

[0082] By way of example, and not limitation, readable media include readable storage media and communication media. Readable storage media stores information such as computer-readable instructions, data structures, program modules, or other data. Communication media generally embodies computer-readable instructions, data structures, program modules, or other data in the form of modulated data signals such as carrier waves or other transmission mechanisms, and includes any information delivery medium. Any combination of the above is also included within the scope of readable media.

[0083] In the specification provided herein, the algorithms and displays are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used with the examples of this invention. The required structure for constructing such systems is apparent from the above description. Furthermore, this invention is not directed to any particular programming language. It should be understood that the contents of the invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of the invention.

[0084] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0085] Similarly, it should be understood that, in order to streamline this disclosure and aid in understanding one or more of the various aspects of the invention, in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof. However, this method of disclosure should not be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into this detailed description, wherein each claim itself is a separate embodiment of the invention.

[0086] Those skilled in the art will understand that modules, units, or components of the devices disclosed in the examples herein can be arranged in the devices described in this embodiment, or alternatively, can be located in one or more devices different from the devices in this example. The modules in the foregoing examples can be combined into a single module or, in addition, can be divided into multiple sub-modules.

[0087] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0088] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0089] Furthermore, some of the embodiments described herein are methods or combinations of method elements that can be implemented by a processor of a computer system or by other means of performing the functions. Therefore, a processor having the necessary instructions for implementing the methods or method elements forms means for implementing the methods or method elements. Furthermore, the elements described herein in the apparatus embodiments are examples of means for implementing the functions performed by elements for the purposes of carrying out the invention.

[0090] As used herein, unless otherwise specified, the use of ordinal numbers such as “first,” “second,” “third,” etc., to describe ordinary objects merely indicates different instances of similar objects and is not intended to imply that the objects being described must have a given order in time, space, ordering, or any other manner.

[0091] Although the invention has been described with respect to a limited number of embodiments, those skilled in the art will understand from the foregoing description that other embodiments are conceivable within the scope of the invention described herein. Furthermore, it should be noted that the language used in this specification has been chosen primarily for readability and edibility purposes, and not for the purpose of interpreting or limiting the subject matter of the invention. Therefore, many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the appended claims. The disclosure of the invention is illustrative rather than restrictive, and the scope of the invention is defined by the appended claims.

Claims

1. A music processing method, the method comprising: Using a voice recognition model, the end time of the voice in the first music file and the start time of the voice in the second music file are identified, and the second music file is automatically played after the first music file. Identify the category of the first music file or the second music file; Insert a predetermined audio corresponding to the category between the end time point of the human voice and the start time point of the human voice; The process of identifying the end time of the human voice in a first music file using a human voice recognition model includes: preprocessing the first music file to obtain a spectrogram corresponding to the first music file; identifying the spectrogram based on the human voice recognition model in the order of audio time from back to front; and taking the time point when the human voice is first identified as the end time of the human voice. The process of identifying the start time of human voices in a second music file using a voice recognition model includes: preprocessing the second music file to obtain a spectrogram corresponding to the second music file; identifying the spectrogram based on the voice recognition model in chronological order of audio time; and taking the time point at which human voices are first identified as the start time point of human voices.

2. The method according to claim 1, wherein, The human voice recognition model is obtained in the following way: Obtain multiple third-party music files; Preprocessing the multiple third music files yields multiple spectrograms corresponding to the multiple third music files; Based on whether the spectrogram includes human voices, the spectrograms corresponding to the multiple third music files are labeled. A human voice recognition model is obtained by training based on multiple spectrograms and their labels corresponding to the multiple third music files.

3. The method according to claim 1 or 2, wherein, Preprocessing of music files includes: Convert music files to mono audio files; Based on the stage of the music, the mono audio file of the music file is divided into multiple audio segments; Extract feature data from each audio segment of a music file and generate multiple spectrograms corresponding to the music file based on the feature data; The steps for identifying the spectrogram include: The multiple spectrograms are identified.

4. The method according to claim 3, wherein, The step of identifying the plurality of spectrograms further includes: Based on multiple spectrograms corresponding to multiple audio segments of the music file, the type of the audio is determined, including instrumental music and remixes; If the detected spectrogram corresponds to an instrumental music audio, then the detection of that spectrogram is skipped.

5. The method according to claim 1, wherein, The steps for identifying the category of the first music file or the second music file include: The category of the first music file or the second music file is identified using a category recognition model. The category recognition model is obtained in the following way: Obtain multiple fourth-generation music files, including various music genres; Convert each of the plurality of fourth music files into a mono audio file; Extract the audio data of the mono audio file of the fourth music file and generate the spectrogram of the fourth music file; According to the category of the fourth music file, the spectrogram of the fourth music file is labeled; A category recognition model is obtained by training based on the spectrograms and labels corresponding to the multiple fourth music files.

6. The method according to claim 1, further comprising: Clear the audio and interval time between the end time of the human voice and the start time of the human voice.

7. A computing device, comprising: At least one processor; as well as A memory storing program instructions, wherein the program instructions are configured to be executed by the at least one processor, the program instructions including instructions for performing the method as claimed in any one of claims 1 to 6.

8. A readable storage medium storing program instructions that, when read and executed by a computing device, cause the computing device to perform the method as described in any one of claims 1 to 6.

Images