Deep learning-based sound isolation method, device and storage medium

Abstract

The application discloses a sound isolation method and device based on deep learning and a storage medium. The method comprises the following steps: obtaining an audio file used for constructing a DeepAudioSep model and preprocessing the audio file used for constructing the DeepAudioSep model; constructing the DeepAudioSep model and training the DeepAudioSep model, wherein the DeepAudioSep model comprises one mixed source input and ten isolated source outputs; and performing sound separation through the DeepAudioSep model. The application introduces the data driving and deep learning idea into sound separation and noise isolation processing, improves the sound separation and noise isolation processing capability in the environmental monitoring field, and therefore has wide noise processing prospects and practical value.

Description

Technical Field

[0001] This application relates to the field of sound isolation, and in particular to sound isolation methods, devices and storage media based on deep learning. Background Technology

[0002] Currently, noise separation and isolation typically employ complex algorithm models for training. However, in practical applications, due to the complexity of sound, models trained with limited sample data often fail to achieve ideal results. Furthermore, conventional noise separation is often specific, but in actual environmental monitoring, noise is often defined based on environmental conditions. When assessing the environment, it often means deducting specific types of noise, such as horns. There is a lack of flexible means to separate mixed sounds into different sounds and deduct certain types of sounds deemed noise as needed.

[0003] Therefore, the aforementioned technical problems in the relevant technologies urgently need to be solved. Summary of the Invention

[0004] This application aims to solve one of the technical problems in related technologies. To this end, embodiments of this application provide a sound isolation method, apparatus, and storage medium based on deep learning, which can improve the accuracy, reliability, and stability of sound separation.

[0005] According to one aspect of an embodiment of this application, a deep learning-based sound isolation method is provided, the method comprising:

[0006] Obtain the audio file used to build the DeepAudioSep model and preprocess the audio file used to build the DeepAudioSep model;

[0007] Construct and train the DeepAudioSep model, which includes a mixed source input and ten isolated source outputs, all of which have undergone feature extraction and normalization.

[0008] Sound separation is performed using the DeepAudioSep model.

[0009] In one embodiment, an audio file for constructing the DeepAudioSep model is obtained, including:

[0010] The first audio file is obtained by synthesizing several categories of sounds, and the several categories of sounds include at least the sounds of wind, rain, thunder, insects, frogs, birds, dogs, and human voices.

[0011] Several randomly selected additional audio files are superimposed onto the first audio file to obtain the audio file used to construct the DeepAudioSep model.

[0012] In one embodiment, the audio file used to construct the DeepAudioSep model is preprocessed, including:

[0013] The audio file used to build the DeepAudioSep model is processed into a 5-second audio segment, with pitch shifted by 2 or 4 steps, stretching by 1.2 times, and loudness shifted between -5 and +5 points.

[0014] Encoded in WAV format using a PCM codec, mono with a sampling frequency of 44100 Hz, the audio file duration was reduced to 3 seconds by converting the input and output audio files to 22050 Hz, and normalization was performed using the minimum MAX parameter.

[0015] In one embodiment, constructing and training the DeepAudioSep model includes:

[0016] Several classes are extracted from the output of the DeepAudioSep model to obtain filters;

[0017] Each filter is multiplied by the input of the DeepAudioSep model to obtain a predicted isolated output filter, which is used as input to obtain ten computed outputs. The loss target output for each class is calculated by measuring the mean square error between the computed outputs and the provided error.

[0018] The loss target outputs for each class are aggregated, and the DeepAudioSep model is optimized using the ADAM optimization algorithm.

[0019] In one embodiment, the method further includes: validating and optimizing the DeepAudioSep model, including:

[0020] Increase and optimize training data;

[0021] Optimize the model network of the DeepAudioSep model;

[0022] Adjust network parameters.

[0023] In one embodiment, the network parameters include: the size of the hidden unit, the input size, the loss optimization, the number of training iterations, the type of normalization technique, and the dataset size.

[0024] In one embodiment, sound separation is performed using the DeepAudioSep model, including:

[0025] Input a mixed audio file;

[0026] The mixed sound file is output to different categories of sound files by calling the model;

[0027] The results are used to evaluate the preservation of sound separation;

[0028] The evaluation of the sound separation retention results is based on the signal distortion ratio, source-to-interference ratio, and source-to-interference ratio, with the highest ratio being taken as the result of sound separation and isolation.

[0029] According to one aspect of an embodiment of this application, a deep learning-based sound isolation device is provided, the device comprising:

[0030] The first module is used to obtain the audio file used to build the DeepAudioSep model and to preprocess the audio file used to build the DeepAudioSep model.

[0031] The second module is used to construct and train the DeepAudioSep model, which includes a mixed source input and ten isolated source outputs, all of which have undergone feature extraction and normalization.

[0032] The third module is used for sound separation using the DeepAudioSep model.

[0033] According to one aspect of an embodiment of this application, a deep learning-based sound isolation device is provided, the device comprising:

[0034] At least one processor;

[0035] At least one memory for storing at least one program;

[0036] When at least one of the programs is executed by at least one of the processors, the deep learning-based sound isolation method as described in the preceding embodiments is implemented.

[0037] According to one aspect of the embodiments of this application, a storage medium is provided, the storage medium storing a processor-executable program, which, when executed by a processor, implements the deep learning-based sound isolation method as described in the preceding embodiments.

[0038] The beneficial effects of the deep learning-based sound isolation method, apparatus, and storage medium provided in this application are as follows: The method includes obtaining an audio file for constructing a DeepAudioSep model and preprocessing the audio file; constructing and training the DeepAudioSep model, wherein the DeepAudioSep model includes a mixed source input and ten isolated source outputs; and performing sound separation through the DeepAudioSep model. This application introduces data-driven and deep learning concepts into sound separation and noise isolation processing, improving the sound separation and noise isolation processing capabilities in the field of environmental monitoring, and therefore has broad prospects and practical value in noise processing.

[0039] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 A flowchart illustrating a deep learning-based sound isolation method provided in this application embodiment;

[0042] Figure 2 A flowchart illustrating the model training process for a deep learning-based sound isolation method provided in this application embodiment;

[0043] Figure 3 A flowchart of the sound separation algorithm provided in the embodiments of this application;

[0044] Figure 4 This is a schematic diagram of the DeepAudioSep model structure provided in the embodiments of this application;

[0045] Figure 5 A schematic diagram of a deep learning-based sound isolation device provided in an embodiment of this application;

[0046] Figure 6 This is a schematic diagram of another deep learning-based sound isolation device provided in an embodiment of this application. Detailed Implementation

[0047] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present application.

[0048] The terms "first," "second," "third," and "fourth," etc., used in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0049] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0050] Currently, noise separation and isolation typically employ complex algorithm models for training. However, in practical applications, due to the complexity of sound, models trained with limited sample data often fail to achieve ideal results. Furthermore, conventional noise separation is often specific, but in actual environmental monitoring, noise is often defined based on environmental conditions. When assessing the environment, it often means deducting specific types of noise, such as horns. There is a lack of flexible means to separate mixed sounds into different sounds and deduct certain types of sounds deemed noise as needed.

[0051] To address the aforementioned issues, this application proposes a sound isolation method, apparatus, and storage medium based on deep learning. This application primarily introduces the concept of a deep autoencoder, which learns the latent frequency and temporal features of each sound class through self-supervised training, and then recovers the features of each sound class through a deep decoder. Existing technologies often suffer from complex models and poor adaptability, making sound separation in different scenarios difficult. This application extracts speech embedding vectors using a deep learning autoencoder, classifies and trains a deep network on various sound vectors, and then recovers the sound spectrum through a decoder. The model parameters are optimized through evaluation, ensuring the accuracy, reliability, and stability of the resulting model's sound separation.

[0052] Terminology Explanation:

[0053] Deep learning: Deep learning is a type of machine learning, and machine learning is an essential path to achieving artificial intelligence. The concept of deep learning originated from research on artificial neural networks; a multilayer perceptron with multiple hidden layers is a type of deep learning structure. Deep learning discovers distributed feature representations of data by combining low-level features to form more abstract high-level representations of attribute categories or features. The motivation for researching deep learning is to build neural networks that simulate the human brain's analytical learning process. It mimics the mechanisms of the human brain to interpret data, such as images, sounds, and text.

[0054] Sound processing: Sound processing is an important research direction in computer science and artificial intelligence. It studies how to use computers to process various sounds to achieve effective communication between humans and computers. Sound processing is mainly applied to sound classification, noise detection, and speech recognition.

[0055] CNN: Convolutional Neural Networks (CNNs or ConvNets) are a class of deep neural networks most commonly used for analyzing visual images. CNNs use a variant of the multilayer perceptron design and require minimal preprocessing. They are also known as shift-invariant or spatially invariant artificial neural networks (SIANNs) based on their shared weight architecture and translation-invariant features.

[0056] Autoencoders (AEs) are a type of artificial neural network (ANN) used in semi-supervised and unsupervised learning. Their function is to learn representations of the input information by using the input information as the learning target. An autoencoder consists of an encoder and a decoder. Based on learning paradigm, autoencoders can be divided into contractive autoencoders, regularized autoencoders, and variational autoencoders (VAEs), where the former two are discriminative models and the latter is a generative model. Based on architecture, autoencoders can be feedforward or recursive neural networks.

[0057] Sound separation: Sound separation is a technology that extracts individual sounds from a mixed sound source. It has been successfully applied, for example, in restoring classic films, eliminating noise on smartphones, and enabling real-time karaoke functionality in music streaming services.

[0058] Noise reduction: A technique for eliminating noise from sound.

[0059] DeepAudioSep: The name of the deep learning model for sound separation constructed using the technology described in this application.

[0060] SDR: Signal to Distortion Ratios.

[0061] SIR: Source to Interference Ratio

[0062] SAR: Sources to Artifacts Ratios

[0063] Figure 1 A flowchart of a deep learning-based sound isolation method provided in this application embodiment is shown below. Figure 1 As shown, a deep learning-based sound isolation method includes:

[0064] S101. Obtain the audio file used to construct the DeepAudioSep model and preprocess the audio file used to construct the DeepAudioSep model.

[0065] In step S101, an audio file for constructing the DeepAudioSep model is obtained, specifically including: synthesizing several categories of sounds to obtain a first audio file, wherein the several categories of sounds include at least wind sounds, rain sounds, thunder sounds, insect sounds, frog sounds, bird sounds, dog sounds, and human voices; and superimposing several randomly selected additional audio files onto the first audio file to obtain the audio file for constructing the DeepAudioSep model.

[0066] Furthermore, the preprocessing of the audio file used to construct the DeepAudioSep model in step S101 includes: processing the audio file used to construct the DeepAudioSep model into a 5-second audio segment, shifting the pitch by 2 or 4 steps, stretching by 1.2 times, and shifting the loudness value between -5 and +5; encoding in WAV format using a PCM codec, mono with a sampling frequency of 44100 Hz, reducing the duration of the audio file to 3 seconds by converting the input and output audio files to 22050 Hz, and normalizing using the minimum MAX parameter.

[0067] The audio files for the training and development sets were created by synthesizing nine sounds corresponding to the categories (1) wind, (2) rain, (3) thunder, (4) insects, (5) frogs, (6) birds, (7) dogs, (8) human voices, and (9) mechanical sounds, and then superimposing them to create the input sounds. This application then superimposes two randomly selected additional audio files onto the input audio files for (10) other classes. All individual audio files are classified into the corresponding output class. As part of the dataset generation, each audio file was processed into a 5-second audio segment with pitch randomly shifted by 2 or 4 steps, randomly stretched by 1.2 times, and loudness shifted by randomly selected values ​​between -5 and +5. This ensures that the model will be trained and evaluated on a robust dataset. The generated audio file is 5 seconds long, encoded in WAV format using a PCM codec, mono, and sampled at 44100 Hz. The duration of the audio file is shortened to 3 seconds by converting the input and output audio files to 22050 Hz, normalizing using the minimum MAX parameter, and extracting the STFT from the input and output files. When converting the audio, the STFT window size used is 23 milliseconds, and the jump point length is the window size. This application uses 50,000 files as the dataset; 92% of the data is used for training, and the remaining 8% is used for testing and validation.

[0068] S102. Construct and train the DeepAudioSep model, which includes a mixed source input and ten isolated source outputs, wherein all source outputs have undergone feature extraction and normalization.

[0069] Optionally, in S102, constructing and training the DeepAudioSep model includes: extracting several classes from the output of the DeepAudioSep model to obtain filters; multiplying each filter by the input of the DeepAudioSep model to obtain a predicted isolated output filter, which is used as input to obtain ten computational outputs; calculating the target loss output for each class by measuring the mean square error between the calculated error and the provided error; summing the target loss outputs for each class; and optimizing the DeepAudioSep model using the ADAM optimization algorithm.

[0070] Figure 4 This is a schematic diagram of the DeepAudioSep model structure provided in the embodiments of this application, as shown below. Figure 4 As shown, the layers in the DeepAudioSep model's autoencoding stage correspond to: (1) a vertical convolutional layer to obtain frequency features, (2) a horizontal convolutional layer to obtain time-related features, thus outputting time-frequency codes, and (3) a fully connected layer with a Corrected Linear Unit (ReLU) as activation, which shares the output with the layers in the decoding stage. In each decoding stage, for every four classes, the layers correspond to: (1) fully connected layers sharing the same dimension as the minimum information loss ReLU activation function for the encoding horizontal convolutional layer, and (2) horizontal and (3) vertical deconvolutional layers sharing the opposite dimensions of the horizontal and vertical convolutional layers in the encoding stage. The outputs of the ten decoding stages are then concatenated to apply a ReLU, which is then used for loss calculation and backpropagation.

[0071] The DeepAudioSep model of this application provides a mixed source input and ten isolated source outputs, all of which undergo feature extraction and normalization, and attempts to train the size of the input STFT while ignoring its phase. Instead of training the size and phase of the input STFT, the model discovers and trains only on the size and phase of the input, with each output providing a small difference in the source-to-perturbation ratio, thus reducing the number of training parameters compared to training on the size and phase of the input STFT. The model then attempts to learn a set of ten different filters applied to the input to obtain ten outputs representing isolated sources, calculated as follows:

[0072] (1) Extract these ten classes from the output of the connected model, divide each extracted output by the sum, and obtain the filter.

[0073] (2) Multiply each filter by the input to obtain the predicted isolated output. The filter is applied to the input to obtain ten computed outputs. The loss target output for each class is calculated by measuring the mean square error (MSE) between the computed output and the provided error.

[0074] (3) The losses are aggregated and the model is optimized using the ADAM optimization algorithm.

[0075] S103. Perform sound separation using the DeepAudioSep model.

[0076] Specifically, the sound separation in step S103 using the DeepAudioSep model includes: inputting a mixed sound file; outputting the mixed sound file to different types of sound files by calling the model; and evaluating the retention result of the sound separation based on the results. The evaluation of the retention result of the sound separation is based on the signal distortion ratio, the source interference ratio, and the source-to-interference ratio, and the ratio with the highest ratio is taken as the result of sound separation and isolation.

[0077] Furthermore, this embodiment can also optimize and adjust the model. The method further includes: validating and tuning the DeepAudioSep model, including: increasing and optimizing training data; optimizing the model network of the DeepAudioSep model; and adjusting network parameters. The network parameters include: the size of hidden units, input size, loss optimization, number of training iterations, normalization technique type, and dataset size.

[0078] Figure 2 A flowchart of a deep learning-based sound isolation method is provided for embodiments of this application, such as... Figure 2 As shown, the model training in this application embodiment generally includes the following steps:

[0079] (1) Data input: audio file training set, test set, model training hyperparameters.

[0080] (2) DeepAudioSep model network training: Build the model network and perform iterative training of the model using the training set.

[0081] (3) Increase and optimize training data, optimize model network, and adjust network hyperparameters.

[0082] (4) Determine whether the loss function threshold or the maximum number of iterations will be reached. If yes, perform DeepAudioSep model verification: use the test set to verify the accuracy and efficiency of the model. If no, return to step (3).

[0083] (5) Determine whether the test performance index requirements are met. If yes, output the optimal model. If no, return to step (3).

[0084] The layers in the DeepAudioSep model's autoencoding stage correspond to: (1) a vertical convolutional layer for obtaining frequency features, (2) a horizontal convolutional layer for obtaining time-related features, thus outputting time-frequency codes, and (3) a fully connected layer with a Corrected Linear Unit (ReLU) as activation, which shares the output with the layers in the decoding stage. In each decoding stage, for every four classes, the layers correspond to: (1) a fully connected layer sharing the same dimension as the encoding horizontal convolutional layer's minimum information loss ReLU activation function, and (2) horizontal and (3) vertical deconvolutional layers sharing the opposite dimensions of the horizontal and vertical convolutional layers in the encoding stage. The outputs of the ten decoding stages are then concatenated to apply a ReLU, which is then used for loss calculation and backpropagation.

[0085] Figure 3 The flowchart of the sound separation algorithm provided in the embodiments of this application is as follows: Figure 3 As shown, the sound separation algorithm of this application includes:

[0086] (1) Input data: Mixed speech data.

[0087] (2) Feature extraction using STFT.

[0088] (3) Perform data normalization.

[0089] (4) Set the model parameters.

[0090] (5) Obtain the DeepAudioSep model.

[0091] (6) Obtain the STFT spectrum diagram of each category of audio.

[0092] (7) Filter spectrum diagram.

[0093] (8) Output the audio separation results.

[0094] In addition, this application also provides a sound isolation device based on deep learning, such as... Figure 5 As shown, the device includes:

[0095] The first module 501 is used to obtain the audio file used to construct the DeepAudioSep model and to preprocess the audio file used to construct the DeepAudioSep model.

[0096] The second module 502 is used to construct and train the DeepAudioSep model, which includes a mixed source input and ten isolated source outputs, wherein all source outputs have undergone feature extraction and normalization.

[0097] The third module 503 is used for sound separation using the DeepAudioSep model.

[0098] In addition, this application also provides a sound isolation device based on deep learning, such as... Figure 5 As shown, the device includes:

[0099] At least one processor 601;

[0100] At least one memory 602, the memory 602 being used to store at least one program;

[0101] When at least one of the programs is executed by at least one of the processors 601, the deep learning-based sound isolation method as described in the preceding embodiments is implemented.

[0102] In addition, this application also provides a storage medium storing a processor-executable program that, when executed by a processor, implements the deep learning-based sound isolation method as described in the preceding embodiments.

[0103] Similarly, the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0104] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this application are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.

[0105] Furthermore, although this application is described in the context of functional modules, it should be understood that, unless otherwise stated to the contrary, one or more of the functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding this application. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional technology for an engineer. Therefore, those skilled in the art can implement the application set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of this application, which is determined by the full scope of the appended claims and their equivalents.

[0106] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0107] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0108] More specific examples (a non-exhaustive list) of computer-readable media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0109] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0110] In the foregoing description of this specification, the references to terms such as "one embodiment," "another embodiment," or "some embodiments," etc., indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0111] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

[0112] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A sound isolation method based on deep learning, characterized in that, The method includes: Obtain the audio file used to build the DeepAudioSep model and preprocess the audio file used to build the DeepAudioSep model; Construct and train the DeepAudioSep model, which includes a mixed source input and ten isolated source outputs, all of which have undergone feature extraction and normalization. Sound separation is performed using the DeepAudioSep model; Constructing and training the DeepAudioSep model includes: Several classes are extracted from the output of the DeepAudioSep model to obtain filters; Each filter is multiplied by the input of the DeepAudioSep model to obtain a predicted isolated output filter, which is used as input to obtain ten computed outputs. The loss target output for each class is calculated by measuring the mean square error between the computed outputs and the provided error. The loss target outputs for each class are aggregated, and the DeepAudioSep model is optimized using the ADAM optimization algorithm.

2. The sound isolation method based on deep learning according to claim 1, characterized in that, Obtain the audio files used to build the DeepAudioSep model, including: The first audio file is obtained by synthesizing several categories of sounds, and the several categories of sounds include at least the sounds of wind, rain, thunder, insects, frogs, birds, dogs, and human voices. Several randomly selected additional audio files are superimposed onto the first audio file to obtain the audio file used to construct the DeepAudioSep model.

3. The sound isolation method based on deep learning according to claim 1, characterized in that, Preprocessing the audio files used to construct the DeepAudioSep model includes: The audio file used to build the DeepAudioSep model is processed into a 5-second audio segment, with pitch shifted by 2 or 4 steps, stretching by 1.2 times, and loudness shifted between -5 and +5 points. Encoded in WAV format using a PCM codec, mono with a sampling frequency of 44100 Hz, the audio file duration was reduced to 3 seconds by converting the input and output audio files to 22050 Hz, and normalization was performed using the minimum MAX parameter.

4. The sound isolation method based on deep learning according to claim 1, characterized in that, The method further includes: validating and optimizing the DeepAudioSep model, including: Increase and optimize training data; Optimize the model network of the DeepAudioSep model; Adjust network parameters.

5. The sound isolation method based on deep learning according to claim 4, characterized in that, The network parameters include: the size of the hidden unit, the input size, the loss optimization, the number of training iterations, the type of normalization technique, and the dataset size.

6. The sound isolation method based on deep learning according to claim 1, characterized in that, Sound separation using the DeepAudioSep model includes: Input a mixed audio file; The mixed sound file is output to different categories of sound files by calling the model; The results are used to evaluate the preservation of sound separation; The evaluation of the sound separation retention results is based on the signal distortion ratio, source-to-interference ratio, and source-to-interference ratio, with the highest ratio being taken as the result of sound separation and isolation.

7. A sound isolation device based on deep learning, characterized in that, The device includes: The first module is used to obtain the audio file used to build the DeepAudioSep model and to preprocess the audio file used to build the DeepAudioSep model. The second module is used to construct and train the DeepAudioSep model, which includes a mixed source input and ten isolated source outputs, all of which have undergone feature extraction and normalization. The third module is used for sound separation using the DeepAudioSep model; Constructing and training the DeepAudioSep model includes: Several classes are extracted from the output of the DeepAudioSep model to obtain filters; Each filter is multiplied by the input of the DeepAudioSep model to obtain a predicted isolated output filter, which is used as input to obtain ten computed outputs. The loss target output for each class is calculated by measuring the mean square error between the computed outputs and the provided error. The loss target outputs for each class are aggregated, and the DeepAudioSep model is optimized using the ADAM optimization algorithm.

8. A sound isolation device based on deep learning, characterized in that, The device includes: At least one processor; At least one memory for storing at least one program; The deep learning-based sound isolation method as described in any one of claims 1-6 is implemented when at least one of the programs is executed by at least one of the processors.

9. A storage medium, characterized in that, The storage medium stores a processor-executable program, which, when executed by the processor, implements the deep learning-based sound isolation method as described in any one of claims 1-6.

Images