Audio separation method and apparatus, electronic device, and storage medium

Abstract

Embodiments of the present application disclose an audio separation method and device, electronic equipment and storage medium, which can reduce the amount of calculation in audio separation by encoding the sample audio spectrum information, thereby improving the efficiency of audio separation; the linear feature vector can be obtained by linear modulation of the sample audio feature vector, and the linear feature vector of the initial bottleneck layer feature vector of the replacement bottleneck layer is decoded and processed, the parameters in the bottleneck layer can be updated, thereby improving the robustness of the deep learning segmentation network; training the deep learning segmentation network according to the target loss value can improve the accuracy of the deep learning segmentation network in audio separation; when the deep learning segmentation network is used for audio separation of the audio to be separated, no matter how short the length of the audio to be separated is, and no matter what kind of financial business audio is, the effect of audio separation can also be achieved, that is, the accuracy of audio separation of the audio to be separated can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of financial technology, and in particular to an audio separation method and device, an electronic device, and a storage medium. BACKGROUND

[0002] Currently, most financial products are accompanied by corresponding product explanation videos, which contain explanation audio. Alternatively, banks, insurance companies, or other financial institutions record customer service calls or sales calls. In some application scenarios, the above financial business audio needs to be separated.

[0003] In related technologies, the audio separation method based on deep learning is usually limited to the audio types in the data set of the training model, and cannot effectively separate the unannotated audio in the data set, thereby reducing the accuracy of audio separation of the audio to be separated. SUMMARY

[0004] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0005] The embodiments of the present application provide an audio separation method, device, electronic device and storage medium, which can improve the accuracy of audio separation of the audio to be separated when separating the financial business audio.

[0006] In a first aspect, the embodiments of the present application provide an audio separation method, comprising:

[0007] obtaining a sample audio, determining sample audio spectrum information of the sample audio, encoding the sample audio spectrum information to obtain a sample audio feature vector corresponding to the sample audio;

[0008] linearly modulating the sample audio feature vector to obtain a linear feature vector;

[0009] inputting the linear feature vector into a bottleneck layer of a deep learning segmentation network, replacing the initial bottleneck layer feature vector of the bottleneck layer with the linear feature vector to obtain a target bottleneck layer feature vector;

[0010] decoding processing according to the target bottleneck layer feature vector to obtain a sample separated audio;

[0011] determining a target loss value of the deep learning segmentation network according to the sample separated audio, and training the deep learning segmentation network according to the target loss value;

[0012] obtaining an audio to be separated, inputting the audio to be separated into the deep learning segmentation network to obtain a target separated audio.

[0013] In some embodiments, the number of the sample audios is multiple, the linear modulation on the sample audio feature vectors is to obtain a linear feature vector, including:

[0014] The mean value of the sample audio feature vectors corresponding to the multiple sample audios is determined to obtain a sample mean vector;

[0015] The linear modulation is performed on the sample mean vector to obtain a linear feature vector.

[0016] In some embodiments, the linear modulation on the sample mean vector is to obtain a linear feature vector, including:

[0017] The initial bottleneck layer feature vector of the bottleneck layer is determined;

[0018] The first parameter matrix based on the sample mean vector and the second parameter matrix based on the sample mean vector are determined;

[0019] The linear modulation is performed on the initial bottleneck layer feature vector, the first parameter matrix and the second parameter matrix to obtain the linear feature vector.

[0020] In some embodiments, the linear modulation on the initial bottleneck layer feature vector, the first parameter matrix and the second parameter matrix is to obtain the linear feature vector, including:

[0021] The initial bottleneck layer feature vector is multiplied by the first parameter matrix to obtain a feature vector product;

[0022] The linear feature vector is obtained according to the sum of the feature vector product and the second parameter matrix.

[0023] In some embodiments, the encoding on the sample audio spectrum information is to obtain the sample audio feature vector corresponding to the sample audio, including:

[0024] The sample audio spectrum information is input into a pre-trained sample adjustment encoder, and the sample audio spectrum information is convoluted to obtain a sample audio feature map;

[0025] The feature flattening is performed on the sample audio feature map to obtain the sample audio feature vector corresponding to the sample audio.

[0026] In some embodiments, the determination of the target loss value of the deep learning segmentation network according to the sample separated audio includes:

[0027] The first loss value is determined according to the sample separated audio, and the second loss value is determined according to the sample separated audio and the sample number of the sample separated audio;

[0028] determine a target loss value of the deep learning segmentation network according to the first loss value and the second loss value.

[0029] In some embodiments, the inputting the audio to be separated into the deep learning segmentation network to obtain target separated audio comprises:

[0030] inputting the audio to be separated into the deep learning segmentation network to perform down-sampling on the audio to be separated to obtain sampled audio to be separated;

[0031] performing up-sampling on the sampled audio to be separated to obtain target separated audio.

[0032] In a second aspect, the embodiments of the present application further provide an audio separation device, comprising:

[0033] an audio acquisition module, configured to acquire sample audio, determine sample audio spectrum information of the sample audio, and encode the sample audio spectrum information to obtain a sample audio feature vector corresponding to the sample audio;

[0034] a feature modulation module, configured to perform linear modulation on the sample audio feature vector to obtain a linear feature vector;

[0035] a feature replacement module, configured to input the linear feature vector into a bottleneck layer of a deep learning segmentation network, replace an initial bottleneck layer feature vector of the bottleneck layer with the linear feature vector, and obtain a target bottleneck layer feature vector;

[0036] a decoding module, configured to perform decoding processing according to the target bottleneck layer feature vector to obtain sample separated audio;

[0037] a network training module, configured to determine a target loss value of the deep learning segmentation network according to the sample separated audio, and train the deep learning segmentation network according to the target loss value;

[0038] a target separated audio generation module, configured to acquire audio to be separated, input the audio to be separated into the deep learning segmentation network, and obtain target separated audio.

[0039] In a third aspect, the embodiments of the present application further provide an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the audio separation method of the first aspect when executing the computer program.

[0040] In a fourth aspect, the embodiments of the present application further provide a computer readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement the audio separation method of the first aspect.

[0041] The embodiments of the present application at least have the following beneficial effects:

[0042] The audio separation method provided by the embodiments of the present application can reduce the calculation amount during audio separation, and further improve the efficiency of audio separation by obtaining a sample audio, determining sample audio spectrum information of the sample audio, encoding the sample audio spectrum information to obtain a sample audio feature vector corresponding to the sample audio, then modulating the sample audio feature vector linearly to obtain a linear feature vector, inputting the linear feature vector into a bottleneck layer of a deep learning segmentation network, replacing an initial bottleneck layer feature vector of the bottleneck layer with the linear feature vector to obtain a target bottleneck layer feature vector, and decoding the target bottleneck layer feature vector; the parameters in the bottleneck layer can be updated through back propagation, and the robustness of the deep learning segmentation network can be improved, so that the effect of audio separation can be improved; the target loss value of the deep learning segmentation network is determined by the sample separated audio, the deep learning segmentation network is trained according to the target loss value, and the accuracy of the deep learning segmentation network in audio separation of the audio to be separated can be improved; finally, when the deep learning segmentation network is used for audio separation of the audio to be separated, the effect of audio separation can be achieved regardless of the length of the audio to be separated and regardless of the type of financial business audio, that is, the accuracy of audio separation of the audio to be separated can be improved.

[0043] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the present application. The objects and other advantages of the present application can be achieved and obtained by the structure particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF DRAWINGS

[0044] The accompanying drawings are included to provide a further understanding of the technical solutions of the present application, and constitute a part of the specification, and are used together with the embodiments of the present application to explain the technical solutions of the present application, and do not constitute a limitation on the technical solutions of the present application.

[0045] Figure 1 A schematic diagram of an implementation environment provided for the embodiments of the present application;

[0046] Figure 2 A flowchart of an audio separation method provided for an embodiment of the present application;

[0047] Figure 3 For Figure 2 A flowchart of a specific method of step 202 in the embodiment;

[0048] Figure 4 For Figure 3 A flowchart of a specific method of step 302 in the embodiment;

[0049] Figure 5 As Figure 4 A flow chart of a specific method of step 403 in the embodiment;

[0050] Figure 6 As Figure 2 A flow chart of a specific method of step 201 in the embodiment;

[0051] Figure 7 As Figure 2 A flow chart of a specific method of step 205 in the embodiment;

[0052] Figure 8 As Figure 2 A flow chart of another specific method of step 206 in the embodiment;

[0053] Figure 9 A flow chart of an audio separation method provided by another embodiment of the present application;

[0054] Figure 10 A structural schematic diagram of an audio separation device provided by the embodiment of the present application;

[0055] Figure 11 A structural schematic diagram of an electronic device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0056] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0057] It should be understood that in the description of the embodiments of the present application, the meaning of several is one or more, the meaning of multiple (or multiple items) is two or more, greater than, less than, more than, etc. are understood as not including the number, above, below, etc. are understood as including the number. If there is a description of "first", "second", etc., it is only used to distinguish technical features for the purpose, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features or implicitly indicating the sequence of indicated technical features.

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application, and are not intended to limit the present application.

[0059] First, the meanings of several terms involved in the present application are analyzed:

[0060] Artificial Intelligence (AI): is a new technical science of studying, developing the theory, method, technology and application system for simulating, extending and expanding human intelligence; Artificial intelligence is a branch of computer science, artificial intelligence attempts to understand the essence of intelligence, and produce a new intelligent machine that can react in a similar way to human intelligence, the research in this field includes robots, language recognition, image recognition, natural language processing and expert system, etc. Artificial intelligence can simulate the information process of human consciousness and thinking. Artificial intelligence is also the theory, method, technology and application system of using digital computer or digital computer controlled machine to simulate, extend and expand human intelligence, perceive environment, acquire knowledge and use knowledge to obtain the best results.

[0061] Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software level technology. Artificial intelligence basic technology generally includes such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning, automatic driving, intelligent transportation and other major directions.

[0062] Machine Learning (ML) is a multi-disciplinary subject, involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It is a branch of computer science that studies how computers simulate or implement human learning behavior to acquire new knowledge or skills, and reorganize existing knowledge structure to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental approach to making computers intelligent, and its applications are widespread in various fields of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and teaching learning.

[0063] Natural Language Processing (NLP): NLP uses computers to process, understand and use human language (such as Chinese, English, etc.), NLP is a branch of artificial intelligence and is an interdisciplinary subject of computer science and linguistics, also known as computational linguistics. Natural language processing includes syntax analysis, semantic analysis, discourse understanding, etc. Natural language processing is commonly used in machine translation, handwritten and printed character recognition, speech recognition and text-to-speech conversion, information retrieval, information extraction and filtering, text classification and clustering, public opinion analysis and opinion mining, etc. It involves language processing related data mining, machine learning, knowledge acquisition, knowledge engineering, artificial intelligence research and language computing related linguistic research.

[0064] Medical cloud: the medical cloud refers to using "cloud computing" to create a medical and health service cloud platform based on new technologies such as cloud computing, mobile technology, multimedia, 4G communication, big data, and the Internet of Things, combined with medical technology, to realize the sharing of medical resources and the expansion of the scope of medical treatment. Because of the combination of cloud computing technology, the medical cloud improves the efficiency of medical institutions and facilitates residents to seek medical treatment. Like the hospital's appointment registration, electronic medical records, medical insurance, etc. are the products of the combination of cloud computing and the medical field, and the medical cloud also has the advantages of data security, information sharing, dynamic expansion, and global layout.

[0065] It should be appreciated that the embodiments of the present application can acquire and process related data based on artificial intelligence technology. Artificial intelligence is the theory, method, technology and application system that uses digital computers or digital computer controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0066] At present, most financial products will be matched with corresponding product explanation videos, which will contain explanation audio, or banks, insurance companies or other financial institutions will record customer service calls or sales calls, and in some application scenarios, the above financial business audio needs to be separated.

[0067] In the related art, the audio separation method based on deep learning is usually limited to the audio types in the data set of the training model, and cannot effectively separate the unlabeled audio in the data set, thereby reducing the accuracy of audio separation on the audio to be separated.

[0068] The audio separation method, device, electronic equipment and storage medium provided by the embodiments of the present application are specifically described by the following embodiments. First, the audio separation method in the embodiments of the present application is described.

[0069] Reference Figure 1 , Figure 1 An implementation environment provided by the embodiments of the present application is shown in the schematic diagram, which includes a terminal 101 and a server 102, wherein the terminal 101 and the server 102 are connected through a communication network 103.

[0070] The server 102 can be a stand-alone physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and basic cloud computing services such as big data and artificial intelligence platforms.

[0071] In addition, server 102 can also be a node server in a blockchain network.

[0072] Terminal 101 can be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smartwatch, vehicle terminal, etc., but is not limited to these. Terminal 101 and server 102 can be directly or indirectly connected via wired or wireless communication, and this embodiment of the invention does not impose any limitations.

[0073] based on Figure 1 The implementation environment shown in this embodiment of the invention provides an audio separation method, which can be implemented by... Figure 1 The server 102 shown can be executed, or it can be... Figure 1 The terminal 101 shown is executed, or by Figure 1 The terminal 101 and server 102 shown cooperate to execute the audio separation method in this embodiment of the invention. Figure 1 The following explanation uses server 102 as an example.

[0074] Reference Figure 2 , Figure 2 This is a flowchart illustrating an audio separation method provided in an embodiment of the present invention. The audio separation method includes, but is not limited to, the following steps 201 to 206.

[0075] Step 201: Obtain sample audio, determine the sample audio spectrum information, encode the sample audio spectrum information, and obtain the sample audio feature vector corresponding to the sample audio.

[0076] Step 202: Linearly modulate the sample audio feature vector to obtain a linear feature vector;

[0077] Step 203: Input the linear feature vector into the bottleneck layer of the deep learning segmentation network, and replace the initial bottleneck layer feature vector with the linear feature vector to obtain the target bottleneck layer feature vector;

[0078] Step 204: Decode the target bottleneck layer feature vector to obtain sample-separated audio;

[0079] Step 205: Determine the target loss value of the deep learning segmentation network based on the audio samples, and train the deep learning segmentation network based on the target loss value;

[0080] Step 206: Obtain the audio to be separated. Input the audio to be separated into a deep learning segmentation network to obtain the target separated audio.

[0081] The sample audio can be an audio segment with a time length of 3 seconds, 4 seconds or other time lengths. The sample audio spectrum information is a spectrum graph of the sample audio. There are many implementation manners for determining the sample audio spectrum information of the sample audio. For example, the audio can be converted into a mel spectrum graph using a librosa library. The mel spectrum graph is a spectrum graph converted from frequency to a mel scale.

[0082] The decoding process is performed according to the target bottleneck layer feature vector to obtain the sample separated audio. Specifically, inverse Fourier transform can be performed on the target bottleneck layer feature vector to obtain the sample separated audio, which is not specifically limited herein.

[0083] It should be noted that the audio separation using the deep learning segmentation network is not limited to the audio types in the data set. Even if the audio in the data set is not labeled, the audio of this type can also be separated.

[0084] For example, it is assumed that the sample separated audio is a product introduction audio, and at the same time of the sales call, walk sound and car sound are transmitted in the call environment. Therefore, the sales call audio is mixed with noise. Based on this, the sample audio spectrum information of the sales call audio as a sample is determined. At this time, the sample audio spectrum information can include the spectrum information of the human voice, walk sound and car sound in the call. Further, the sample audio spectrum information is encoded to obtain the corresponding sample audio feature vector. The linear feature vector is input to the bottleneck layer of the deep learning segmentation network. The sample audio feature vector is linearly modulated to obtain the linear feature vector. The linear feature vector replaces the initial bottleneck layer feature vector of the bottleneck layer to obtain the target bottleneck layer feature vector. The decoding process is performed according to the target bottleneck layer feature vector to obtain the sample separated audio. At this time, the sample separated audio should be the human voice in the call. Therefore, the deep learning segmentation network can be further trained according to the sample separated audio. After the deep learning segmentation network is trained, the sales call audio to be segmented including the human voice and the noise can be input to the trained deep learning segmentation network. Therefore, the human voice in the sales call audio to be segmented can be accurately obtained.

[0085] In this embodiment, by adopting the audio separation method comprising the steps 201 to 206, the calculation amount during audio separation can be reduced by obtaining the sample audio, determining the sample audio spectrum information of the sample audio, encoding the sample audio spectrum information, and obtaining the sample audio feature vector corresponding to the sample audio, thereby improving the efficiency of audio separation. Then, by linearly modulating the sample audio feature vector, the linear feature vector is obtained, the linear feature vector is input into the bottleneck layer of the deep learning segmentation network, the initial bottleneck layer feature vector of the bottleneck layer is replaced by the linear feature vector, the target bottleneck layer feature vector is obtained, and the decoding processing is performed according to the target bottleneck layer feature vector. The parameters in the bottleneck layer can be updated through back propagation, thereby improving the robustness of the deep learning segmentation network, and the effect of audio separation can be improved. Then, the target loss value of the deep learning segmentation network is determined by the sample separation audio, and the deep learning segmentation network is trained according to the target loss value, so that the accuracy of the deep learning segmentation network in audio separation of the audio to be separated can be improved. Finally, when the audio to be separated is separated by the deep learning segmentation network, no matter how short the length of the audio to be separated is, and no matter what kind of financial business audio is, the effect of audio separation can also be achieved, that is, the accuracy of audio separation of the audio to be separated can be improved.

[0086] In a possible implementation manner, the embodiment of the present application introduces a plurality of sample audios, and linearly modulates the sample audio feature vectors of the plurality of sample audios to obtain linear feature vectors. Referring to Figure 3 , Figure 3 The flowchart for linearly modulating the sample audio feature vectors to obtain the linear feature vectors provided by the embodiment of the present application is shown in the figure. In the step 202, the sample audio feature vectors are linearly modulated to obtain the linear feature vectors, which can specifically include the following steps 301 to 302.

[0087] Step 301: determining the mean value of the sample audio feature vectors corresponding to the plurality of sample audios to obtain a sample mean vector;

[0088] Step 302: linearly modulating the sample mean vector to obtain the linear feature vector.

[0089] In a possible implementation manner, the sample mean vector can be linearly modulated by using the feature linear modulation (Feature-wise liner modulation, FiLM) technology, and the embodiment of the present application does not make specific limitation thereto.

[0090] In the embodiment, by adopting the audio separation method including the steps 301 to 302, the mean value of the sample audio feature vectors corresponding to the plurality of sample audios can be determined, the sample mean vector is obtained, the sample mean vector is linearly modulated to obtain the linear feature vector, the parameters in the bottleneck layer of the deep learning segmentation network can be updated using the sample mean vector in the subsequent steps, and the effect of the audio separation of the deep learning segmentation network can be improved.

[0091] In a possible implementation, when the number of sample audios is a plurality, the mode feature value can be determined from the sample audio feature vectors corresponding to the plurality of sample audios, the linear feature vector is obtained by linearly modulating the mode feature value; or the maximum feature value can be determined from the sample audio feature vectors corresponding to the plurality of sample audios, the linear feature vector is obtained by linearly modulating the maximum feature value; or the standard deviation of the sample audio feature vectors corresponding to the plurality of sample audios is determined to obtain the sample standard deviation, the linear feature vector is obtained by linearly modulating the sample standard deviation, and the like. The embodiments of the application do not make specific limitations on this.

[0092] In a possible implementation, the initial bottleneck layer feature vector of the bottleneck layer is introduced again, the sample mean vector is linearly modulated to obtain the linear feature vector. Referring to Figure 4 , Figure 4 The specific flowchart of the audio separation method provided by the embodiment of the application is shown in the foregoing step 302, the sample mean vector is linearly modulated to obtain the linear feature vector, and the specific steps 401 to 403 can be included.

[0093] Step 401: determining the initial bottleneck layer feature vector of the bottleneck layer;

[0094] Step 402: determining the first parameter matrix based on the sample mean vector and the second parameter matrix based on the sample mean vector;

[0095] Step 403: linearly modulating the initial bottleneck layer feature vector, the first parameter matrix, and the second parameter matrix to obtain the linear feature vector.

[0096] The first parameter matrix based on the sample mean vector can be a matrix formed by parameters obtained by a first function with the sample mean vector as an independent variable. Similarly, the second parameter matrix based on the sample mean vector can be a matrix formed by parameters obtained by a second function with the sample mean vector as an independent variable. The first function and the second function can be any function, such as two neural networks. Alternatively, the first function and the second function can be the same function, and can be any function, such as the same neural network. The embodiments of the present application do not make specific limitations in this regard. It can be understood that, since the first function and the second function share the sample mean vector, the linear feature vector obtained by linear modulation of the initial bottleneck layer feature vector, the first parameter matrix based on the first function, and the second parameter matrix based on the second function can improve the training effect and network performance of the deep learning segmentation network.

[0097] In the embodiment, by adopting the audio separation method including the steps 401 to 403, the initial bottleneck layer feature vector of the bottleneck layer is determined, and the first parameter matrix based on the sample mean vector and the second parameter matrix based on the sample mean vector are determined. The initial bottleneck layer feature vector, the first parameter matrix, and the second parameter matrix are linearly modulated to obtain the linear feature vector, so that the parameters in the bottleneck layer of the deep learning segmentation network are updated using the linear feature vector in the subsequent steps, and the effect of audio separation of the deep learning segmentation network can be improved.

[0098] In a possible implementation manner, referring to Figure 5 , Figure 5 The flowchart of the audio separation method provided by the embodiment of the present application is shown in the above step 403. The initial bottleneck layer feature vector, the first parameter matrix, and the second parameter matrix are linearly modulated to obtain the linear feature vector, which can specifically include the following steps 501 to 502.

[0099] Step 501: multiplying the initial bottleneck layer feature vector and the first parameter matrix to obtain a feature vector product;

[0100] Step 502: obtaining the linear feature vector according to the sum of the feature vector product and the second parameter matrix.

[0101] In a possible implementation manner, the linear feature vector FiLM(x) can be represented by formula (1), that is:

[0102] FiLM(x) = a(z)x + b(z) (1)

[0103] In the formula (1), x represents an initial bottleneck layer feature vector, z represents a sample mean vector, a(z) represents a first parameter matrix, b(z) represents a second parameter matrix, and a(z)x represents a feature vector product.

[0104] It should be noted that the first parameter matrix and the second parameter matrix are both learnable parameter matrices, that is, the parameters in the first parameter matrix and the second parameter matrix can be updated through back propagation (that is, through the decoding process of the target bottleneck layer feature vector replaced by the linear feature vector).

[0105] In this embodiment, by adopting the audio separation method including the steps 501 to 502, the initial bottleneck layer feature vector can be multiplied by the first parameter matrix to obtain a feature vector product, and then a linear feature vector can be obtained according to the sum of the feature vector product and the second parameter matrix, that is, the initial bottleneck layer feature vector can be scaled by multiplying the initial bottleneck layer feature vector by the first parameter matrix, and the initial bottleneck layer feature vector can be translated according to the sum of the feature vector product and the second parameter matrix. In addition, the first parameter matrix is obtained based on the sample mean vector, and the second parameter matrix is obtained based on the sample mean vector, so that the initial bottleneck layer feature vector can be linearly modulated based on the sample mean vector to obtain a linear feature vector. In order to update the parameters in the bottleneck layer of the deep learning segmentation network using the linear feature vector in the subsequent step, the effect of audio separation of the deep learning segmentation network can be improved.

[0106] In a possible implementation, a sample adjustment encoder is introduced in the embodiment of the present application. Then, the sample audio spectrum information is encoded by the pre-trained sample adjustment encoder to obtain a sample audio feature vector corresponding to the sample audio. Referring to Figure 6 , Figure 6 The specific flowchart of the audio separation method provided by the embodiment of the present application is shown in the above step 201, the sample audio spectrum information is encoded to obtain a sample audio feature vector corresponding to the sample audio, which can specifically include the following steps 601 to 602.

[0107] Step 601: input the sample audio spectrum information into the pre-trained sample adjustment encoder, and perform convolution processing on the sample audio spectrum information to obtain a sample audio feature map;

[0108] Step 602: flattening the sample audio feature map to obtain a sample audio feature vector corresponding to the sample audio.

[0109] For example, the sample adjustment encoder is sequentially provided with four convolution blocks and a flattening layer, each of the convolution blocks includes 64 3x3 convolution kernels, a batch normalization layer, an activation function (such as a ReLU function), and a 2x2 max-pooling layer, and the convolution kernel can be used for mapping processing or dimension reduction processing of the vector. The flattening layer can be used for feature flattening of the sample audio feature map to obtain the sample audio corresponding sample audio feature vector. Specifically, the sample audio spectrum information is input into the convolution block of the pre-trained sample adjustment encoder, and the sample audio spectrum information is sequentially convolved by the four convolution blocks to obtain a sample audio feature map, and then the sample audio feature map is flattened by the flattening layer to obtain the sample audio corresponding sample audio feature vector. It can be understood that in the convolution block, as the network depth increases, the feature value distribution of each layer gradually approaches the upper and lower ends of the output interval of the activation function (the saturation interval of the activation function), which can cause gradient disappearance, and the batch normalization layer can pull the feature value distribution back to the standard normal distribution, and the feature value will fall in the interval sensitive to the input of the activation function, so that a small change in the input can cause a large change in the loss function, so that the gradient becomes large to avoid gradient disappearance, and also speeds up the convergence speed. The size of the max-pooling layer is 2x2, and the step is also 2, so the length of the output sample audio feature map is half of the length of the sample audio spectrum information, and the width of the output sample audio feature map is half of the width of the sample audio spectrum information, so that the calculation amount during audio separation can be reduced, and the efficiency of audio separation can be improved.

[0110] In the embodiment, by adopting the audio separation method including the steps 601 to 602, the sample audio spectrum information is input into the pre-trained sample adjustment encoder, the sample audio spectrum information is convolved to obtain a sample audio feature map, the calculation amount during audio separation can be reduced, and the efficiency of audio separation can be improved, and then the sample audio feature map is flattened to make the sample audio feature map into a vector form convenient for processing by the deep learning segmentation network, and the training efficiency and training effect of the deep learning segmentation network can be improved.

[0111] In a possible implementation, the target loss value is introduced in the embodiment of the application. Then, the target loss value of the deep learning segmentation network is determined by determining the first loss value and the second loss value. For details, refer to Figure 7 , Figure 7 The specific flowchart of the audio separation method provided by the embodiment of the application is shown in the above step 205, the target loss value of the deep learning segmentation network is determined according to the sample separated audio, which can specifically include the following steps 701 to 702.

[0112] Step 701: determining a first loss value according to the sample-separated audio, and determining a second loss value according to the sample-separated audio and the sample quantity of the sample-separated audio;

[0113] Step 702: determining a target loss value of the deep learning segmentation network according to the first loss value and the second loss value.

[0114] For example, each sample-separated audio corresponds to a label, which is a separated label or a non-separated label. If the sample quantity of the sample-separated audio is n, the n sample-separated audios are input into the deep learning segmentation network, and the prediction value of the audio separation of each sample-separated audio can be obtained.

[0115] Then, the first loss value Loss1 is calculated according to the prediction value of the audio separation of the n sample-separated audios and the true label of the n sample-separated audios (for example, the label can be 1 or 0, 1 for audio separation and 0 for non-separated audio), specifically:

[0116] Suppose the true label of the i-th sample-separated audio is x(i), and the prediction value of the audio separation of the i-th sample-separated audio is y(i), and the first loss value Loss1 of the n sample-separated audios can be represented as

[0117]

[0118] In formula (2), ∈ is a regularization term coefficient, which can make the argument of the logarithmic function greater than 0, that is, It can be understood that the deep learning segmentation network is trained by the first loss value, which can improve the robustness of the deep learning segmentation network.

[0119] Similarly, suppose the prediction value of the i-th sample-separated audio is x i , and the true value of the audio separation of the i-th sample-separated audio is y i , and the second loss value Loss2 of the n sample-separated audios can be represented as

[0120]

[0121] Further, the first loss value and the second loss value can be weighted to obtain the target loss value of the deep learning segmentation network, or the mean value of the first loss value and the second loss value can be calculated, and the mean value is determined as the target loss value of the deep learning segmentation network, and the like, which is not specifically limited by the embodiments of the present application.

[0122] It can be understood that the first loss value can be obtained by the SignalDistrtionRatio function in torchmetrics.audio; and the second loss value can be obtained by the MAELoss function of pytorch.

[0123] In the embodiment, by adopting the audio separation method including the steps 701 to 702, the first loss value can be determined according to the sample separation audio, the second loss value can be determined according to the sample separation audio and the sample number of the sample separation audio, and then the target loss value of the deep learning segmentation network can be determined according to the first loss value and the second loss value, so as to train the deep learning segmentation network according to the target loss value in the subsequent step, to improve the robustness of the deep learning segmentation network, and further improve the accuracy of the deep learning segmentation network in audio separation of the to-be-separated audio.

[0124] In a possible implementation, the embodiment of the present application introduces the to-be-separated audio, and separates the to-be-separated audio by the deep learning segmentation network to obtain the target separation audio. Referring to Figure 8 , Figure 8 The specific flowchart of the audio separation method provided by the embodiment of the present application is shown in the above step 206, the to-be-separated audio is input into the deep learning segmentation network to obtain the target separation audio, which can specifically include the following steps 801 to 802.

[0125] Step 801: input the to-be-separated audio into the deep learning segmentation network, downsample the to-be-separated audio to obtain the sampled to-be-separated audio;

[0126] Step 802: upsample the sampled to-be-separated audio to obtain the target separation audio.

[0127] The to-be-separated audio is a mixed audio, for example, a mixed audio including two or more than two kinds of drum sound, piano sound, bass sound, triangle iron, cymbal, tambourine, sand hammer, cymbal, human voice, and the like, which will not be listed one by one here.

[0128] It can be understood that the down-sampling is performed on the to-be-separated audio to obtain a sampled to-be-separated audio, specifically, the feature extraction can be performed on the to-be-separated audio by using convolution operation to obtain the sampled to-be-separated audio; the up-sampling is performed on the sampled to-be-separated audio, specifically, the up-sampling can be performed on the sampled to-be-separated audio by using deconvolution operation, and the target separated audio can be obtained. Moreover, since the processing object of the up-sampling is the feature extracted by the down-sampling, that is, the sampled to-be-separated audio, the scale fusion can be performed on the features with the same number of channels in the sampled to-be-separated audio in the process of the up-sampling, and the multi-scale feature fusion is realized. In the process of the feature fusion, since the bottleneck layer feature vector of the bottleneck layer of the deep learning segmentation network is replaced by the linear feature vector obtained by linearly modulating the sample audio feature vector, the effect of the feature fusion can be improved, and thus the accuracy of the audio separation on the to-be-separated audio can be improved.

[0129] For example, assuming that the to-be-separated audio is a sales call audio, and at the same time of the sales call, the walking sound and the car sound are transmitted in the call environment, and thus the sales call audio is mixed with the noise. If the call content (i.e., the human voice in the sales call audio) in the sales call audio needs to be obtained, the sales call audio including the human voice and the noise can be input into the deep learning segmentation network, the down-sampling is performed on the sales call audio to obtain a sampled sales call audio, and then the up-sampling is performed on the sampled sales call audio to obtain the human voice (i.e., the target separated audio) in the sales call audio. Since the initial bottleneck layer feature vector of the bottleneck layer of the deep learning segmentation network is replaced by the linear feature vector, the robustness of the deep learning segmentation network is high, and thus the conference content can be accurately separated from the sales call audio regardless of the length of the sales call audio.

[0130] For example, assuming that the to-be-separated audio is a product explanation video of a financial product, and the explanation audio of the product explanation video can include background music, and the background music can include sounds of various musical instruments, such as clarinet, oboe, British tube, bassoon, saxophone, snare drum, bass drum, triangle, cymbals, tambourine, sand hammer, cymbals, gong, etc. If all the sounds of the musical instruments in the background music need to be separated, the background music can be input into the deep learning segmentation network, the down-sampling is performed on the background music to obtain a sampled background music, and then the up-sampling is performed on the sampled background music to obtain the audio of the various musical instruments in the background music (i.e., the target separated audio). Since the initial bottleneck layer feature vector of the bottleneck layer of the deep learning segmentation network is replaced by the linear feature vector, the effect of the audio separation can be achieved regardless of the type of the audio, that is, the audio of all the musical instruments can be accurately separated from the background music.

[0131] It is understandable that directly upsampling the audio to be separated would lead to the loss of information in the audio during the upsampling process. Therefore, in this embodiment, by adopting the audio separation method including the above steps 801 to 802, the audio to be separated can be input into a deep learning segmentation network, downsampled to obtain the sampled audio to be separated, and then upsampled to obtain the target audio to be separated. That is to say, by replacing the bottleneck layer feature vector with a linear feature vector obtained by linearly modulating the sample audio feature vector, the deep learning segmentation network corresponding to the bottleneck layer performs audio separation on the audio to be separated, which can improve the effect of feature fusion. In other words, it can make the size of the feature map corresponding to the target audio to be separated the same as the size of the feature map corresponding to the audio to be separated. This not only enhances the receptive field of the deep feature map, but also ensures that the shallow features have texture and other detailed features, while avoiding the loss of information in the audio to be separated during the upsampling process. Furthermore, since the initial bottleneck layer feature vector of the bottleneck layer of the deep learning segmentation network is replaced with a linear feature vector, the deep learning segmentation network has high robustness. Therefore, no matter how short the duration of the audio to be separated is, or what kind of financial business audio it is, it can achieve the effect of audio separation, which can improve the accuracy of audio separation.

[0132] In one possible implementation, audio separation can be used as an upstream task of other tasks (i.e., the audio separation method in this application embodiment can be used as an upstream task of other tasks), such as singing detection, melody extraction and lyric transcription, etc. This application embodiment does not impose specific limitations on this.

[0133] The audio separation method in this application embodiment is described in detail below with specific examples.

[0134] like Figure 9 As shown, firstly, n sample audio recordings of the insurance recording are obtained. Then, the librosa library is used to convert the n sample audio recordings into MEL spectrograms to obtain the sample audio spectrum information of the n sample audio recordings. The sample audio spectrum information is then input into the sample adjustment encoder to encode the sample audio spectrum information and obtain the sample audio feature vectors corresponding to the n sample audio recordings.

[0135] Next, the mean of the n sample audio feature vectors is determined to obtain the sample mean vector. The sample mean vector is linearly modulated to obtain the linear feature vector. The linear feature vector is input into the bottleneck layer of the deep learning segmentation network. The linear feature vector replaces the initial bottleneck layer feature vector of the bottleneck layer to obtain the target bottleneck layer feature vector. The target bottleneck layer feature vector is decoded by the decoder to obtain the sample separated audio.

[0136] Then, a target loss value of the deep learning segmentation network is determined according to the sample separated audio, and the deep learning segmentation network is trained according to the target loss value.

[0137] Finally, a medical insurance recording is obtained, the medical insurance recording is taken as the to-be-separated audio, the medical insurance recording is input into the deep learning segmentation network, a target separated audio is obtained, the to-be-separated audio is down-sampled by the encoder to obtain a sampled to-be-separated audio, and then the sampled to-be-separated audio is up-sampled by the decoder to obtain the target separated audio (such as an audio of a voice of a medical insurance staff).

[0138] It can be understood that the same number of features in the sampled to-be-separated audio can be scale fused in the up-sampling process to realize multi-scale feature fusion. In the process of feature fusion, since the bottleneck layer feature vector of the bottleneck layer of the deep learning segmentation network is replaced by the linear feature vector obtained by linearly modulating the sample audio feature vector, the effect of feature fusion can be improved, and thus the accuracy of audio separation on the to-be-separated audio can be improved.

[0139] It can be understood that although each step in each of the above flowcharts is displayed in sequence according to the arrow, these steps are not necessarily executed in the order indicated by the arrow. Unless otherwise specified in this embodiment, the execution of these steps has no strict order limitation, and these steps can be executed in other orders. Moreover, at least part of the steps in the above flowcharts can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be alternately executed with at least part of other steps or steps or stages in other steps.

[0140] It should be noted that in each specific embodiment of the present application, when relevant processing needs to be performed according to object attribute information (such as audio of a voice), user historical data, and user location information, and other data related to the identity or characteristics of the user, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the present application needs to obtain sensitive personal information of the user, the user's separate permission or separate consent will be obtained through a pop-up window or by jumping to a confirmation page, and after obtaining the separate permission or separate consent of the user, the necessary user-related data for enabling the present application to function normally will be obtained.

[0141] In addition, with reference to Figure 10 , Figure 10A structural schematic diagram of an audio separation device provided by an embodiment of the present application is provided, and the embodiment of the present application further provides an audio separation device 1000, comprising:

[0142] An audio acquisition module 1001 is configured to acquire sample audio, determine sample audio spectrum information of the sample audio, encode the sample audio spectrum information, and obtain a sample audio feature vector corresponding to the sample audio;

[0143] A feature modulation module 1002 is configured to linearly modulate the sample audio feature vector to obtain a linear feature vector;

[0144] A feature replacement module 1003 is configured to input the linear feature vector to a bottleneck layer of a deep learning segmentation network, replace an initial bottleneck layer feature vector of the bottleneck layer with the linear feature vector, and obtain a target bottleneck layer feature vector;

[0145] A decoding module 1004 is configured to perform decoding processing according to the target bottleneck layer feature vector to obtain sample separated audio;

[0146] A network training module 1005 is configured to determine a target loss value of the deep learning segmentation network according to the sample separated audio, and train the deep learning segmentation network according to the target loss value;

[0147] A target separated audio generation module 1006 is configured to acquire audio to be separated, input the audio to be separated to the deep learning segmentation network, and obtain target separated audio.

[0148] The audio separation device 1000 and the audio separation method described above are based on the same inventive concept. By acquiring sample audio, determining sample audio spectrum information of the sample audio, encoding the sample audio spectrum information, and obtaining a sample audio feature vector corresponding to the sample audio, the calculation amount during audio separation can be reduced, and the efficiency of audio separation can be improved. Then, by linearly modulating the sample audio feature vector to obtain a linear feature vector, inputting the linear feature vector to a bottleneck layer of a deep learning segmentation network, replacing an initial bottleneck layer feature vector of the bottleneck layer with the linear feature vector, and obtaining a target bottleneck layer feature vector, decoding processing according to the target bottleneck layer feature vector can update parameters in the bottleneck layer through back propagation, and the robustness of the deep learning segmentation network can be improved, thereby improving the effect of audio separation. By determining a target loss value of the deep learning segmentation network according to sample separated audio, and training the deep learning segmentation network according to the target loss value, the accuracy of the deep learning segmentation network in performing audio separation on audio to be separated can be improved. Finally, when performing audio separation on audio to be separated through the deep learning segmentation network, the effect of audio separation can be achieved regardless of the length of the audio to be separated, and regardless of the type of financial business audio, that is, the accuracy of audio separation on the audio to be separated can be improved.

[0149] It is understood that the audio separation device 1000 described above can also be specifically used to execute various processes described in the above audio separation method embodiments.

[0150] Reference Figure 11 , Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. The electronic device 1100 includes: a memory 1101, a processor 1102, and a computer program stored in the memory 1101 and executable on the processor 1102. When the computer program is executed, it is used to perform the above-described audio separation method.

[0151] The processor 1102 and the memory 1101 can be connected via a bus or other means.

[0152] The memory 1101, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs, such as the audio separation method described in the embodiments of the present invention. The processor 1102 implements the above-described audio separation method by running the non-transitory software program and instructions stored in the memory 1101.

[0153] The memory 1101 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store the audio separation method described above. Furthermore, the memory 1101 may include high-speed random access memory (RAM) and non-transitory memory, such as at least one storage device, flash memory, or other non-transitory solid-state storage device. In some embodiments, the memory 1101 may optionally include remotely located memories 1101 relative to the processor 1102, which can be connected to the electronic device 1100 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks (LANs), mobile communication networks, and combinations thereof.

[0154] The non-transient software program and instructions required to implement the above-described audio separation method are stored in memory 1101. When executed by one or more processors 1102, the above-described audio separation method is performed.

[0155] The embodiments described in this disclosure are for the purpose of more clearly illustrating the technical solutions of this disclosure and do not constitute a limitation on the technical solutions provided by this disclosure. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by this disclosure are also applicable to similar technical problems.

[0156] It will be understood by those skilled in the art that Figures 2-8The technical solutions shown in the above figures do not constitute a limitation on the embodiments of the present disclosure, and can include more or fewer steps than those shown, or combine certain steps, or different steps.

[0157] The apparatus embodiments described above are merely illustrative, and units described as separate components can or can not be physically separate, i.e., can be located in one place, or can be distributed over multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments.

[0158] Those of ordinary skill in the art can understand that all or some of the steps in the above disclosed method, the functions of the modules / units in the system and the device can be implemented as software, firmware, hardware and appropriate combinations thereof.

[0159] The terms "first", "second", "third", "fourth" and the like used in the specification of the present application and the above-described figures, if any, are used to distinguish similar objects, and do not necessarily have to describe a particular order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0160] It should be understood that in the present application, "at least one" means one or more, and "multiple" means two or more. "And / or" is used to describe the relationship between the associated objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0161] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented by other manners. For example, the apparatus embodiments described above are merely illustrative, for example, the division of the units is merely a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, apparatuses or units, and can be electrical, mechanical or other forms.

[0162] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, can be located in one place, or can be distributed to a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0163] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0164] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part of the prior art that contributes to the technical solutions or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program storage media.

[0165] The preferred embodiments of the present disclosure are described above with reference to the accompanying drawings, and the scope of the present disclosure is not limited thereto. Any modification, equivalent replacement and improvement made by those skilled in the art without departing from the scope and essence of the present disclosure shall be within the scope of the present disclosure.

Claims

1. An audio separation method, characterized in that, include: Acquire sample audio, determine the sample audio spectrum information of the sample audio, encode the sample audio spectrum information, and obtain the sample audio feature vector corresponding to the sample audio; The sample audio feature vector is linearly modulated to obtain a linear feature vector; The linear feature vector is input into the bottleneck layer of the deep learning segmentation network, and the linear feature vector replaces the initial bottleneck layer feature vector of the bottleneck layer to obtain the target bottleneck layer feature vector. The target bottleneck layer feature vector is used for decoding to obtain sample-separated audio. The target loss value of the deep learning segmentation network is determined based on the audio samples, and the deep learning segmentation network is trained based on the target loss value. The audio to be separated is obtained and input into the deep learning segmentation network to obtain the target separated audio.

2. The audio separation method according to claim 1, characterized in that, There are multiple sample audio files, and the linear modulation of the feature vectors of the sample audio files to obtain linear feature vectors includes: The mean value of the feature vectors of the sample audio corresponding to multiple sample audio samples is determined to obtain the sample mean vector; The sample mean vector is linearly modulated to obtain a linear feature vector.

3. The audio separation method according to claim 2, characterized in that, The step of linearly modulating the sample mean vector to obtain a linear feature vector includes: Determine the initial bottleneck layer feature vector; Determine a first parameter matrix and a second parameter matrix based on the sample mean vector; The initial bottleneck layer feature vector, the first parameter matrix, and the second parameter matrix are linearly modulated to obtain the linear feature vector.

4. The audio separation method according to claim 3, characterized in that, The step of linearly modulating the initial bottleneck layer feature vector, the first parameter matrix, and the second parameter matrix to obtain the linear feature vector includes: Multiply the initial bottleneck layer feature vector with the first parameter matrix to obtain the feature vector product; The linear eigenvector is obtained by summing the product of the eigenvectors and the second parameter matrix.

5. The audio separation method according to claim 1, characterized in that, The process of encoding the sample audio spectrum information to obtain the sample audio feature vector corresponding to the sample audio includes: The sample audio spectrum information is input into a pre-trained sample conditioning encoder, and the sample audio spectrum information is convolved to obtain a sample audio feature map. The sample audio feature map is flattened to obtain the sample audio feature vector corresponding to the sample audio.

6. The audio separation method according to claim 1, characterized in that, The step of determining the target loss value of the deep learning segmentation network based on the audio separation of the samples includes: A first loss value is determined based on the sample-separated audio, and a second loss value is determined based on the sample-separated audio and the number of samples in the sample-separated audio. The target loss value of the deep learning segmentation network is determined based on the first loss value and the second loss value.

7. The audio separation method according to claim 1, characterized in that, The step of inputting the audio to be separated into the deep learning segmentation network to obtain the target separated audio includes: The audio to be separated is input into the deep learning segmentation network, and the audio to be separated is downsampled to obtain the sampled audio to be separated; The sampled audio to be separated is upsampled to obtain the target separated audio.

8. An audio separation device, characterized in that, include: The audio acquisition module is used to acquire sample audio, determine the sample audio spectrum information of the sample audio, encode the sample audio spectrum information, and obtain the sample audio feature vector corresponding to the sample audio. The feature modulation module is used to linearly modulate the sample audio feature vector to obtain a linear feature vector. The feature replacement module is used to input the linear feature vector into the bottleneck layer of the deep learning segmentation network, and replace the initial bottleneck layer feature vector of the bottleneck layer with the linear feature vector to obtain the target bottleneck layer feature vector. The decoding module is used to perform decoding processing based on the feature vector of the target bottleneck layer to obtain sample-separated audio. The network training module is used to determine the target loss value of the deep learning segmentation network based on the audio separation of the samples, and to train the deep learning segmentation network based on the target loss value. The target-separated audio generation module is used to acquire the audio to be separated, input the audio to be separated into the deep learning segmentation network, and obtain the target-separated audio.

9. An electronic device, characterized in that, The device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the audio separation method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a program that is executed by a processor to implement the audio separation method according to any one of claims 1 to 7.

Images