Speech separation method and system combining acoustic features and state space model

Abstract

The present invention relates to the field of speech separation technology, and more particularly to a speech separation method and system combining acoustic features and a state space model. In order to achieve speech separation, the present invention provides a speech separation model, which firstly performs speech separation on a mixed speech by an audio coding unit. Voice mix Encode to get N Audio features of different resolutions { F 1~ F N}, then the acoustic feature extraction unit is based on { F 1~ F N}Extract multi-scale speech features G' m , and then through the state space network G' m Perform long-term modeling to obtain long-term features I T,out , and finally through the audio decoding unit combined with { F 1~ F N}and I T,out Decode and reconstruct { Voice 1,…, Voice N The present invention solves the problem that the existing speech separation method has modeling limitations when facing long sequence audio, which affects the speech separation performance.

Description

Technical Field

[0001] The present invention relates to the field of speech separation technology, and more specifically, to: 1. a speech separation method combining acoustic features and a state space model; 2. a speech separation system combining acoustic features and a state space model. Background Art

[0002] Speech separation refers to the use of specific technical means to extract separate speech components from the received mixed speech signal. The core goal of this technology is to identify and separate specific speech signals in a speech environment with multiple sound sources, aiming to improve the clarity of human and machine speech recognition. Speech separation has shown its effectiveness in many practical application scenarios, such as playing an important role in hearing aids, noise-canceling headphones, and smart speakers.

[0003] Previous speech separation methods have limited capabilities in modeling long-sequence audio and high computational complexity, which restricts their practical applications. Existing methods emphasize the importance of multi-scale feature representation in effective audio modeling; however, due to the homogeneity of semantic content of long-sequence audio features, actual multi-scale features often face limitations in acoustic modeling, which affects the performance of speech separation. Summary of the invention

[0004] Based on this, it is necessary to provide a speech separation method and system that combines acoustic features and state space models to address the problem that existing speech separation methods have modeling limitations when facing long sequence audio, which affects the speech separation performance.

[0005] The present invention is implemented by the following technical solutions:

[0006] In a first aspect, the present invention discloses a speech separation method combining acoustic features and a state space model, comprising:

[0007] Step 1: Get mixed audio Voice mix ;

[0008] Step 2: Use the trained speech separation model to Voice mix Process and obtain N The separated speech Voice 1,…, Voice N}; N >1.

[0009] The speech separation model includes: an audio encoding unit, an acoustic feature extraction unit, a state space network unit, and an audio decoding unit.

[0010] The audio coding section is used to Voicemix Encode to get N Audio features of different resolutions { F 1~ F N}.

[0011] The acoustic feature extraction unit is used based on { F 1~ F N}Extract multi-scale speech features G' m .

[0012] The state-space network is used to G' m Perform long-term modeling to obtain long-term features I T,out .

[0013] The audio decoding unit is used to combine { F 1~ F N}and I T,out Decode and reconstruct { Voice 1,…, Voice N}.

[0014] This speech separation method combining acoustic features and state-space models implements the method or process according to an embodiment of the present disclosure.

[0015] In a second aspect, the present invention discloses a speech separation system combining acoustic features and a state-space model, which uses the speech separation method combining acoustic features and a state-space model disclosed in the first aspect.

[0016] The speech separation system combining acoustic features and state space model includes: a speech acquisition module and a speech separation module.

[0017] The voice acquisition module is used to obtain mixed voice Voice mix .

[0018] The speech separation module is used to use the trained speech separation model to Voice mix Process and obtain N The separated speech Voice 1,…, Voice N}; N >1.

[0019] The speech separation system combining acoustic features and state-space models implements the method or process according to the embodiments of the present disclosure.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] The present invention provides a speech separation model with a novel structural design to achieve speech separation. The first step is to separate the mixed speech by an audio coding unit. Voice mix Encode to get N Audio features of different resolutions { F 1~ F N}, then the acoustic feature extraction unit is based on { F 1~ F N}Extract multi-scale speech features G' m To provide rich encoding information, and then through the state space network G' m Perform long-term modeling to obtain long-term features I T,out Thereby reducing semantic redundancy and enhancing the long-term audio modeling effect, and finally combining { F 1~ F N}and I T,out Decode and reconstruct { Voice 1,…, Voice N}; It can be seen from simulation verification that the method of the present invention is effective and has certain advantages in performance indicators compared with the existing methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0023] Figure 1 A data flow diagram of a speech separation method combining acoustic features and a state space model provided in Example 1 of the present invention;

[0024] Figure 2 for Figure 1 Data flow diagram of the audio coding unit;

[0025] Figure 3 for Figure 1 Data flow diagram of the acoustic feature extraction unit;

[0026] Figure 4 for Figure 1Data flow diagram when the state space network department uses the IndBiMamba layer design;

[0027] Figure 5 for Figure 1 Data flow diagram when the state space network department adopts DouBiMamba layer design;

[0028] Figure 6 for Figure 4 , Figure 5 Data flow diagram of the Backward-SSM layer;

[0029] Figure 7 for Figure 1 Data flow diagram of the audio decoding unit. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] It should be noted that when a component is referred to as being "mounted on" another component, it may be directly on the other component or there may be a central component. When a component is considered to be "set on" another component, it may be directly set on the other component or there may be a central component at the same time. When a component is considered to be "fixed to" another component, it may be directly fixed on the other component or there may be a central component at the same time.

[0032] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "or / and" used herein includes any and all combinations of one or more of the related listed items.

[0033] Example 1

[0034] See also Figure 1 , Figure 1 A data flow diagram of a speech separation method combining acoustic features and a state space model is shown, and a flowchart of the speech separation method combining acoustic features and a state space model is also shown, which includes:

[0035] Step 1: Get mixed audio Voice mix .

[0036] Mixed Voice Voice mix Included N The present invention aims to separate the speech information mixed from multiple sound sources.

[0037] Step 2: Use the trained speech separation model to Voice mix Process and obtain N The separated speech Voice 1,…, Voice N}; N >1.

[0038] The core of the present invention is to provide a speech separation model with a new structure, which can be divided into four parts according to its functions: ① audio encoding part, ② acoustic feature extraction part, ③ state space network part, and ④ audio decoding part.

[0039] The following explains them one by one:

[0040] ① The audio encoding unit is used to Voice mix Encode to get N Audio features of different resolutions { F 1~ F N}.

[0041] See also Figure 2 , the audio encoding unit can be designed to include: 1 multi-dimensional convolutional layer.

[0042] In the audio encoding section:

[0043] The input of the multi-dimensional convolutional layer is Voice mix , the output is { F 1~ F N}.

[0044] That is, the multi-dimensional convolutional layer takes the input Voice mix Perform convolution in multiple dimensions to achieve encoding, thus obtaining { F 1~ F N}.

[0045] The processing of the audio encoding unit can be expressed as follows:

[0046] { F 1~ F N = Encoder ( Voice mix);

[0047] In the formula, Encoder (.) represents the multi-dimensional convolutional layer processing process.

[0048] ② The acoustic feature extraction unit is used based on { F 1~ F N}Extract multi-scale speech features G' m .

[0049] See also Figure 3 The acoustic feature extraction unit can be designed to include: 1 upsampling layer, 1 average pooling layer, 1 stacking layer, 1 linear self-attention layer, 1 multi-layer perceptron layer, and 1 normalization layer.

[0050] In the acoustic feature extraction section:

[0051] The upsampling layer is used to F i ~ F 2Upsampling to F 1 has the same time dimension;

[0052] The average pooling layer is used to perform average pooling on the output of the upsampling layer;

[0053] The stacking layer is used to stack the output of the average pooling layer to obtain integrated speech features. G m ;

[0054] The linear self-attention layer is used to G m Apply linear attention to extract key information;

[0055] The multi-layer perceptron layer is used to perform multi-layer perceptual processing on the output of the linear self-attention layer;

[0056] The normalization layer is used to normalize the output of the multi-layer perceptron layer to obtain G' m .

[0057] The processing process of the acoustic feature extraction unit can be expressed as follows:

[0058] ;

[0059] G' m = Φ Norm ( MLP ( Attention ( G m )));

[0060] In the formula, Represents the average pooling layer processing process; up (.) indicates the upsampling layer processing process; Φ Norm (.) indicates the normalization layer processing; MLP (.) represents the multi-layer perceptron layer processing process; Attention (.) represents the linear self-attention layer processing process.

[0061] ③ The state space network is used to G' m Perform long-term modeling to obtain long-term features I T,out .

[0062] It should be noted that the state space network department can adopt the following two structural designs:

[0063] like Figure 4 The first structure based on IndBiMamba layer design is shown in FIG. Figure 5 The second structure based on DouBiMamba layer design is shown.

[0064] 301, for the first structure, it includes: M IndBiMamba layers; M≥1.

[0065] Among them, the output of the previous IndBiMamba layer is used as the input of the next IndBiMamba layer.

[0066] See also Figure 4 , The IndBiMamba layer includes: 2 linear layers, 1 inversion layer, 1 activation function layer, 2 Backward-SSM layers, 2 product layers, and 1 superposition layer.

[0067] Then, in any IndBiMamba layer:

[0068] The first linear layer is used as the input to the IndBiMamba layer Input Perform linear projection to obtain intermediate features I T ;

[0069] The inversion layer is used to I T Reverse the sequence order to obtain the intermediate features I' T ;

[0070] The activation function layer is used to adjust the intermediate features I T Transform to get intermediate features Id ;

[0071] The first Backward-SSM layer is used to I T Perform long-term modeling to obtain intermediate features I T,f ;

[0072] The second Backward-SSM layer is used to I' T Perform long-term modeling to obtain intermediate features I T,b ;

[0073] The first product layer is used to transform I T,f , I d Perform product processing;

[0074] The second product layer is used to I T,b , I d Perform product processing;

[0075] The superposition layer is used to superimpose the output of the first product layer and the output of the second product layer;

[0076] The second linear layer is used to perform linear projection on the output of the superposition layer to obtain the output of the IndBiMamba layer. Output ;

[0077] Among them, the input of the first IndBiMamba layer Input for G' m ;No. M The output of the IndBiMamba layer Output for I T,out .

[0078] The processing process of the first structure can be expressed as follows:

[0079] I T = Φ L ( Input );

[0080] I' T = Reverse ( I T );

[0081] Id = σ ( I T );

[0082] I T,f =Backward-SSM( I T );

[0083] I T,b =Backward-SSM( I' T );

[0084] ;

[0085] In the formula, Φ L ( . ) represents the processing of the linear layer; Reverse ( . ) represents the processing of the inversion layer; σ (.) represents the processing process of the activation function layer; Backward-SSM(.) represents the processing process of the Backward-SSM layer.

[0086] It should be noted that, see Figure 6 ,The Backward-SSM layer in the first structure includes: 1 depth convolution layer, 1 SSM layer, and 1 RMS normalization layer.

[0087] Then, in any Backward-SSM layer:

[0088] The deep convolutional layer is used as the input to the Backward-SSM layer. In Perform depthwise convolution;

[0089] The SSM layer is used to model the long-term features of the output of the deep convolutional layer;

[0090] The RMS normalization layer is used to perform RMS normalization on the output of the SSM layer to obtain the output of the Backward-SSM layer. Out .

[0091] The processing of the Backward-SSM layer can be expressed as follows:

[0092] Out = Φ RMS-Norm ( Φ SSM ( Φ conv ( In )))

[0093] In the formula, Φ conv (.) represents the deep convolutional layer processing process; Φ SSM (.) indicates the SSM layer processing;

[0094] Φ RMS-Norm (.) indicates the RMS normalization layer processing process.

[0095] 302, for the second structure, it includes: M DouBiMamba layers; M≥1.

[0096] Among them, the output of the previous DouBiMamba layer serves as the input of the next DouBiMamba layer.

[0097] Then, in any DouBiMamba layer:

[0098] The first linear layer is used as the input to the DouBiMamba layer Input Perform linear projection to obtain intermediate features I T ;

[0099] The inversion layer is used to I T Reverse the sequence order to obtain the intermediate features I' T ;

[0100] The first activation function layer is used to I T Transform to get intermediate features I d,f ;

[0101] The second activation function layer is used for the intermediate features I' T Transform to get intermediate features I d,b ;

[0102] The first Backward-SSM layer is used to I T Perform long-term modeling to obtain intermediate features I T,f ;

[0103] The second Backward-SSM layer is used to I' T Perform long-term modeling to obtain intermediate features I T,b;

[0104] The first product layer is used to transform I T,f , I d,f Perform product processing;

[0105] The second product layer is used to I T,b , I d,b Perform product processing;

[0106] The second linear layer is used to perform linear projection on the output of the first product layer;

[0107] The third linear layer is used to perform linear projection on the output of the second product layer;

[0108] The superposition layer is used to superimpose the output of the second linear layer and the output of the third linear layer to obtain the output of the DouBiMamba layer. Output ;

[0109] Among them, the input of the first DouBiMamba layer Input for G' m ;No. M The output of the DouBiMamba layer Output for I T,out .

[0110] The processing of the second structure can be expressed as follows:

[0111] I T = Φ L ( Input );

[0112] I' T = Reverse ( I T );

[0113] I d,f = σ ( I T );

[0114] I d,b = σ ( I' T );

[0115] I T,f =Backward-SSM( I T );

[0116] I T,b =Backward-SSM( I' T );

[0117] ;

[0118] In the formula, Φ L ( . ) represents the processing of the linear layer; Reverse ( . ) represents the processing of the inversion layer; σ (.) represents the processing process of the activation function layer; Backward-SSM(.) represents the processing process of the Backward-SSM layer.

[0119] It should be noted that the structure of the Backward-SSM layer in the second structure is the same as that of the Backward-SSM layer in the first structure, and will not be repeated here.

[0120] The state space network of the above two structures is adapted to long sequence audio, which can reduce semantic redundancy and enhance the long-term audio modeling effect.

[0121] ④ The audio decoding unit is used to combine { F 1~ F N}and I T,out Decode and reconstruct { Voice 1,…, Voice N}.

[0122] See also Figure 7 , the audio decoding unit can be designed to include: an activation function layer and a bias convolution layer.

[0123] In the audio decoding section:

[0124] The activation function layer is used to I T,out Convert to mask m ;

[0125] The bias convolution layer is used to F n and m Perform Hadamard product processing and convert it into Voice n ;n ∈[1, N ].

[0126] The processing of the above audio decoding unit can be expressed by the formula:

[0127] m = σ ( I T,out );

[0128] Voice n = Decoder ( F n ⊙ m );

[0129] In the formula, σ (.) represents the activation function layer processing; ⊙ represents the Hadamard product processing; Decoder (.) represents the conversion process of the bias convolution layer.

[0130] It should be noted that the present invention requires the use of a trained speech separation model whose network parameters are in an optimal state and has the best speech separation effect.

[0131] Then, the method for obtaining the trained speech separation model (i.e., the training method of the speech separation model) includes:

[0132] S1, obtain a speech sample set with real labels;

[0133] S2, divide the speech sample set with real labels into training set and validation set;

[0134] S3, the speech separation model is tested based on the training set N Rounds of training, verifying the speech separation model after each round of training based on the validation set until the speech separation model with the best performance is selected;

[0135] Among them, the gradient descent method is used to adjust the model parameters during training.

[0136] Simulation Verification

[0137] This embodiment 1 performs simulation verification on the above-mentioned method:

[0138] (1) First, an ablation experiment was conducted to compare the performance of 6 groups of models on the LRS3 and VoxCeleb2 datasets.

[0139] Among them, the 6 groups of models include:

[0140] Model 1: Based on the speech separation model proposed in this embodiment 1, its acoustic feature extraction part is replaced by a common convolutional feature extraction network, and the state space network part is replaced by a multiple convolutional network.

[0141] Model 2: Based on the speech separation model proposed in Example 1, the acoustic feature extraction part is removed and the state space network part (1 IndBiMamba layer) is retained.

[0142] Model 3: Based on the speech separation model proposed in Example 1, the state space network part adopts 1 IndBiMamba layer.

[0143] Model 4: Based on the speech separation model proposed in Example 1, the state space network part adopts 2 IndBiMamba layers.

[0144] Model 5: Based on the speech separation model proposed in Example 1, the state space network part adopts 1 DouBiMamba layer.

[0145] Model 6: Based on the speech separation model proposed in Example 1, the state space network part is obtained using 2 DouBiMamba layers.

[0146] Speech separation indicators (including SI-SNRi and SDRi, the larger the value, the better) and model parameter Params are introduced for evaluation. See Table 1 below for details.

[0147] Table 1. Comparison results of ablation experiments

[0148]

[0149] As can be seen from Table 1, models 3-6 have improved speech separation indicators compared to models 1 and 2, and the number of network model parameters has also decreased significantly compared to model 1, and has a smaller increase compared to model 2, which illustrates the effectiveness of the acoustic feature extraction unit and the state space network unit proposed in Example 1. Among them, model 6 can achieve a better balance between performance and parameter quantity, and is more recommended.

[0150] (2) The performance of model 6 (abbreviated as MSMa-Net) in (1) was tested on the LRS3 and VoxCeleb2 speech datasets respectively, together with the other seven existing models (DPCL, uPIT, Conv-TasNet, SuDORM-RF, A-FRCNN, TDANet, and Mossformer).

[0151] Speech separation indicators (including SI-SNRi and SDRi, the larger the value, the better) and model parameter Params are introduced for evaluation. See Table 2 below for details.

[0152] Table 2 Comparison results of model experiments

[0153]

[0154] It can be seen from Table 2 that MSMa-Net is better than the other 7 groups of models in terms of speech separation indicators, and the number of network model parameters is also at a low level among the 8 groups of models; this also shows that the method proposed in Example 1 has certain advantages.

[0155] Example 2

[0156] This embodiment 2 provides a speech separation system combining acoustic features and state-space models, which uses the speech separation method combining acoustic features and state-space models disclosed in embodiment 1.

[0157] The speech separation system combining acoustic features and state space model includes: a speech acquisition module and a speech separation module.

[0158] The voice acquisition module is used to obtain mixed voice Voice mix .

[0159] The speech separation module is used to use the trained speech separation model to Voice mix Process and obtain N The separated speech Voice 1,…, Voice N}; N >1.

[0160] Among them, the speech separation module includes: an audio encoding submodule (corresponding to the audio encoding unit), an acoustic feature extraction submodule (corresponding to the acoustic feature extraction unit), a state-space network submodule (corresponding to the state-space network unit), and an audio decoding submodule (corresponding to the audio decoding unit), which will not be repeated here.

[0161] Since this system uses the speech separation method combining acoustic features and state space models in Example 1, it also has the same effect and will not be repeated here.

[0162] Example 3

[0163] This embodiment 3 discloses a computer device, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of the speech separation method combining acoustic features and state space models disclosed in embodiment 1 are implemented.

[0164] This embodiment 3 also discloses a readable storage medium, in which computer program instructions are stored. When the computer program instructions are read and executed by a processor, the steps of the speech separation method combining acoustic features and state-space models disclosed in embodiment 1 are executed.

[0165] This embodiment 3 also discloses a computer program product, including a computer program. When the computer program is executed by a processor, the steps of the speech separation method combining acoustic features and state space model disclosed in embodiment 1 are implemented.

[0166] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A speech separation method combining acoustic features and a state space model, comprising: Step 1: Get mixed audio Voice mix ; It is characterized by further comprising: Step 2: Use the trained speech separation model to Voice mix Process and obtain N The separated speech Voice 1,…, Voice N }; N >1; Among them, the speech separation model includes: Audio encoding unit, used for Voice mix Encode to get N Audio features of different resolutions { F 1~ F N }; Acoustic feature extraction unit, for based on { F 1~ F N }Extract multi-scale speech features G' m ; State space network unit, used for G' m Perform long-term modeling to obtain long-term features I T,out ; The audio decoding unit is used to combine { F 1~ F N }and I T,out Decode and reconstruct { Voice 1,…, Voice N }; The state space network part includes: M IndBiMamba layers; M ≥ 1; the output of the previous IndBiMamba layer is used as the input of the next IndBiMamba layer; The IndBiMamba layer includes: 2 linear layers, 1 inversion layer, 1 activation function layer, 2 Backward-SSM layers, 2 product layers, and 1 superposition layer; In any IndBiMamba layer, The first linear layer is the input to the IndBiMamba layer Input Perform linear projection to obtain intermediate features I T ; Invert layer pairs I T Reverse the sequence order to obtain the intermediate features I' T ; Activation function layer for intermediate features I T Transform to get intermediate features I d ; The first Backward-SSM layer pair I T Perform long-term modeling to obtain intermediate features I T,f ; The second Backward-SSM layer pair I' T Perform long-term modeling to obtain intermediate features I T,b ; The first product layer will I T,f , I d Perform product processing; The second product layer will I T,b , I d Perform product processing; The superposition layer superimposes the output of the first product layer and the output of the second product layer; The second linear layer performs linear projection on the output of the superposition layer to obtain the output of the IndBiMamba layer Output ; Input to the first IndBiMamba layer Input for G' m ;No. M The output of the IndBiMamba layer Output for I T,out .

2. A speech separation method combining acoustic features and a state space model, comprising: Step 1: Get mixed audio Voice mix ; It is characterized by further comprising: Step 2: Use the trained speech separation model to Voice mix Process and obtain N The separated speech Voice 1,…, Voice N }; N >1; The speech separation model includes: Audio encoding unit, used for Voice mix Encode to get N Audio features of different resolutions { F 1~ F N }; Acoustic feature extraction unit, for based on { F 1~ F N }Extract multi-scale speech features G' m ; State space network unit, used for G' m Perform long-term modeling to obtain long-term features I T,out ; The audio decoding unit is used to combine { F 1~ F N }and I T,out Decode and reconstruct { Voice 1,…, Voice N }; The state space network part includes: M DouBiMamba layers; M≥1; The output of the previous DouBiMamba layer is used as the input of the next DouBiMamba layer; DouBiMamba layers include: 3 linear layers, 1 inversion layer, 2 activation function layers, 2 Backward-SSM layers, 2 product layers, and 1 superposition layer; In any DouBiMamba layer, The first linear layer is the input of the DouBiMamba layer Input Perform linear projection to obtain intermediate features I T ; Invert layer pairs I T Reverse the sequence order to obtain the intermediate features I' T ; The first activation function layer has an effect on the intermediate features. I T Transform to get intermediate features I d,f ; The second activation function layer has an effect on the intermediate features. I' T Transform to get intermediate features I d,b ; The first Backward-SSM layer pair I T Perform long-term modeling to obtain intermediate features I T,f ; The second Backward-SSM layer pair I' T Perform long-term modeling to obtain intermediate features I T,b ; The first product layer will I T,f , I d,f Perform product processing; The second product layer will I T,b , I d,b Perform product processing; The second linear layer performs linear projection on the output of the first product layer; The third linear layer performs linear projection on the output of the second product layer; The superposition layer superimposes the output of the second linear layer and the output of the third linear layer to obtain the output of the DouBiMamba layer Output ; Input of the first DouBiMamba layer Input for G' m ;No. M The output of the DouBiMamba layer Output for I T,out .

3. The speech separation method combining acoustic features and state space model according to claim 1 or 2, characterized in that: The audio encoding unit includes: 1 multi-dimensional convolution layer; In the audio encoding section: The input of the multi-dimensional convolutional layer is Voice mix , the output is { F 1~ F N }.

4. The speech separation method combining acoustic features and state space model according to claim 1 or 2, characterized in that: The acoustic feature extraction unit comprises: 1 upsampling layer, which is used to F i ~ F 2Upsampling to F 1 has the same time dimension; 1 average pooling layer, which is used to perform average pooling on the output of the upsampling layer; 1 stacking layer, which is used to stack the output of the average pooling layer to obtain the integrated speech features G m ; 1 linear self-attention layer, which is used to G m Apply linear attention to extract key information; 1 multi-layer perceptron layer, which is used to perform multi-layer perceptron processing on the output of the linear self-attention layer; as well as 1 normalization layer, which is used to normalize the output of the multi-layer perceptron layer to obtain G' m .

5. The speech separation method combining acoustic features and state space model according to claim 1 or 2, characterized in that: The Backward-SSM layer includes: 1 deep convolution layer, 1 SSM layer, and 1 RMS normalization layer; In any Backward-SSM layer, The deep convolutional layer is used as the input to the Backward-SSM layer. In Perform depthwise convolution; The SSM layer is used to model the long-term features of the output of the deep convolutional layer; The RMS normalization layer is used to perform RMS normalization on the output of the SSM layer to obtain the output of the Backward-SSM layer. Out .

6. The speech separation method combining acoustic features and state space model according to claim 1 or 2, characterized in that: The audio decoding unit includes: an activation function layer and a bias convolution layer; In the audio decoding section: The activation function layer is used to I T,out Convert to mask m ; The bias convolution layer is used to F n and m Perform Hadamard product processing and convert it into Voice n ; n ∈[1, N ].

7. The speech separation method combining acoustic features and state space model according to claim 1 or 2, characterized in that: Methods for obtaining a trained speech separation model include: S1, obtain a speech sample set with real labels; S2, divide the speech sample set with real labels into training set and validation set; S3, the speech separation model is tested based on the training set N Rounds of training, verifying the speech separation model after each round of training based on the validation set until the speech separation model with the best performance is selected; Among them, the gradient descent method is used to adjust the model parameters during training.

8. A speech separation system combining acoustic features and state space model, characterized in that: It uses the speech separation method combining acoustic features and state space model as described in any one of claims 1-7; The speech separation system combining acoustic features and state space model comprises: The voice acquisition module is used to acquire mixed voice Voice mix ; as well as The speech separation module is used to use the trained speech separation model to Voice mix Process and obtain N The separated speech Voice 1,…, Voice N }; N >1.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the speech separation method combining acoustic features and a state-space model as described in any one of claims 1 to 7 are implemented.

Images