Deep neural network assisted mechanical sound source real-time separation method and system

Abstract

The invention discloses a deep neural network assisted mechanical sound source real-time separation method and a deep neural network assisted mechanical sound source real-time separation system, which are used for separating complex aliasing sound source signals collected by a sensor and distinguishing different devices which operate at the same time aiming at an application scene of intelligent perception of a mine excavation working face. The method comprises the following steps: firstly, carrying out feature extraction based on an MLVDR-TCN network; secondly, a complex ideal ratio mask cIRM is adopted as a training label, and the cIRM is compressed to optimize neural network training; then, the MLVDR-TCN is utilized to predict a real part quantity and an imaginary part quantity respectively, and a mean square error is utilized to optimize network parameters; and finally, collecting an aliasing sound source signal of the mechanical equipment, obtaining a prediction result through the trained network, restoring an uncompressed mask, multiplying the uncompressed mask by a complex spectrum of the aliasing sound source signal, and reconstructing an original sound source signal of each mechanical equipment. According to the method, the mechanical aliasing sound source signals can be accurately and effectively separated in real time in practical application.

Description

Technical Field

[0001] The present invention relates to a sound source separation technology, and in particular to a deep neural network-assisted real-time separation method and system for mechanical sound sources under a mining working face. Background Art

[0002] The separation technology of complex aliasing sound sources in the fifth generation mobile communication (5G) smart mine excavation face is one of the current research hotspots in the mining industry, and provides important technical support for improving the efficiency and safety of mining operations. In addition, the 5G smart mine excavation face uses the high-speed transmission and low-latency characteristics of the 5G network to achieve real-time data transmission and remote control. By arranging sound sensors and monitoring equipment on the mining face, the operating status of the equipment can be monitored in real time, and the data can be transmitted to the remote monitoring center through the 5G network. Mining technicians can monitor and operate the working face in real time through high-definition video and virtual reality technology.

[0003] In mining operations, there are complex aliasing sounds from multiple sound sources, such as mechanical operation sounds, blasting sounds, and worker communication sounds. These sounds are superimposed on each other, which brings challenges to the separation and identification of sounds. At present, there are only a few studies on complex aliasing sound source separation technology based on deep learning in the context of mines. Most traditional complex aliasing sound source separation methods are based on signal processing related technologies, including spectral subtraction, Wiener filtering, and minimum mean square error estimation. These methods need to meet corresponding constraints when modeling signals, such as the noise is additive noise, the noise is relatively stable, etc. This results in a significant reduction in the separation effect in actual environments where these assumptions cannot be met.

[0004] In the actual mining context, sound sensors are deployed near multiple coal mining equipment. If the collected aliased sound source data is directly transmitted to the working face, the staff will not be able to distinguish the sounds of each device, and thus cannot determine whether the working conditions of each device are normal. Summary of the invention

[0005] Purpose of the invention: In view of the above-mentioned problems existing in the prior art, the purpose of the present invention is to propose a real-time separation method and system for mechanical sound sources assisted by a deep neural network, which can realize accurate and effective separation of mechanical aliasing sound source signals through deep learning technology, thereby improving the safety and production efficiency of the mining face.

[0006] Technical solution: To achieve the above-mentioned invention object, the present invention adopts the following technical solution:

[0007] In a first aspect, the present invention provides a real-time separation method of mechanical sound sources assisted by a deep neural network, comprising the following steps:

[0008] (1) A multi-layer variable dilation rate temporal convolutional network (MLVDR-TCN) is used to extract features from the complex spectrum of the aliased sound source signal; the MLVDR-TCN improves the time domain convolutional network TCN by first stacking a plurality of dilated convolution blocks with different dilation coefficients, then stacking the plurality of dilated convolution blocks multiple times, and then replacing the neurons of the hidden layer of the TCN with the stacked dilated convolution blocks;

[0009] (2) Complex Ideal Ratio Mask (cIRM) is used as the training label, and the hyperbolic tangent function is used to compress cIRM to optimize the MLVDR-TCN network training;

[0010] (3) Two MLVDR-TCN networks with the same structure are used to predict the real and imaginary components of cIRM respectively, and the mean square error is used to optimize the network parameters;

[0011] (4) The aliased sound source signals of mechanical equipment are collected, short-time Fourier transform is performed to obtain the complex spectrum, and the complex spectrum is input into the trained MLVDR-TCN network to obtain the predicted compressed mask and restore the uncompressed mask through the inverse hyperbolic tangent function. The mask is then multiplied with the complex spectrum of the aliased sound source signal to reconstruct the original sound source signals of each mechanical equipment.

[0012] Furthermore, in the MLVDR-TCN, the dilation coefficient of the dilated convolutional block increases exponentially with the number of layers, and the dilation factors are 1, 2, …, 2 respectively. X-1 The dilated convolution blocks of are arranged in sequence, where X is the number of dilated convolution blocks; the input data is operated in each dilated convolution block in sequence, and the residual path and skip connection path are used in each dilated convolution block; the residual path output of a dilated convolution block is used as the input of the next dilated convolution block, and the sum of the skip connection paths of all dilated convolution blocks is used as the output of MLVDR-TCN after a ReLU function, a one-dimensional convolution calculation and a Sigmoid function.

[0013] Furthermore, the expression of the cIRM is:

[0014]

[0015] Among them, M r 、M i are the real and imaginary components of cIRM, Y r , Y iare the real and imaginary components of the complex spectrum of the mixed sound source with noise, S r , S i are the real and imaginary components of the complex spectrum of a pure sound source, respectively, and i marks the imaginary part.

[0016] Furthermore, the cIRM is expressed as:

[0017]

[0018]

[0019] Among them, cIRM r and cIRM i are the real and imaginary components of the compressed mask, K is used to control the mask value range, and C is used to control the steepness.

[0020] Furthermore, in the MLVDR-TCN, the convolution operation adopts a depth-separable convolution operation.

[0021] Furthermore, the loss functions for training the two MLVDR-TCN networks are expressed as:

[0022]

[0023]

[0024] Where N represents the number of sound sources, M n,r 、M n,i They represent the real and imaginary components of the theoretically calculated cIRM of the nth sound source, denote the neural network estimates of the real and imaginary components of the nth sound source cIRM, L r , L i represent the loss of real and imaginary parts respectively.

[0025] Furthermore, during the sound source reconstruction process, the uncompressed mask is restored using the following formula:

[0026]

[0027]

[0028] Among them, U r , U i They represent the compressed real and imaginary components of the MLVDR-TCN output, respectively. They represent the real and imaginary parts of the uncompressed cIRM respectively.

[0029] In a second aspect, the present invention provides a deep neural network-assisted real-time mechanical sound source separation system, comprising:

[0030] A feature extraction module is used to extract features from the complex spectrum of the aliased sound source signal using a multi-layer variable dilation coefficient time domain convolutional network MLVDR-TCN; the MLVDR-TCN improves the time domain convolutional network TCN by first stacking a plurality of dilated convolutional blocks using different dilation coefficients, then stacking the plurality of dilated convolutional blocks multiple times, and then replacing the neurons of the TCN hidden layer with the stacked dilated convolutional blocks;

[0031] A network training module is used to use the complex ideal ratio mask cIRM as a training label and compress cIRM using a hyperbolic tangent function to optimize the MLVDR-TCN network training; two MLVDR-TCN networks with the same structure are used to predict the real and imaginary components of cIRM respectively, and the mean square error is used to optimize the network parameters;

[0032] And the sound source reconstruction module is used to collect the aliased sound source signals of mechanical equipment, perform short-time Fourier transform to obtain the complex spectrum, input it into the trained MLVDR-TCN network, obtain the predicted compressed mask and restore the uncompressed mask through the inverse hyperbolic tangent function, and then multiply it with the complex spectrum of the aliased sound source signal to reconstruct the original sound source signal of each mechanical equipment.

[0033] In a third aspect, the present invention provides a computer system comprising a memory, a processor, and a computer program / instructions stored in the memory and executable on the processor, wherein when the computer program / instructions are executed by the processor, the steps of a deep neural network-assisted real-time mechanical sound source separation method described in the first aspect are implemented.

[0034] In a fourth aspect, the present invention provides a computer program product, comprising a computer program / instruction, which, when executed by a processor, implements the steps of a deep neural network-assisted real-time mechanical sound source separation method described in the first aspect.

[0035] Beneficial effects: The present invention proposes a real-time separation method of mechanical sound sources assisted by a deep neural network for the application scenario of the mining face of a smart mine. The method proposes an improved time domain convolutional network MLVDR-TCN based on the traditional time domain convolutional network to enhance the model's ability to capture signal timing information. First, MLVDR-TCN is used for feature extraction, and the network's receptive field is expanded by stacking hole convolution blocks with different expansion coefficients. Secondly, a complex ideal ratio mask is used as a label for neural network feature extraction, and the hyperbolic tangent is used to compress the cIRM so that the mask value is within a certain range, which is convenient for neural network training. Furthermore, in order to reduce the computational cost and the number of parameters, a deep separable convolution is used to decouple the standard convolution operation into two continuous operations, channel-by-channel convolution and point-by-point convolution, thereby achieving lightweight network. Finally, two MLVDR-TCNs with the same structure are used to predict the real and imaginary components respectively, and the loss functions of the real and imaginary parts are expressed by the mean square error function. In this way, the network parameters are optimized, and the source signal is decompressed and reconstructed after the estimated cIRM is obtained. The present invention achieves a good balance between separation accuracy and stability.

[0036] To sum up, in the application scenario of smart mine excavation working face, for the mechanical aliasing sound source data collected by the sound sensors deployed in the mine, based on the method proposed in the present invention, after the deep learning training is completed, the trained neural network model parameters can be uploaded to the cloud processor, and the aliasing sound source samples can be separated with good stability and accuracy during actual measurement, and the separated sound sources of each mechanical equipment can be transmitted back to the working face, which is convenient for the staff to carry out subsequent work processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is an overall flow chart of an embodiment of the present invention.

[0038] Figure 2 Schematic diagram of the structure of the MLVDR-TCN in an embodiment of the present invention.

[0039] Figure 3 This is a detailed structural design diagram of the hole convolution block in the MLVDR-TCN in an embodiment of the present invention.

[0040] Figure 4 This is a flow chart of network training according to an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0042] The core idea of ​​the present invention is: for the application scenario of smart mine excavation working face, based on the traditional time domain convolutional network, an improved time domain convolutional network-Multi-Layer Variable Dilation Rate Temporal Convolutional Network (MLVDR-TCN) is proposed to enhance the model's ability to capture signal timing information. First, the complex spectrum of the input aliasing signal is extracted by using MLVDR-TCN, and the receptive field of the network is expanded by stacking the hole convolution blocks with different dilation coefficients. Secondly, the complex ideal ratio mask is used as the label of the neural network to extract features, and the hyperbolic tangent compression cIRM is used to make the mask value within a certain range, which is convenient for the training of the neural network. Then, in order to further reduce the computing cost and the number of parameters, the depth separable convolution is used to decouple the standard convolution operation into two continuous operations of channel-by-channel convolution and point-by-point convolution, so as to achieve the lightweight of the network. Finally, the real and imaginary components are predicted using MLVDR-TCN, and the loss functions of the real and imaginary components are expressed by the mean square error function. In this way, the network parameters are optimized, and the source signal is decompressed and reconstructed after the estimated cIRM is obtained. The present invention is further described in detail below.

[0043] like Figure 1 As shown, a deep neural network-assisted real-time separation method for mechanical sound sources disclosed in an embodiment of the present invention mainly includes the following steps:

[0044] (1) The complex spectrum of the aliased sound source signal is extracted using the multi-layer variable dilation coefficient time-domain convolutional network (MLVDR-TCN).

[0045] (2) The complex ideal ratio mask cIRM is used as the training label, and the hyperbolic tangent function is used to compress cIRM to optimize the MLVDR-TCN network training.

[0046] (3) Two MLVDR-TCN networks with the same structure are used to predict the real and imaginary components of cIRM respectively, and the mean square error is used to optimize the network parameters.

[0047] (4) The aliased sound source signals of mechanical equipment are collected, and short-time Fourier transform (STFT) is performed to obtain the complex spectrum, which is input into the trained MLVDR-TCN network to obtain the predicted compressed mask and restore the uncompressed mask through the inverse hyperbolic tangent function. The mask is then multiplied with the complex spectrum of the aliased sound source signal to reconstruct the original sound source signals of each mechanical equipment.

[0048] Specifically, in step (1), the complex spectrum of the aliased sound source signal is first obtained (divided into real and imaginary parts, both with dimensions of F×T, where F is the number of frequency points and T is the number of time frames), and the traditional temporal convolutional network (TCN) is improved by using MLVDR-TCN. First, multiple dilated convolution blocks with different expansion coefficients are stacked, and then the multiple dilated convolution blocks are stacked multiple times, and then the neurons of the TCN hidden layer are replaced with the stacked dilated convolution blocks. The input of each block is padded with zeros to ensure that the output length is the same as the input.

[0049] The traditional time domain convolutional network mainly consists of causal convolution, dilated convolution and residual block. The specific definition of causal convolution is that, assuming the sequence Q = (q1, q2, ..., q E ), filter W=(w1,w2,...,w J ), E is the sequence length, J is the length of the filter, in q e The causal convolution at is:

[0050]

[0051] The value of the next layer at time e in TCN only depends on the value of the previous layer at time e and before. Its structure is unidirectional and cannot see future data. It is a strict time constraint model.

[0052] The dilated convolution means that, assuming the input sequence Q = (q1, q2, ..., q E ), filter W=(w1,w2,...,w J ), in q e When the expansion coefficient is d, the formula is expressed as:

[0053]

[0054] Since the dilated convolution allows for spaced sampling of the input during convolution, the sampling rate is controlled by d. Generally speaking, the higher the layer, the larger the d value used, so that the size of the effective window grows exponentially with the number of layers, so that a large receptive field can be obtained with fewer layers.

[0055] The residual block contains two layers of convolution and nonlinear mapping, and weight normalization and dropout are added to each convolution layer to regularize the network.

[0056] When extracting features from complex mixed sound source signals with huge data volumes, traditional time-domain convolutional networks are unable to fully capture the long-term dependency features of signals due to their limited receptive field. To solve this problem, an innovative improvement was made to the time-domain convolutional network, and a multi-layer variable dilation coefficient time-domain convolutional network (MLVDR-TCN) was proposed as a feature extraction network.

[0057] First, dilated convolution blocks with different dilation coefficients d are stacked, such as Figure 2 As shown in FIG, the first X dilated convolution blocks (X is preferably 7 or 8) are copied and stacked multiple times (such as 2 to 4 times), and then the neurons of the traditional TCN hidden layer are replaced with the stacked dilated convolution blocks (the specific design of the dilated convolution blocks is shown in FIG. Figure 3 As shown in the figure, the receptive field is expanded, which effectively enhances the model’s ability to capture the timing information of the signal, and enables the network to better understand and process long-distance dependencies, which is especially important for feature extraction of aliased sound source signals.

[0058] Specifically, the dilation coefficient of the dilated convolution block increases exponentially with the number of layers, and the X dilation factors are 1, 2, …, 2 X-1 The hole convolution blocks are arranged in sequence, the input data is operated in each hole convolution block in turn, and the residual path ( Figure 2 The thick solid line in the figure) and the skip connection path ( Figure 2 The residual path output of a dilated convolution block is used as the input of the next dilated convolution block, and the sum of the skip connection paths of all blocks is further processed by a ReLU function, a one-dimensional convolution calculation, and a Sigmoid function as the output of MLVDR-TCN.

[0059] In step (2), the complex ideal ratio mask (cIRM) is used as the label for the neural network to extract features. cIRM considers both the amplitude spectrum and phase spectrum of the sound source signal, and enhances the amplitude and phase information of the pure sound source signal by complex multiplication. Assume that S(t,f) and Y(t,f) are the complex spectra of the pure sound source and the noisy mixed sound source, respectively, t is time, f is frequency, M(t,f) is cIRM, and * is complex multiplication, then:

[0060] S(t,f)=M(t,f)*Y(t,f)

[0061] Write S(t,f), Y(t,f), and M(t,f) in the form of real and imaginary parts. The real part represents the amplitude of the cosine wave component of the signal at the corresponding frequency, and the imaginary part represents the amplitude of the sine wave component of the signal at the corresponding frequency:

[0062]

[0063] Then we have:

[0064] S r +iS i =(M r +iM i )*(Y r +iY i )=(M r Y r -M i Y i )+i(M r Y i +M i Y r )

[0065] Solving the equation yields:

[0066]

[0067] The expression of cIRM can be obtained:

[0068]

[0069] Since the complex mask may have large real and imaginary components, whose values ​​are in the range of (-∞,∞), the hyperbolic tangent function is used to compress cIRM, as shown in the following formula:

[0070]

[0071] Where x is r or i, indicating the real component and the imaginary component. K is used to control the range of the mask value, that is, the mask value generated by compression is within [-K, K], and C is used to control the steepness. In this embodiment, several values ​​of K and C are evaluated, and K = 10 and C = 0.1 perform best empirically and are used to train the neural network.

[0072] In order to further reduce the computational cost and the number of parameters, in this embodiment, a depthwise separable convolution is used. When extracting feature maps, the standard convolution operation is decoupled into two consecutive operations, namely, channel-by-channel convolution (Depthwise Convolution, DW) and the subsequent pointwise convolution (Pointwise Convolution, PW).

[0073] In channel-by-channel convolution (denoted as D-conv(·)), each convolution kernel is responsible for one channel, and one channel is convolved by only one convolution kernel. The number of feature maps generated by this process is exactly the same as the number of input channels. The mathematical expression is:

[0074]

[0075] Among them, A∈R M×H is the input vector of the depthwise separable convolution, M and H are the two dimensions of the input matrix A, B∈R M×P is a convolution kernel of size P (value 3). j ∈R 1×H and b j ∈R 1×P are the rows of matrices A and B respectively. Denotes convolution. The D-conv(·) operation convolves each row of the input A with the corresponding row of the matrix B. At the beginning, M is equal to F.

[0076] The operation of point-by-point convolution is very similar to that of regular convolution, but the size of its convolution kernel is 1×1×I, where I is the number of channels in the previous layer. Point-by-point convolution performs a weighted combination of the feature maps generated in the previous step in the depth direction to generate a new feature map, and the number of output feature maps is equal to the number of convolution kernels.

[0077] Point-by-point convolution performs a linear transformation on the result of D-conv(·), so the mathematical expression of the depth-separable convolution operation is:

[0078]

[0079] Among them, L∈R M×V×1 is a convolution kernel of size 1. V is the number of channels of the convolution block.

[0080] In step (3), two MLVDR-TCN networks with the same structure are used to predict the real and imaginary components respectively. The loss functions of the real and imaginary parts can be expressed by mean square error functions respectively:

[0081]

[0082]

[0083] Where N represents the number of sound sources, M n,r 、M n,i They represent the real and imaginary components of the theoretically calculated cIRM of the nth sound source, denote the neural network estimates of the real and imaginary components of the nth sound source cIRM, L r , L i represent the loss of real and imaginary parts respectively.

[0084] The network training process is as follows Figure 4 The trained neural network parameters will be uploaded to the cloud server. In subsequent practical applications, no training is required. Only the data collected by the sensor needs to be transmitted to the cloud neural network for processing.

[0085] In step (4), during the testing phase, the sound sensors deployed in the mine collect the actual mixed sound source signals of various mechanical equipment, transmit them to the cloud processor, and then perform short-time Fourier transform to obtain the complex spectrum, which is then input into the neural network. Assume that the output of the neural network is U x , use the following inverse function to recover the uncompressed mask:

[0086]

[0087] After obtaining the uncompressed mask, it is multiplied by the complex spectrum of the aliased sound source signal to reconstruct the original sound source signal of each device and then transmitted back to the mine working face.

[0088] Based on the original sound source signals of each device after separation of the aliased sound sources transmitted back to the mine working face, the staff can judge the working status of each mechanical equipment based on experience and carry out subsequent work.

[0089] According to the description of this embodiment, it should not be difficult for those skilled in the art to see that the real-time separation method of mechanical sound sources under the mining face assisted by a deep neural network proposed in the present invention can effectively reduce the time delay of the separation process and achieve a good balance between the accuracy and stability of the separation. The contents not described in detail in the present invention belong to the prior art known to professional and technical personnel in the field.

[0090] Based on the same inventive concept, an embodiment of the present invention also discloses a deep neural network-assisted real-time mechanical sound source separation system, including: a feature extraction module, used to use a multi-layer variable dilation coefficient time-domain convolutional network MLVDR-TCN to extract features from the complex spectrum of the aliased sound source signal; the MLVDR-TCN improves the time-domain convolutional network TCN, first stacking a plurality of dilated convolution blocks with different dilation coefficients, then stacking the plurality of dilated convolution blocks multiple times, and then replacing the neurons of the TCN hidden layer with the stacked dilated convolution blocks; a network training module, used to use a complex ideal ratio mask cIRM as The invention provides a training label and uses a hyperbolic tangent function to compress cIRM to optimize the MLVDR-TCN network training; two MLVDR-TCN networks with the same structure are used to predict the real and imaginary components of cIRM respectively, and the mean square error is used to optimize the network parameters; and a sound source reconstruction module is used to collect the aliased sound source signals of mechanical equipment, perform short-time Fourier transform to obtain the complex spectrum, input it into the trained MLVDR-TCN network, obtain the predicted compressed mask and restore the uncompressed mask through the inverse hyperbolic tangent function, and then multiply it with the complex spectrum of the aliased sound source signal to reconstruct the original sound source signal of each mechanical equipment. The specific implementation details of each module in the system embodiment refer to the above method embodiment and will not be repeated here.

[0091] An embodiment of the present invention also discloses a computer system, including a memory, a processor, and a computer program / instructions stored in the memory and executable on the processor, wherein the computer program / instructions, when executed by the processor, implement the steps of the aforementioned deep neural network-assisted real-time separation method for mechanical sound sources.

[0092] An embodiment of the present invention also discloses a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the aforementioned deep neural network-assisted real-time separation method of mechanical sound sources.

Claims

1. A deep neural network-assisted real-time separation method for mechanical sound sources, characterized in that: The steps include: (1) A multi-layer variable dilation coefficient time-domain convolutional network (MLVDR-TCN) is used to extract features from the complex spectrum of the aliased sound source signal; the MLVDR-TCN improves the time-domain convolutional network (TCN), firstly stacks a plurality of dilated convolutional blocks with different dilation coefficients, then stacks the plurality of dilated convolutional blocks multiple times, and then replaces the neurons of the hidden layer of the TCN with the stacked dilated convolutional blocks; (2) The complex ideal ratio mask cIRM is used as the training label, and the hyperbolic tangent function is used to compress cIRM to optimize the MLVDR-TCN network training; (3) Two MLVDR-TCN networks with the same structure are used to predict the real and imaginary components of cIRM respectively, and the mean square error is used to optimize the network parameters; (4) The aliased sound source signals of mechanical equipment are collected, short-time Fourier transform is performed to obtain the complex spectrum, and the complex spectrum is input into the trained MLVDR-TCN network to obtain the predicted compressed mask and restore the uncompressed mask through the inverse hyperbolic tangent function. The mask is then multiplied with the complex spectrum of the aliased sound source signal to reconstruct the original sound source signals of each mechanical equipment.

2. According to claim 1, a deep neural network-assisted real-time mechanical sound source separation method is characterized in that: In the MLVDR-TCN, the dilation coefficient of the dilated convolutional block increases exponentially with the number of layers, and the dilation factors are 1, 2, …, 2 X-1 The dilated convolution blocks of are arranged in sequence, where X is the number of dilated convolution blocks; the input data is operated in each dilated convolution block in sequence, and the residual path and skip connection path are used in each dilated convolution block; the residual path output of a dilated convolution block is used as the input of the next dilated convolution block, and the sum of the skip connection paths of all dilated convolution blocks is used as the output of MLVDR-TCN after a ReLU function, a one-dimensional convolution calculation and a Sigmoid function.

3. The method for real-time separation of mechanical sound sources assisted by a deep neural network according to claim 1, characterized in that: The expression of cIRM is: Among them, M r 、M i are the real and imaginary components of cIRM, Y r , Y i are the real and imaginary components of the complex spectrum of the mixed sound source with noise, S r , S i are the real and imaginary components of the complex spectrum of a pure sound source, respectively, and i marks the imaginary part.

4. The method for real-time separation of mechanical sound sources assisted by a deep neural network according to claim 3, characterized in that: The cIRM is expressed as: Among them, cIRM r and cIRM i are the real and imaginary components of the compressed mask, K is used to control the mask value range, and C is used to control the steepness.

5. The method for real-time separation of mechanical sound sources assisted by a deep neural network according to claim 1, characterized in that: In the MLVDR-TCN, the convolution operation adopts a depth-wise separable convolution operation.

6. The method for real-time separation of mechanical sound sources assisted by a deep neural network according to claim 1, characterized in that: The loss functions for training the two MLVDR-TCN networks are expressed as: Where N represents the number of sound sources, M n,r 、M n,i They represent the real and imaginary components of the theoretically calculated cIRM of the nth sound source, denote the neural network estimates of the real and imaginary components of the nth sound source cIRM, L r , L i represent the loss of real and imaginary parts respectively.

7. The method for real-time separation of mechanical sound sources assisted by a deep neural network according to claim 1, characterized in that: During the sound source reconstruction process, the uncompressed mask is recovered using the following formula: Among them, U r , U i They represent the compressed real and imaginary components of the MLVDR-TCN output, respectively. They represent the real and imaginary components of the uncompressed cIRM respectively, K is used to control the mask value range, and C is used to control the steepness.

8. A deep neural network-assisted real-time mechanical sound source separation system, characterized in that: include: A feature extraction module is used to extract features of the complex spectrum of the aliased sound source signal using a multi-layer variable dilation coefficient time domain convolutional network MLVDR-TCN; The MLVDR-TCN improves the time domain convolutional network TCN by first stacking a plurality of dilated convolution blocks with different expansion coefficients, then stacking the plurality of dilated convolution blocks multiple times, and then replacing the neurons of the TCN hidden layer with the stacked dilated convolution blocks; A network training module is used to use the complex ideal ratio mask cIRM as a training label and compress cIRM using a hyperbolic tangent function to optimize the MLVDR-TCN network training; two MLVDR-TCN networks with the same structure are used to predict the real and imaginary components of cIRM respectively, and the mean square error is used to optimize the network parameters; And the sound source reconstruction module is used to collect the aliased sound source signals of mechanical equipment, perform short-time Fourier transform to obtain the complex spectrum, input it into the trained MLVDR-TCN network, obtain the predicted compressed mask and restore the uncompressed mask through the inverse hyperbolic tangent function, and then multiply it with the complex spectrum of the aliased sound source signal to reconstruct the original sound source signal of each mechanical equipment.

9. A computer system comprising a memory, a processor, and a computer program / instruction stored in the memory and executable on the processor, characterized in that: When the computer program / instructions are executed by the processor, the steps of a deep neural network-assisted real-time mechanical sound source separation method according to any one of claims 1 to 7 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by the processor, the steps of a deep neural network-assisted real-time mechanical sound source separation method according to any one of claims 1 to 7 are implemented.

Images