A method and system for real-time separation of mechanical sound sources with deep neural network assistance

Abstract

This invention discloses a deep neural network-assisted real-time separation method and system for mechanical sound sources. Targeting the application scenario of intelligent sensing in mining faces, it separates complex aliased sound source signals collected by sensors, distinguishing between different simultaneously operating devices. The invention first extracts features based on an MLVDR-TCN network; secondly, it uses a complex ideal ratio mask (cIRM) as the training label and compresses the cIRM to optimize neural network training; next, it uses the MLVDR-TCN network to predict the real and imaginary components separately, using mean square error to optimize the network parameters; finally, it collects the aliased sound source signals of the mechanical equipment, obtains the prediction results through the trained network, recovers the uncompressed mask, and multiplies it with the complex spectrum of the aliased sound source signals to reconstruct the original sound source signals of each mechanical device. This invention can achieve real-time, accurate, and effective separation of aliased mechanical sound source signals in practical applications.

Description

Technical Field

[0001] This invention relates to a sound source separation technology, and more particularly to a method and system for real-time separation of mechanical sound sources assisted by a deep neural network in a mining face. Background Technology

[0002] The technology for separating complex aliased sound sources in 5G smart mine working faces is currently a research hotspot in the mining industry, providing crucial technical support for improving the efficiency and safety of mining operations. Furthermore, 5G smart mine working faces leverage the high-speed transmission and low latency characteristics of 5G networks to achieve real-time data transmission and remote control. By deploying sound sensors and monitoring equipment at the working face, the operating status of the equipment can be monitored in real time, and this data can be transmitted to a remote monitoring center via the 5G network. Mining technicians can then monitor and operate the working face in real time using high-definition video and virtual reality technologies.

[0003] In mining operations, complex aliased sounds from multiple sources exist, such as machinery operation noise, blasting sounds, and workers' conversations. These sounds overlap, posing a challenge to sound separation and identification. Currently, only a few studies have been conducted on deep learning-based techniques for separating complex aliased sound sources in mining environments. Traditional methods for separating complex aliased sound sources are mostly based on signal processing techniques, including spectral subtraction, Wiener filtering, and minimum mean square error estimation. These methods require certain constraints to be met when modeling the signal, such as the noise being additive or relatively stationary. This leads to a significant reduction in separation performance in real-world environments where these assumptions cannot be met.

[0004] In a real mining context, sound sensors are deployed near multiple coal mining equipment. If the collected aliased sound source data is directly transmitted to the working face, workers cannot distinguish the sounds of each piece of equipment, and therefore cannot determine whether the operating conditions of each piece of equipment are normal. Summary of the Invention

[0005] Purpose of the invention: In view of the above-mentioned problems in the existing technology, the purpose of this invention is to propose a method and system for real-time separation of mechanical sound sources assisted by deep neural networks. By using deep learning technology, the method can accurately and effectively separate mechanical aliased sound source signals, thereby improving the safety and production efficiency of mining faces.

[0006] Technical solution: To achieve the above-mentioned objectives, the present invention adopts the following technical solution:

[0007] In a first aspect, the present invention provides a deep neural network-assisted real-time separation method for mechanical sound sources, comprising the following steps:

[0008] (1) The complex spectrum of the aliased sound source signal is extracted using a Multi-Layer Variable Dilation Rate Temporal Convolutional Network (MLVDR-TCN). The MLVDR-TCN improves the Temporal Convolutional Network (TCN) by first stacking multiple dilated convolutional blocks with different dilation rates, then stacking the multiple dilated convolutional blocks multiple times, and then replacing the neurons in the hidden layer of the TCN with the stacked dilated convolutional blocks.

[0009] (2) The Complex Ideal Ratio Mask (cIRM) is used as the training label, and the hyperbolic tangent function is used to compress the cIRM to optimize the training of the MLVDR-TCN network.

[0010] (3) Two MLVDR-TCN networks with the same structure are used to predict the real and imaginary components of cIRM respectively, and the network parameters are optimized by mean square error.

[0011] (4) Collect the aliased sound source signals of the mechanical equipment, perform short-time Fourier transform to obtain the complex spectrum, input it into the trained MLVDR-TCN network, obtain the predicted compressed mask, and recover the uncompressed mask through the hyperbolic tangent inverse function. Then multiply it with the complex spectrum of the aliased sound source signals to reconstruct the original sound source signals of each mechanical equipment.

[0012] Furthermore, in the MLVDR-TCN, the dilation factor of the dilated convolutional block increases exponentially with the number of layers, with dilation factors of 1, 2, ..., 2. X-1 The dilated convolutional blocks are arranged sequentially, where X is the number of dilated convolutional blocks. The input data is processed sequentially in each dilated convolutional block, and residual paths and skipped connection paths are applied in each dilated convolutional block. The residual path output of one dilated convolutional block is used as the input of the next dilated convolutional block. The sum of the skipped connection paths of all dilated convolutional blocks is processed by a ReLU function, a one-dimensional convolution calculation, and a Sigmoid function to become the output of MLVDR-TCN.

[0013] Furthermore, the expression for cIRM is:

[0014]

[0015] Among them, M r M i These are the real and imaginary components of the cIRM, Y. r Y iThese are the real and imaginary components of the complex spectrum of a noisy mixed sound source, S. r S i These are the real and imaginary components of the complex spectrum of a pure sound source, respectively, with i marking the imaginary part.

[0016] Furthermore, the cIRM compressed using the hyperbolic tangent function is expressed as:

[0017]

[0018]

[0019] Among them, cIRM r and cIRM i These are the real and imaginary components of the compressed mask. K controls the range of mask values, and C controls the steepness.

[0020] Furthermore, in the MLVDR-TCN, the convolution operation adopts depth-separable convolution operation.

[0021] Furthermore, the loss functions for training the two MLVDR-TCN networks are respectively expressed as:

[0022]

[0023]

[0024] Where N represents the number of sound sources, M n,r M n,i Let represent the real and imaginary components of the cIRM calculated theoretically for the nth sound source, respectively. Let L represent the neural network estimates of the real and imaginary components of the cIRM for the nth sound source, respectively. r L i These represent the losses of the real and imaginary parts, respectively.

[0025] Furthermore, during the sound source reconstruction process, the uncompressed mask is recovered using the following formula:

[0026]

[0027]

[0028] Among them, U r U i These represent the real and imaginary components of the compressed output of MLVDR-TCN, respectively. These represent the real and imaginary components of the uncompressed cIRM, respectively.

[0029] Secondly, the present invention provides a deep neural network-assisted real-time separation system for mechanical sound sources, comprising:

[0030] The feature extraction module is used to extract features from the complex spectrum of the aliased sound source signal using a multilayer variable dilation coefficient temporal convolutional network MLVDR-TCN. The MLVDR-TCN improves the temporal convolutional network TCN by first stacking multiple dilated convolutional blocks with different dilation coefficients, then stacking the multiple dilated convolutional blocks multiple times, and then replacing the neurons of the TCN hidden layer with the stacked dilated convolutional blocks.

[0031] The network training module is used to use the complex ideal ratio mask cIRM as the training label and to compress the cIRM using the hyperbolic tangent function to optimize the training of the MLVDR-TCN network. Two MLVDR-TCN networks with the same structure are used to predict the real and imaginary components of the cIRM respectively, and the network parameters are optimized using mean square error.

[0032] The system also includes a sound source reconstruction module, which is used to collect aliased sound source signals from mechanical equipment, perform short-time Fourier transform to obtain complex spectra, input them into a trained MLVDR-TCN network, obtain a predicted compressed mask, recover the uncompressed mask through the inverse hyperbolic tangent function, and multiply it with the complex spectrum of the aliased sound source signals to reconstruct the original sound source signals of each mechanical equipment.

[0033] Thirdly, the present invention provides a computer system including a memory, a processor, and a computer program / instructions stored in the memory and executable on the processor, wherein the computer program / instructions, when executed by the processor, implement the steps of the deep neural network-assisted real-time separation method for mechanical sound sources described in the first aspect.

[0034] Fourthly, the present invention provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the deep neural network-assisted real-time separation method for mechanical sound sources described in the first aspect.

[0035] Beneficial Effects: This invention proposes a deep neural network-assisted real-time separation method for mechanical sound sources, targeting the application scenario of smart mining faces. Based on traditional temporal convolutional networks, this method proposes an improved temporal convolutional network, MLVDR-TCN, to enhance the model's ability to capture temporal information of signals. First, feature extraction is performed using MLVDR-TCN by stacking dilated convolutional blocks with different dilation coefficients to expand the network's receptive field. Second, a complex ideal ratio mask is used as the label for feature extraction by the neural network, and hyperbolic tangent compression (cIRM) is used to ensure the mask value is within a certain range, facilitating neural network training. Furthermore, to reduce computational cost and the number of parameters, depthwise separable convolution is employed, decoupling the standard convolution operation into two consecutive operations: channel-wise convolution and pointwise convolution, achieving network lightweighting. Finally, two structurally identical MLVDR-TCNs are used to predict the real and imaginary components, respectively. The loss functions for the real and imaginary components are represented by the mean squared error function. This optimizes the network parameters, and after obtaining the estimated cIRM, the source signal is decompressed and reconstructed. This invention achieves a good balance between the accuracy and stability of separation.

[0036] In summary, in the application scenario of smart mining face, for the mechanical aliasing sound source data collected by the sound sensors deployed in the mine, the method proposed in this invention can upload the trained neural network model parameters to the cloud processor after the deep learning training is completed. In actual testing, it can maintain good separation stability and accuracy of aliasing sound source samples, and then transmit the separated sound sources of each mechanical equipment back to the working face, which is convenient for workers to carry out subsequent work processing. Attached Figure Description

[0037] Figure 1 This is a general flowchart of an embodiment of the present invention.

[0038] Figure 2 This is a schematic diagram of the structure of MLVDR-TCN in an embodiment of the present invention.

[0039] Figure 3 This is a detailed structural design diagram of the dilated convolution block in the MLVDR-TCN of this invention.

[0040] Figure 4 This is a flowchart illustrating the network training process according to an embodiment of the present invention. Detailed Implementation

[0041] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0042] The core idea of ​​this invention is to propose an improved temporal convolutional network—Multi-Layer Variable Dilation Rate Temporal Convolutional Network (MLVDR-TCN)—based on traditional temporal convolutional networks, specifically for smart mining face applications, to enhance the model's ability to capture temporal information of signals. First, MLVDR-TCN is used to extract features from the complex spectrum of the input aliased signal by stacking dilated convolutional blocks with different dilation coefficients, thereby expanding the network's receptive field. Second, a complex ideal ratio mask is used as the label for feature extraction by the neural network, and hyperbolic tangent compression (cIRM) is used to ensure that the mask values ​​are within a certain range, facilitating neural network training. Third, to further reduce computational cost and the number of parameters, depthwise separable convolution is employed, decoupling the standard convolution operation into two consecutive operations: channel-wise convolution and pointwise convolution, thus achieving network lightweighting. Finally, MLVDR-TCN is used to predict the real and imaginary components separately. The loss functions for the real and imaginary components are represented by the mean square error function. The network parameters are optimized in this way, and the source signal is decompressed and reconstructed after obtaining the estimated cIRM. The invention will now be described in further detail.

[0043] like Figure 1 As shown in the figure, a deep neural network-assisted real-time separation method for mechanical sound sources disclosed in this invention mainly includes the following steps:

[0044] (1) The complex spectrum of the aliased sound source signal is extracted using the multilayer variable dilation coefficient temporal convolutional network MLVDR-TCN.

[0045] (2) The complex ideal ratio mask cIRM is used as the training label, and the hyperbolic tangent function is used to compress the cIRM to optimize the training of the MLVDR-TCN network.

[0046] (3) Two MLVDR-TCN networks with the same structure are used to predict the real and imaginary components of cIRM respectively, and the network parameters are optimized using mean square error.

[0047] (4) Collect the aliased sound source signals of the mechanical equipment, perform short-time Fourier transform (STFT) to obtain the complex spectrum, input it into the trained MLVDR-TCN network, obtain the predicted compressed mask and recover the uncompressed mask through the hyperbolic tangent inverse function, and multiply it with the complex spectrum of the aliased sound source signals to reconstruct the original sound source signals of each mechanical equipment.

[0048] Specifically, in step (1), the complex spectrum of the aliased sound source signal is first obtained (divided into real and imaginary parts, both with dimensions F×T, where F is the number of frequency points and T is the number of time frames). The traditional Temporal Convolutional Network (TCN) is improved using MLVDR-TCN. First, multiple dilated convolutional blocks with different dilation coefficients are stacked, and then these multiple dilated convolutional blocks are stacked multiple times. Then, the neurons in the hidden layer of the TCN are replaced with the stacked dilated convolutional blocks. The input of each block is padded with zeros accordingly to ensure that the output length is the same as the input.

[0049] Traditional temporal convolutional networks mainly consist of causal convolution, dilated convolution, and residual blocks. The specific definition of causal convolution is: assuming a sequence Q = (q1, q2, ..., q...). E The filter W = (w1, w2, ..., w) J ), where E is the sequence length and J is the filter length, in q e The causal convolution at the point is:

[0050]

[0051] In a TCN, the value at time e of a later layer depends only on the values ​​at time e and before that of the previous layer. Its structure is unidirectional, cannot see future data, and is a strictly time-constrained model.

[0052] Dilated convolution refers to the process of performing a dilated convolution on an input sequence Q = (q1, q2, ..., q...). E The filter W = (w1, w2, ..., w) J ), in q e When the expansion coefficient is d, the formula is expressed as:

[0053]

[0054] Because dilated convolution allows for interval sampling of the input during convolution, the sampling rate is controlled by d. Generally speaking, higher layers use larger d values, causing the effective window size to grow exponentially with the number of layers, thus achieving a large receptive field with fewer layers.

[0055] The residual block contains two layers of convolution and non-linear mapping, and weight normalization and dropout are added to each convolution layer to regularize the network.

[0056] When extracting features from complex, aliased sound source signals with massive amounts of data, traditional temporal convolutional networks (TCNNs) struggle to capture the long-term dependency features of the signals due to their limited receptive fields. To address this issue, an innovative improvement to TCNNs has been made, proposing the Multilayer Variable Dilation Coefficient Temporal Convolutional Network (MLVDR-TCN) as a feature extraction network.

[0057] First, stacked dilated convolutional blocks with different dilation coefficients d are used, such as... Figure 2 As shown, the X dilated convolutional blocks are then copied and stacked multiple times (e.g., 2-4 times) (X is preferably 7 or 8). Then, the neurons in the traditional TCN hidden layer are replaced with the stacked dilated convolutional blocks (the specific design of the dilated convolutional blocks is as follows). Figure 3 As shown in the figure, this expands the receptive field and effectively enhances the model's ability to capture the temporal information of the signal, enabling the network to better understand and process long-distance dependencies, which is particularly important for feature extraction of aliased sound source signals.

[0058] Specifically, the dilation factor of the dilated convolution block increases exponentially with the number of layers, where X dilation factors are 1, 2, ..., 2. X-1 The dilated convolutional blocks are arranged sequentially, and the input data is processed sequentially in each dilated convolutional block, with residual paths applied in each block. Figure 2 (thick solid line in the middle) and skip connection path ( Figure 2 (The dashed line in the diagram). The residual path output of one dilated convolution block serves as the input to the next dilated convolution block, while the sum of the skipped connection paths of all blocks is further processed by a ReLU function, a one-dimensional convolution calculation, and a Sigmoid function to become the output of MLVDR-TCN.

[0059] In step (2), a Complex Ideal Ratio Mask (cIRM) is used as the label for feature extraction by the neural network. The cIRM considers both the amplitude and phase spectra of the sound source signal, enhancing the amplitude and phase information of the pure sound source signal through complex multiplication. Assuming S(t,f) and Y(t,f) are the complex spectra of the pure sound source and the noisy mixed sound source, respectively, where t is time, f is frequency, M(t,f) is the cIRM, and * represents complex multiplication, then we have:

[0060] S(t,f)=M(t,f)*Y(t,f)

[0061] S(t,f), Y(t,f), and M(t,f) can be written in terms of their real and imaginary parts, respectively. The real part represents the amplitude of the cosine component of the signal at the corresponding frequency, and the imaginary part represents the amplitude of the sine component of the signal at the corresponding frequency.

[0062]

[0063] Therefore, we have:

[0064] S r +iS i =(M r +iM i )*(Y r +iY i )=(M r Y r -M i Y i )+i(M r Y i +M i Y r )

[0065] Solving the equation, we get:

[0066]

[0067] The expression for cIRM can then be obtained:

[0068]

[0069] Since complex masks may have large real and imaginary components with values ​​in the range (-∞, ∞), the hyperbolic tangent function is used to compress cIRM, as shown in the following equation:

[0070]

[0071] Where x is r or i, representing the real and imaginary components. K controls the range of mask values, i.e., the mask values ​​generated by compression are within [-K, K], and C controls the steepness. In this embodiment, several values ​​of K and C were evaluated, and K=10 and C=0.1 performed best empirically and were used to train the neural network.

[0072] To further reduce computational costs and the number of parameters, this embodiment employs depthwise separable convolution. When extracting feature maps, the standard convolution operation is decoupled into two consecutive operations: channel-wise convolution (DW) and the subsequent pointwise convolution (PW).

[0073] In channel-wise convolution (denoted as D-conv(·)), each convolutional kernel is responsible for one channel, and each channel is convolved by only one convolutional kernel. The number of feature maps generated in this process is exactly the same as the number of input channels. The mathematical expression is:

[0074]

[0075] Where A∈R M×H The input vector is a depthwise separable convolution, where M and H are the two dimensions of the input matrix A, and B ∈ R. M×P It is a convolution kernel of size P (taking the value 3). j ∈R 1×H and b j ∈R 1×P These are the rows of matrices A and B, respectively. This represents convolution. The D-conv(·) operation convolves each row of input A with the corresponding row of matrix B. Initially, M equals F.

[0076] Pointwise convolution is very similar to regular convolution, but its kernel size is 1×1×I, where I is the number of channels in the previous layer. Pointwise convolution weights and combines the feature maps generated in the previous step along the depth direction to generate new feature maps. The number of output feature maps is equal to the number of convolution kernels.

[0077] If pointwise convolution is applied linearly to the result of D-conv(·), then the mathematical expression for depthwise separable convolution is:

[0078]

[0079] Where, L∈R M×V×1 It is a convolution kernel of size 1. V is the number of channels in the convolution block.

[0080] In step (3), two MLVDR-TCN networks with identical structures are used to predict the real and imaginary components, respectively. The loss functions for the real and imaginary components can be expressed by the mean squared error function, respectively:

[0081]

[0082]

[0083] Where N represents the number of sound sources, M n,r M n,i Let represent the real and imaginary components of the cIRM calculated theoretically for the nth sound source, respectively. Let L represent the neural network estimates of the real and imaginary components of the cIRM for the nth sound source, respectively. r L i These represent the losses of the real and imaginary parts, respectively.

[0084] The training process of the network is as follows Figure 4 As shown, the trained neural network parameters will be uploaded to a cloud server. In subsequent practical applications, no further training is required; simply transmit the data collected by the sensor to the cloud-based neural network for processing.

[0085] In step (4), during the testing phase, sound sensors deployed within the mine collect aliased sound source signals from various mechanical equipment, transmit them to the cloud processor, perform a short-time Fourier transform to obtain the complex spectrum, and then input it into the neural network. Assume the output of the neural network is U. x Use the following inverse function to recover the uncompressed mask:

[0086]

[0087] After obtaining the uncompressed mask, it is multiplied with the complex spectrum of the aliased sound source signal to reconstruct the original sound source signal of each device, which is then transmitted back to the mine working face.

[0088] Based on the original sound source signals of each device after separation of the superimposed sound sources transmitted back to the mine working face, the staff can judge the working status of each mechanical device based on experience and carry out subsequent work.

[0089] Based on the description of this embodiment, those skilled in the art should readily recognize that the real-time separation method for mechanical sound sources assisted by deep neural networks in mining faces proposed in this invention can effectively reduce the time delay of the separation process and achieve a good balance between separation accuracy and stability. Contents not described in detail in this invention belong to prior art known to those skilled in the art.

[0090] Based on the same inventive concept, this invention also discloses a deep neural network-assisted real-time separation system for mechanical sound sources, comprising: a feature extraction module, used to extract features from the complex spectrum of aliased sound source signals using a multilayer variable dilation coefficient temporal convolutional network (MLVDR-TCN); wherein the MLVDR-TCN improves upon the temporal convolutional network (TCN) by first stacking multiple dilated convolutional blocks with different dilation coefficients, then stacking the multiple dilated convolutional blocks multiple times, and then replacing the neurons in the hidden layer of the TCN with the stacked dilated convolutional blocks; and a network training module, used to train the system using a complex ideal ratio mask (cIRM). To train the labels, the hyperbolic tangent function is used to compress the cIRM to optimize the MLVDR-TCN network training. Two structurally identical MLVDR-TCN networks are used to predict the real and imaginary components of the cIRM, respectively, and the mean square error is used to optimize the network parameters. A sound source reconstruction module is also included, which collects the aliased sound source signals of the mechanical equipment, performs a short-time Fourier transform to obtain the complex spectrum, inputs it into the trained MLVDR-TCN network, obtains the predicted compressed mask, recovers the uncompressed mask using the inverse hyperbolic tangent function, and multiplies it with the complex spectrum of the aliased sound source signals to reconstruct the original sound source signals of each mechanical equipment. Specific implementation details of each module in this system embodiment are described in the above method embodiment and will not be repeated here.

[0091] This invention also discloses a computer system, including a memory, a processor, and a computer program / instructions stored in the memory and executable on the processor. When the computer program / instructions are executed by the processor, they implement the steps of the aforementioned deep neural network-assisted real-time separation method for mechanical sound sources.

[0092] This invention also discloses a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the aforementioned deep neural network-assisted real-time separation method for mechanical sound sources.

Claims

1. A method for real-time separation of mechanical sound sources assisted by a deep neural network, characterized in that, Includes the following steps: (1) The complex spectrum of the aliased sound source signal is extracted using the multilayer variable dilation coefficient temporal convolutional network MLVDR-TCN; the MLVDR-TCN improves the temporal convolutional network TCN by first stacking multiple dilated convolutional blocks with different dilation coefficients, then stacking the multiple dilated convolutional blocks multiple times, and then replacing the neurons of the TCN hidden layer with the stacked dilated convolutional blocks. (2) The complex ideal ratio mask cIRM is used as the training label, and the hyperbolic tangent function is used to compress cIRM to optimize the training of MLVDR-TCN network. (3) Two MLVDR-TCN networks with the same structure are used to predict the real and imaginary components of cIRM respectively, and the network parameters are optimized by mean square error. (4) Collect the aliased sound source signals of the mechanical equipment, perform short-time Fourier transform to obtain the complex spectrum, input it into the trained MLVDR-TCN network, obtain the predicted compressed mask, and recover the uncompressed mask through the hyperbolic tangent inverse function. Then multiply it with the complex spectrum of the aliased sound source signals to reconstruct the original sound source signals of each mechanical equipment.

2. The method for real-time separation of mechanical sound sources assisted by a deep neural network according to claim 1, characterized in that, In the MLVDR-TCN, the dilation factor of the dilated convolutional block increases exponentially with the number of layers, with dilation factors of 1, 2, ..., 2. X-1 The dilated convolutional blocks are arranged sequentially, where X is the number of dilated convolutional blocks. The input data is processed sequentially in each dilated convolutional block, and residual paths and skipped connection paths are applied in each dilated convolutional block. The residual path output of one dilated convolutional block is used as the input of the next dilated convolutional block. The sum of the skipped connection paths of all dilated convolutional blocks is processed by a ReLU function, a one-dimensional convolution calculation, and a Sigmoid function to become the output of MLVDR-TCN.

3. The method for real-time separation of mechanical sound sources assisted by a deep neural network according to claim 1, characterized in that, The expression for cIRM is: Among them, M r M i These are the real and imaginary components of the cIRM, Y. r Y i These are the real and imaginary components of the complex spectrum of a noisy mixed sound source, S. r S i These are the real and imaginary components of the complex spectrum of a pure sound source, respectively, with i marking the imaginary part.

4. The method for real-time separation of mechanical sound sources assisted by a deep neural network according to claim 3, characterized in that, The cIRM compressed using the hyperbolic tangent function is expressed as: Among them, cIRM r and cIRM i These are the real and imaginary components of the compressed mask. K controls the range of mask values, and C controls the steepness.

5. The method for real-time separation of mechanical sound sources assisted by a deep neural network according to claim 1, characterized in that, In the MLVDR-TCN, the convolution operation uses depthwise separable convolution.

6. The method for real-time separation of mechanical sound sources assisted by a deep neural network according to claim 1, characterized in that, The loss functions for training the two MLVDR-TCN networks are expressed as follows: Where N represents the number of sound sources, M n,r M n,i Let represent the real and imaginary components of the cIRM calculated theoretically for the nth sound source, respectively. Let L represent the neural network estimates of the real and imaginary components of the cIRM for the nth sound source, respectively. r L i These represent the losses of the real and imaginary parts, respectively.

7. The method for real-time separation of mechanical sound sources assisted by a deep neural network according to claim 1, characterized in that, During the sound source reconstruction process, the uncompressed mask is recovered using the following formula: Among them, U r U i These represent the real and imaginary components of the compressed output of MLVDR-TCN, respectively. These represent the real and imaginary components of the uncompressed cIRM, respectively. K controls the range of mask values, and C controls the steepness.

8. A deep neural network-assisted real-time separation system for mechanical sound sources, characterized in that, include: The feature extraction module is used to extract features from the complex spectrum of the aliased sound source signal using a multilayer variable dilation coefficient temporal convolutional network MLVDR-TCN. The MLVDR-TCN improves upon the temporal convolutional network TCN by first stacking multiple dilated convolutional blocks with different dilation coefficients, then stacking these multiple dilated convolutional blocks multiple times, and finally replacing the neurons in the TCN hidden layer with the stacked dilated convolutional blocks. The network training module is used to use the complex ideal ratio mask cIRM as the training label and to compress the cIRM using the hyperbolic tangent function to optimize the training of the MLVDR-TCN network. Two MLVDR-TCN networks with the same structure are used to predict the real and imaginary components of the cIRM respectively, and the network parameters are optimized using mean square error. The system also includes a sound source reconstruction module, which is used to collect aliased sound source signals from mechanical equipment, perform short-time Fourier transform to obtain complex spectra, input them into a trained MLVDR-TCN network, obtain a predicted compressed mask, recover the uncompressed mask through the inverse hyperbolic tangent function, and multiply it with the complex spectrum of the aliased sound source signals to reconstruct the original sound source signals of each mechanical equipment.

9. A computer system comprising a memory, a processor, and computer programs / instructions stored in the memory and executable on the processor, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of a deep neural network-assisted real-time separation method for mechanical sound sources according to any one of claims 1-7.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of a deep neural network-assisted real-time separation method for mechanical sound sources according to any one of claims 1-7.

Images