Deep learning based robot non-stationary noise filtering method

Abstract

The application relates to the technical field of deep learning, in particular to a robot non-stationary noise filtering method based on deep learning, which comprises the following steps: acquiring an audio signal, performing frequency domain conversion on the audio signal to obtain a multi-channel complex spectrum; performing directional estimation on the multi-channel complex spectrum to obtain a target direction; and performing beamforming on the multi-channel complex spectrum based on the target direction to obtain a preliminary enhanced signal.In the application, the preliminary enhanced signal is denoised to obtain a pure speech signal according to an environmental background noise spectrum and a target human voice confidence degree, a fine-grained confidence index is used to dynamically control the update rate of the noise spectrum and the suppression intensity of the filter, conservative denoising is adopted in a frequency band highly confirmed as the target human voice to protect the speech integrity, and active suppression is performed in a period confirmed as non-target human voice or pure noise, so that the intelligibility and fidelity of the target speech are maximally guaranteed while eliminating non-stationary competitive interference.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology, and in particular to a deep learning-based method for filtering non-stationary noise in robots. Background Technology

[0002] In scenarios such as service robots, smart homes, and conferencing systems, clear voice interaction is crucial. Existing robot voice noise reduction technologies can be mainly categorized as follows:

[0003] 1. Single-microphone beamforming: This method uses a microphone array to form a directional sound pickup beam, suppressing noise outside the beam. It is effective against steady-state noise in a fixed direction, but has poor suppression of non-target human voices or sudden noises within the beam.

[0004] 2. Spectral noise reduction: This method estimates the noise spectrum and subtracts it from the noisy speech spectrum. It is simple to implement, but inaccurate noise estimation can lead to residual "musical noise" or speech distortion.

[0005] 3. End-to-end noise reduction based on deep learning: This method uses neural networks to directly map noisy speech to clean speech. While effective, this approach suffers from large model size, high computational cost, strong dependence on training data, and potentially reduced generalization ability in unknown noise environments.

[0006] 4. Simple Voice Detection (VAD): Updates the noise spectrum during silent segments and performs noise reduction during speech segments. However, when faced with continuous background noise (such as television or conversations with others), it is prone to mistakenly preserving non-target speech.

[0007] Therefore, improvements to existing technologies are necessary. Summary of the Invention

[0008] The purpose of this invention is to address the shortcomings of existing technologies by proposing a deep learning-based method for filtering non-stationary noise in robots.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: a deep learning-based method for filtering non-stationary noise in robots, comprising the following steps:

[0010] Acquire an audio signal, perform frequency domain transformation on the audio signal, and obtain a multi-channel complex spectrum;

[0011] The target direction is obtained by performing orientation estimation on the multi-channel complex spectrum;

[0012] Beamforming is performed on the multi-channel complex spectrum based on the target direction to obtain a preliminary enhanced signal;

[0013] The voiceprint embedding features of the preliminary enhanced signal are extracted, and human voice activity analysis is performed on the preliminary enhanced signal to obtain human voice probability parameters;

[0014] The voiceprint embedding features are compared with the registered voiceprints to obtain the voiceprint matching similarity.

[0015] By fusing the human voice probability parameters with the voiceprint matching similarity, a target human voice confidence score is generated.

[0016] Background noise is estimated based on the target human voice confidence level to obtain the environmental background noise spectrum;

[0017] The initial enhanced signal is denoised based on the ambient background noise spectrum and the target human voice confidence level to obtain a clean speech signal.

[0018] Preferably, the steps of acquiring the audio signal and performing frequency domain transformation on the audio signal to obtain a multi-channel complex spectrum are as follows:

[0019] Multiple raw audio data streams are acquired using a microphone array, filtered using a low-pass filter to obtain filtered audio data, and the time series of the filtered audio data is extracted. The time series is then used to construct an audio signal, which is input into a pre-emphasis filter for high-frequency boosting to obtain a pre-emphasis time-domain signal. The pre-emphasis time-domain signal is then segmented into frames to obtain multiple overlapping data frames. The multiple overlapping data frames are then windowed and smoothed using a window function to obtain a windowed audio frame sequence. Finally, a short-time Fourier transform is performed on the windowed audio frame sequence to generate a multi-channel complex spectrum.

[0020] Preferably, the step of performing orientation estimation on the multi-channel complex spectrum to obtain the target direction specifically includes:

[0021] The cross-power spectral density parameters of each channel in the multi-channel complex spectrum are extracted, and the cross-power spectral density parameters are normalized using a phase transformation function to obtain the cross-correlation spectral parameters.

[0022] The cross-correlation spectrum parameters are subjected to inverse Fourier transform to obtain a spatial time delay matrix. The peak energy distribution of the spatial time delay matrix is ​​analyzed, the maximum peak parameter is extracted, the maximum peak parameter is converted into spatial angular coordinates, the horizontal azimuth angle corresponding to the spatial angular coordinates is extracted, and the horizontal azimuth angle is input to the signal classification model for spatial spectrum search to obtain a high-resolution spatial spectrum.

[0023] The direction vectors of the extreme points in the high-resolution spatial spectrum are extracted and determined as the target directions.

[0024] Preferably, the step of beamforming the multi-channel complex spectrum based on the target direction to obtain a preliminary enhanced signal specifically includes:

[0025] An image sensor is used to acquire an environmental image. A neural network model is used to perform face recognition on the environmental image to obtain the face position coordinates. The face position coordinates are mapped to an acoustic coordinate system to obtain an auxiliary reference direction. The spatial angle between the target direction and the auxiliary reference direction is calculated. The target direction is calibrated using the spatial angle value to obtain a calibrated target direction. The noise covariance matrix of the multi-channel complex spectrum is extracted. The steering vector parameters are extracted based on the calibrated target direction. The noise covariance matrix and the steering vector parameters are solved using a distortion-free response algorithm to obtain a beamforming weight matrix. The beamforming weight matrix is ​​then multiplied by the multi-channel complex spectrum to generate a preliminary enhancement signal.

[0026] Preferably, the steps of extracting the voiceprint embedding features of the preliminary enhanced signal and performing voice activity analysis on the preliminary enhanced signal to obtain voice probability parameters are as follows:

[0027] The initial enhancement signal is divided into multiple short-time analysis frames, and acoustic feature parameters of each short-time analysis frame are extracted. The acoustic feature parameters are input into a preset neural network for feature mapping, and a high-dimensional voiceprint representation sequence is output. The high-dimensional voiceprint representation sequence is reduced in dimensionality using a pooling function to generate voiceprint embedding features. The initial enhancement signal is input into the frequency domain layer of the human voice detection model to extract local feature maps.

[0028] Logistic regression is performed on the local feature map using a fully connected layer to calculate the current frame probability of the preliminary enhanced signal in each of the short-time analysis frames. Based on the time series, all the current frame probabilities are spliced ​​and integrated to construct the human voice probability parameters.

[0029] Preferably, the step of comparing the embedded voiceprint features with the registered voiceprint to obtain the voiceprint matching similarity is as follows:

[0030] Extract the target user's identity information, load the registered voiceprint based on the identity information, calculate the vector inner product of the voiceprint embedding feature and the registered voiceprint, calculate the feature norm of the voiceprint embedding feature and the reference norm of the registered voiceprint, and divide the vector inner product by the product of the feature norm and the reference norm to obtain the cosine distance value.

[0031] Load a pre-set interference model library, extract the global interference centroid parameter of the interference model library, calculate the Euclidean distance attenuation value between the voiceprint embedding feature and the global interference centroid parameter, and use the Euclidean distance attenuation value to negatively compensate the cosine distance value to obtain the voiceprint matching similarity.

[0032] Preferably, the step of fusing the human voice probability parameters with the voiceprint matching similarity to generate the target human voice confidence score specifically includes:

[0033] Extract the deviation angle between the target direction and the central axis of the preset area, extract the current frame probability from the human voice probability parameters, construct a collaborative confidence evaluation model, and jointly solve the model by inputting the deviation angle, the current frame probability, and the voiceprint matching similarity to generate the target human voice confidence score, as shown in the formula: ;

[0034] in, This indicates the confidence level of the target human voice. Indicates the probability of the current frame. This indicates the similarity of the voiceprint matching. Indicates the deviation angle, Represents the angle sensitivity coefficient. This represents the collaborative gain coefficient, which is used to perform probability compensation on the confidence level of the target human voice.

[0035] Preferably, the step of estimating background noise based on the target human voice confidence level to obtain the environmental background noise spectrum specifically includes:

[0036] Extract the background noise spectrum parameters of the previous frame from the historical time period, calculate the frequency domain power spectrum parameters of the preliminary enhanced signal in the current frame at the current time, and calculate the translational motion update rate based on the target human voice confidence level, using the following formula: ,in, This represents the translational sliding velocity update rate. Represents the basic smoothing coefficient. This indicates the confidence level of the target human voice. Indicates the adaptive damping coefficient;

[0037] The adaptive damping coefficient is used to protect speech features from being covered. The current newly added noise component is obtained by multiplying the translational speed update rate by the current frame frequency domain power spectrum parameter. The difference between 1 and the translational speed update rate is multiplied by the previous frame background noise spectrum parameter to obtain the historically retained noise component. The current newly added noise component and the historically retained noise component are summed to generate the current frame ambient background noise spectrum. The ambient background noise spectrum is obtained by integrating the current frame ambient background noise spectra of each frame.

[0038] Preferably, the step of denoising the preliminary enhanced signal based on the ambient background noise spectrum and the target human voice confidence level to obtain a clean speech signal specifically includes:

[0039] The frequency band signal-to-noise ratio parameter is calculated using the ambient background noise spectrum, the gain function of the Wiener filter is modulated using the target human voice confidence, and the time-frequency masking matrix is ​​calculated using the Wiener filter.

[0040] The time-frequency masking matrix is ​​multiplied with the preliminary enhancement signal to obtain the filtered enhancement spectrum. The filtered enhancement spectrum is then inverted to the time domain by performing an inverse short-time Fourier transform. The boundary truncation effect is eliminated by using an overlap-add algorithm to generate a clean speech signal. The clean speech signal is then transmitted to the speech recognition engine for parsing.

[0041] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0042] In this invention, an audio signal is acquired and frequency-domain transformed to obtain a multi-channel complex spectrum. The multi-channel complex spectrum is then directionally estimated to obtain the target direction. Based on the target direction, beamforming is applied to the multi-channel complex spectrum to obtain a preliminary enhanced signal. This effectively suppresses background noise deviating from the target direction in the spatial dimension and improves the initial signal-to-noise ratio. Voiceprint embedding features of the preliminary enhanced signal are extracted, and voice activity analysis is performed to obtain voice probability parameters. The voiceprint embedding features are compared with registered voiceprints to obtain voiceprint matching similarity. The voice probability parameters and voiceprint matching similarity are fused to generate the target voice confidence score. This process combines physical-level spatial filtering with deep identity feature recognition to distinguish the target user's voice from other interfering voices, avoiding misjudging non-target voices such as conversations in the environment as valid signals. Background noise is estimated based on the confidence level of the target human voice to obtain the ambient background noise spectrum. The ambient background noise spectrum and the confidence level of the target human voice are used to denoise the preliminary enhanced signal to obtain a clean speech signal. Fine-grained confidence index is used to dynamically control the update rate of the noise spectrum and the suppression intensity of the filter. Conservative noise reduction is adopted in the frequency band that is highly identified as the target human voice to protect the speech integrity. Active suppression is performed in the time period that is identified as non-target human voice or pure noise, thereby eliminating non-stationary competitive interference while maximizing the clarity and fidelity of the target speech. Attached Figure Description

[0043] Figure 1 This is a schematic diagram of the steps of the present invention;

[0044] Figure 2 This is a network structure diagram of a signal classification model. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0046] Please see Figure 1-2 This invention provides a technical solution: a deep learning-based method for filtering non-stationary noise in robots, comprising the following steps:

[0047] The audio signal is acquired, and its frequency domain is transformed to obtain a multi-channel complex spectrum.

[0048] The target direction is obtained by performing orientation estimation on the multi-channel complex spectrum.

[0049] Beamforming is performed on the multi-channel complex spectrum based on the target direction to obtain a preliminary enhanced signal;

[0050] The voiceprint embedding features of the preliminary enhanced signal are extracted, and the voice activity analysis is performed on the preliminary enhanced signal to obtain the voice probability parameters;

[0051] The voiceprint embedding features are compared with the registered voiceprints to obtain the voiceprint matching similarity.

[0052] By fusing human voice probability parameters with voiceprint matching similarity, a target human voice confidence score is generated.

[0053] Background noise is estimated based on the target human voice confidence level to obtain the environmental background noise spectrum;

[0054] The initial enhanced signal is denoised based on the background noise spectrum and the confidence level of the target human voice to obtain a clean speech signal.

[0055] In this embodiment, the steps of acquiring audio signals, performing frequency domain transformation on the audio signals, and obtaining multi-channel complex spectra are as follows: acquiring multiple channels of raw audio data through a microphone array, filtering them using a low-pass filter to obtain filtered audio data, extracting the time series of the filtered audio data, constructing the time series into an audio signal, inputting the audio signal into a pre-emphasis filter for high-frequency boosting to obtain a pre-emphasis time-domain signal, dividing the pre-emphasis time-domain signal into frames to obtain multiple overlapping data frames, applying a window function to smooth the multiple overlapping data frames to obtain a windowed audio frame sequence, and performing a short-time Fourier transform on the windowed audio frame sequence to generate a multi-channel complex spectrum.

[0056] Specifically, the audio signal is acquired, acquisition channels are set, and multiple raw audio data are acquired through a microphone array. A low-pass filter component is configured, and the cutoff frequency of the low-pass filter is set to 8000Hz, because the effective information of the formants of human speech is mostly concentrated in the frequency band below 8000Hz. A Butterworth low-pass filter is used to calculate the attenuation coefficient to suppress high-frequency environmental noise and electromagnetic interference signals beyond this frequency band. Filtered audio data is obtained by using the low-pass filter. Continuous analog-to-digital conversion data points are read at a fixed sampling frequency of 16,000 times per second. A one-dimensional data vector is established to store these discrete sampled values. The time series of the filtered audio data is extracted, and the time points acquired from each channel are aligned and combined into a multi-dimensional matrix format. The time series is constructed into an audio signal. The digital filter equation is constructed as the current value minus the previous value multiplied by an emphasis factor. The pre-emphasis factor is set to 0.97 to compensate for the high-frequency energy attenuation of the speech signal during propagation and to flatten the spectral envelope. The audio signal is then input into the pre-emphasis filter for high-frequency boosting. The pre-emphasized time-domain signal is obtained. The window length along the time axis is set to 320 sampling points, corresponding to a duration of 20 milliseconds. The step size for each slide is set to 160 sampling points, or 10 milliseconds. The pre-emphasized time-domain signal is divided into frames to ensure that there is a 50% overlap rate between adjacent data blocks to prevent feature loss. Multiple overlapping data frames are obtained. A Hamming window is used as the smoothing function. The weights are distributed according to the shape of the cosine function, with the amplitude in the middle being 1 and gradually decreasing to 0.08 at both ends. The weight of each sampling point in the window is multiplied by the weight of the corresponding position of the Hamming window to reduce the abrupt amplitude at both ends of the frame boundary and reduce the spectral leakage phenomenon in the short-time Fourier transform process. The window function is used to smooth the multiple overlapping data frames to obtain a windowed audio frame sequence. A 512-point high-speed Fourier transform basis function matrix is ​​configured. The discrete sampling points in the time domain are projected onto an orthogonal frequency basis. The real and imaginary parts in each frequency interval are calculated. A short-time Fourier transform is performed on the windowed audio frame sequence. The two-dimensional time-frequency matrices generated by each microphone channel are merged to generate a multi-channel complex spectrum.

[0057] In this embodiment, the specific steps for directional estimation of the multi-channel complex spectrum to obtain the target direction are as follows: extracting the cross-power spectral density parameters of each channel in the multi-channel complex spectrum, normalizing the cross-power spectral density parameters using a phase transformation function to obtain cross-correlation spectral parameters; performing an inverse Fourier transform on the cross-correlation spectral parameters to obtain a spatial time delay matrix, analyzing the peak energy distribution of the spatial time delay matrix, extracting the maximum peak parameter, converting the maximum peak parameter into spatial angular coordinates, extracting the horizontal azimuth angle corresponding to the spatial angular coordinates, using the horizontal azimuth angle as input to the signal classification model for spatial spectrum search to obtain a high-resolution spatial spectrum; and extracting the direction vector of the extreme points in the high-resolution spatial spectrum to determine the target direction.

[0058] Specifically, the multi-channel complex spectrum is directionally estimated by traversing all channel pairings at the receiver. Assuming four microphone nodes are deployed, six cross-pairing terms are calculated through permutations and combinations. The conjugate product of each signal pair in the frequency domain is calculated, and the cross-power spectral density parameters of each channel in the multi-channel complex spectrum are extracted. The data is processed using phase transform weighting to extract the phase information of the cross-power spectrum and divide it by its absolute amplitude to eliminate the interference of the sound source signal's own spectral characteristics on peak localization. The cross-power spectral density parameters are normalized using a phase transform function to obtain the cross-correlation spectral parameters. The phasor moments in the frequency domain are then transformed using an inverse discrete Fourier transform. Mapping the array back to the time delay domain, finding the coordinates of the peak position of energy accumulation on the correlation curve, performing an inverse Fourier transform on the cross-correlation spectrum parameters to obtain the spatial time delay matrix, scanning the numerical distribution on the time delay axis, finding the location of the data point with the highest energy value, analyzing the peak energy distribution of the spatial time delay matrix, calculating the offset of the highest point from the zero time delay point, assuming a microphone spacing of 0.05 meters and a sound speed of 340 meters per second, the maximum theoretical physical time delay is approximately 0.000147 seconds, extracting the highest bulge value exceeding the noise floor, extracting the maximum peak parameter, and substituting the time delay value into the spatial solution containing the microphone geometric topological coordinates. The wavefront arrival angle is solved from the analytical geometric equations, and the maximum peak value parameter is converted into spatial angular coordinates. The angle value projected onto the horizontal plane in the three-dimensional spherical coordinate system is extracted, and the corresponding horizontal azimuth angle is extracted. A signal classification model is constructed and trained. 100,000 multi-channel recording data of microphone arrays under different indoor reverberation environments are collected. The corresponding real sound source azimuth angles are manually labeled as supervision labels. A deep neural network structure containing one input layer, three one-dimensional convolutional layers, two fully connected layers, and one output layer is constructed. The input layer is configured to receive the horizontal azimuth angle and the phase delay sequence of multiple historical frames. The one-dimensional convolutional layer is configured... A convolutional kernel of size 3 is used in each layer, and a linear rectified activation function is used to extract local correlation features of spatial phase. A fully connected layer containing 512 neurons is configured to map local features to the global spatial spectrum dimension. The output layer is set to output a probability distribution vector of length 360 using a normalized exponential function to represent the energy response values ​​in each direction from 0 to 359 degrees. A cross-entropy loss function is defined to calculate the deviation between the predicted probability vector and the true sharpened spatial spectrum vector. The Adam algorithm is used to update the weight parameters through backpropagation with an initial learning rate of 0.001. The batch size is set to 64, and 100 iterations are performed until the loss value converges to 0.Below 01, the network parameters are solidified to obtain a signal classification model. The response probability at each angle is calculated through forward propagation. The horizontal azimuth input signal classification model is then subjected to a spatial spectrum search. The output 360-dimensional vector is mapped to a three-dimensional surface spectrum in polar coordinates to obtain a high-resolution spatial spectrum. The extreme point with the most pronounced probability peak is found in the generated three-dimensional surface spectrum. The direction vector of this extreme point is extracted and determined as the target direction.

[0059] In this embodiment, the steps for beamforming a multi-channel complex spectrum based on the target direction to obtain a preliminary enhanced signal are as follows: An image sensor is used to acquire an environmental image; a neural network model is used to perform face recognition on the environmental image to obtain the face position coordinates; the face position coordinates are mapped to an acoustic coordinate system to obtain an auxiliary reference direction; the spatial angle between the target direction and the auxiliary reference direction is calculated; the spatial angle is used to calibrate the target direction to obtain the calibrated target direction; the noise covariance matrix of the multi-channel complex spectrum is extracted; the steering vector parameters are extracted based on the calibrated target direction; a distortion-free response algorithm is used to solve for the noise covariance matrix and the steering vector parameters to obtain the beamforming weight matrix; and the beamforming weight matrix is ​​used to perform an inner product operation with the multi-channel complex spectrum to generate a preliminary enhanced signal.

[0060] Specifically, beamforming is performed on the multi-channel complex spectrum based on the target direction. The front-facing camera component is enabled to access the image sensor to acquire environmental images. A RetinaFace visual object detection network containing multi-layer convolutional structures and fully connected layers is configured. The training process of this network includes a dataset of face photos with different lighting conditions and face rotation angles. The four vertex pixel coordinates of the outer bounding box of the face are manually labeled for each image as the ground truth label. A feature pyramid network structure is built to extract multi-scale image texture features. The positional deviation between the predicted bounding box and the ground truth bounding box is calculated using the cross-entropy loss function. A stochastic gradient descent algorithm is used with a learning rate of 0.001. The weight parameters in the forward propagation network are updated. After 50 cycles of forward and backward iterations until the loss value converges below the threshold of 0.05, the network parameters are solidified to obtain a mature model file. The currently captured environmental image is input into the forward propagation layer of this model, which outputs the predicted bounding box data. The neural network model is used to perform face recognition on the environmental image. The two-dimensional pixel coordinates of the intersection of the diagonals of the bounding box are calculated to obtain the face position coordinates. Based on the camera's internal focal length matrix and external translation and rotation matrix, the line of sight ray in three-dimensional space is inferred according to the principle of pinhole imaging geometric projection. The face position coordinates are mapped to the acoustic coordinate system to obtain the auxiliary reference direction. Based on vector... The dot product formula is used to calculate the spatial angle between the acoustic estimation direction and the visual reference direction. The spatial angle between the target direction and the auxiliary reference direction is calculated and compared to a preset 5-degree tolerance range. If the angle is less than 5 degrees, the two coordinates are averaged and fused. If the angle is greater than 5 degrees, the visual auxiliary direction is trusted and overlaid with a weight of 0.8 on the original acoustic direction. The spatial angle value is used to calibrate the target direction, resulting in the calibrated target direction. During a clean background period with no speech activity, the average value of the multi-frame spectral energy matrix is ​​calculated. The noise covariance matrix of the multi-channel complex spectrum is extracted, and the results are determined based on the array geometry and spatial acoustic wave characteristics. The theoretical phase delay difference between each pickup node is calculated. Based on the calibrated target direction, the steering vector parameters are extracted. A constrained optimization objective function is constructed with a constant response of 1 in the target direction and the requirement to minimize the total output noise power. The closed-form solution formula is derived using the Lagrange multiplier method. The noise covariance matrix and steering vector parameters are solved using a distortionless response algorithm. The complex weight parameters assigned to each channel are calculated by matrix inversion to obtain the beamforming weight matrix. The weight parameters of each channel are multiplied element-wise with the corresponding frequency domain data and then summed between channels. The beamforming weight matrix is ​​then multiplied with the complex spectrum of the multi-channel signal to generate the initial enhanced signal.

[0061] In this embodiment, the steps of extracting the voiceprint embedding features of the preliminary enhancement signal and performing voice activity analysis on the preliminary enhancement signal to obtain voice probability parameters are as follows: the preliminary enhancement signal is divided into multiple short-time analysis frames, the acoustic feature parameters of each short-time analysis frame are extracted, the acoustic feature parameters are input into a preset neural network for feature mapping, and a high-dimensional voiceprint representation sequence is output. The high-dimensional voiceprint representation sequence is reduced in dimensionality using a pooling function to generate voiceprint embedding features. The preliminary enhancement signal is input into the frequency domain layer of the voice detection model to extract local feature maps. Logistic regression is performed on the local feature maps using a fully connected layer to calculate the current frame probability of the preliminary enhancement signal in each short-time analysis frame. Based on the time series, all current frame probabilities are spliced ​​and integrated to construct the voice probability parameters.

[0062] Specifically, the speaker embedding features of the initial enhanced signal are extracted, and human voice activity analysis is performed on the initial enhanced signal. The continuous data after beamforming is segmented and separated along the time axis at preset fixed data point steps, dividing the initial enhanced signal into multiple short-time analysis frames. Acoustic feature parameters of each short-time analysis frame are extracted sequentially, and a set of values, including Mel frequency cepstral coefficients, is calculated. A triangular filter bank is used to map linear frequencies onto a nonlinear Mel scale that conforms to human auditory perception. A speaker extraction model based on a deep time-delay neural network is configured. The training process of this model involves compiling long-segment speech fragments from 10,000 different speakers to construct a training corpus, and then segmenting each speech fragment... Each model is assigned a unique digital identity tag. The extracted acoustic feature matrix sequence is input and passed through five time-delay layers configured with dilated convolutions to capture large-scale temporal dependencies in the speech context. A global statistical pooling layer is then added at the end of the network to calculate the mean and standard deviation feature vectors across the entire time dimension. A classification cross-entropy loss function is used to evaluate the difference between the predicted identity and the true label, and backpropagation is used to update the parameters. The model is trained for 100 epochs with a learning rate of 0.005, enabling it to distinguish different distributions of human voice feature vectors in the latent space. The acoustic feature parameters are then input into a pre-defined neural network for feature mapping, outputting a high-dimensional voiceprint representation sequence, where each frame corresponds to a 512-dimensional... The floating-point vector is used to compress the sequence length by calling the averaging operation in mathematical statistics functions. The high-dimensional voiceprint representation sequence is reduced in dimensionality using the pooling function. By calculating the average of all frames in the 512 dimensions, it is compressed into a single compact feature vector to generate voiceprint embedding features. A speech activity detection model composed of multi-layer two-dimensional convolutional kernels and long short-term memory network layers is configured. The training process of this detection model collects various audio samples mixed with environmental noise and clean human voices as sample data. Each frame is manually labeled with 0 to indicate no human voice and 1 to indicate human voice. The input frequency domain data is processed by convolutional layers to extract texture feature contours and then fed into the memory network to capture the temporal dynamic correlation of the speech. The loss error is calculated using binary cross-entropy, and the internal node parameters are updated using the Adam algorithm. The initial enhanced signal is input into the frequency domain layer of the human voice detection model to extract local feature maps, which are then mapped from multi-dimensional features to one-dimensional vectors. Logistic regression is performed on the local feature maps using a fully connected layer. By introducing a non-linear Sigmoid activation function, the output values ​​without boundaries are forcibly mapped and compressed to a probability range of 0 to 1. The current frame probability of the initial enhanced signal in each short-time analysis frame is calculated. An empty buffer array is created, and discrete probability values ​​are written in ascending order of time. Based on the time series, all current frame probabilities are concatenated and integrated to construct the human voice probability parameters.

[0063] In this embodiment, the specific steps for comparing the voiceprint embedding feature with the registered voiceprint to obtain the voiceprint matching similarity are as follows: extract the target user's identity information, load the registered voiceprint based on the identity information, calculate the vector inner product value of the voiceprint embedding feature and the registered voiceprint, calculate the feature norm of the voiceprint embedding feature and the reference norm of the registered voiceprint, divide the vector inner product value by the product of the feature norm and the reference norm to obtain the cosine distance value; load the preset interference model library, extract the global interference centroid parameter of the interference model library, calculate the Euclidean distance attenuation value between the voiceprint embedding feature and the global interference centroid parameter, and use the Euclidean distance attenuation value to negatively compensate the cosine distance value to obtain the voiceprint matching similarity.

[0064] Specifically, the process involves comparing the embedded voiceprint features with the registered voiceprint, reading the user identity code received by the external interactive components or searching for the set master control account ID number in the local pre-configured configuration file, extracting the target user's identity information, performing a matching query in the local read-only memory database to find the standard voiceprint vector template pre-recorded and processed by the model for that ID account, loading the registered voiceprint based on the identity information, performing linear algebraic operations of element-wise multiplication and summation between the two multidimensional vectors, calculating the vector inner product value of the embedded voiceprint features and the registered voiceprint, calculating the sum of squares of all elements in all dimensions of these two long bar vectors and taking the square root of the sum, calculating the feature norm of the embedded voiceprint features and the reference norm of the registered voiceprint, multiplying the norms of the two as the denominator to constrain the size boundary of the inner product data, dividing the vector inner product value by the product of the feature norm and the reference norm, and deriving a decimal result whose value is always limited to between -1 and 1 according to the formula of the cosine similarity theorem to obtain the cosine distance value, and reading the pre-written data containing television broadcasts from the internal solid-state storage medium. The system collects a set of interference features, including human voices and noisy conversations among densely packed crowds. It loads a pre-built interference model library, iterates through all interference feature vector sequences in the library, and calculates the average value across each dimension as the center point representing this type of interference. It extracts the global interference centroid parameter from the interference model library, calculates the sum of squared differences between the currently extracted voiceprint feature and the center point vector in each corresponding dimension, and takes the square root of the final sum. It then calculates the Euclidean distance attenuation between the voiceprint embedding feature and the global interference centroid parameter. A multi-branch conditional judgment compensation logic is designed to address similarity misjudgments. When the calculated Euclidean distance attenuation is less than 0.5, it indicates a high overlap between the current pickup feature and the interference source feature. In this case, 0.3 is subtracted from the original cosine distance value as a penalty to forcibly lower the matching degree. If the Euclidean distance value is greater than 0.5, it indicates deviation from the interference source, and no deduction is made. The Euclidean distance attenuation value is used to negatively compensate the cosine distance value, correcting the potentially high false positive matching values ​​caused by overlapping acoustic channel features, thus obtaining the voiceprint matching similarity.

[0065] In this embodiment, the specific steps for generating the target voice confidence score by fusing voice probability parameters and voiceprint matching similarity are as follows: extracting the deviation angle between the target direction and the central axis of the preset region, extracting the current frame probability from the voice probability parameters, constructing a collaborative confidence evaluation model, and inputting the deviation angle, the current frame probability, and the voiceprint matching similarity into the collaborative confidence evaluation model for joint solution to generate the target voice confidence score, as shown in the formula: ;in, Indicates the confidence level of the target human voice. Indicates the probability of the current frame. Indicates the similarity of voiceprint matching. Indicates the deviation angle. Represents the angle sensitivity coefficient. This represents the collaborative gain coefficient, which is used for probability compensation of the confidence level of the target human voice.

[0066] Specifically, the process integrates voice probability parameters with voiceprint matching similarity, reads the absolute deviation values ​​of the spatial angles from the baseline output by previous spatial orientation and visual calibration, calculates the spatial angle between the target direction ray and the central axis of the reference line directly in front using trigonometric functions, extracts the deviation angle between the target direction and the central axis of the preset area, retrieves the specific value of the current processing frame's time scale from the one-dimensional array of voice probability parameters containing timestamps, extracts the current frame probability from the voice probability parameters, designs a multivariate fusion mathematical calculation framework compatible with input signals of various physical dimensions, constructs a collaborative confidence evaluation model, and jointly solves the collaborative confidence evaluation model by inputting the deviation angle, current frame probability, and voiceprint matching similarity to generate the target voice confidence score, as shown in the formula: ,in, The confidence level of the target human voice is represented by a floating-point number between 0 and 1. A value closer to 1 indicates a more certain determination that the voice belongs to the target user. It is obtained by comprehensively calculating the probability and similarity product, the angle exponent penalty term, and the logarithmic gain. This represents the probability of the current frame containing human vocal activity. This parameter is truncated between 0 and 1 by the logistic regression output of the detection network. For example, 0.85 indicates an extremely high probability of containing human voice. This indicates the similarity of voiceprint matching, which quantifies the degree of fit between the real-time extracted voiceprint and the locally registered voiceprint. It is obtained by inner product operation and is a decimal between 0 and 1, such as 0.9 representing a high degree of consistency. This indicates the deviation angle, recording the absolute value of the spatial angle formed by the sound source position and the central axis directly in front, obtained through positioning calculations, such as a deviation of 30 degrees; This represents the angle sensitivity coefficient, used to adjust the rate attenuation of confidence as the deviation angle increases. It is calibrated by testing the pickup beamwidth. The coefficient is calculated to be 0.05 by setting the attenuation to half when the deviation is 45 degrees. This represents an exponential function with the natural constant as its base, used to convert linear angle differences into nonlinear suppression denominators, such that the suppression increases exponentially with increasing deviation, and the base of the constant is approximately 2.718. This represents the collaborative gain coefficient, which provides an additional confidence boost when the target is in the central region and the voiceprint matches. The highest value was found by grid search in a 1,000-hour test set, and this value was set to 0.15. The collaborative gain coefficient is used to probabilistically compensate for the confidence of the target's human voice. This represents a base-2 logarithmic function used to smooth the growth curve of the product value and prevent the compensation part from becoming too large and exceeding the limit. The input variable is 1 plus the product of probability and similarity.

[0067] In this embodiment, the specific steps for estimating background noise based on the target human voice confidence level to obtain the environmental background noise spectrum are as follows: extracting the background noise spectrum parameters of the previous frame from historical moments, calculating the frequency domain power spectrum parameters of the preliminary enhanced signal in the current frame at the current moment, and calculating the translational momentum update rate based on the target human voice confidence level, using the following formula: ,in, Indicates the translational sliding volume update rate. Represents the basic smoothing coefficient. Indicates the confidence level of the target human voice. The adaptive damping coefficient is used to protect speech features from being covered. The newly added noise component is obtained by multiplying the translational speed update rate by the current frame frequency domain power spectrum parameter. The difference between 1 and the translational speed update rate is multiplied by the previous frame background noise spectrum parameter to obtain the historical retained noise component. The newly added noise component and the historical retained noise component are summed to generate the current frame environmental background noise spectrum. The environmental background noise spectrum is obtained by integrating the current frame environmental background noise spectra of each frame.

[0068] Specifically, background noise is estimated based on the target human voice confidence level. The smoothed noise power matrix data, stored in the previous data processing cycle, is retrieved from the high-speed cache storage area of ​​memory. The background noise spectrum parameters of the previous frame are extracted. The square of the complex modulus of each frequency domain data component in the current frame is calculated to characterize the energy distribution intensity at the current physical moment. The frequency domain power spectrum parameters of the preliminary enhanced signal at the current frame are calculated. These parameters are then substituted into a specially designed mathematical update formula based on non-stationary state change rules to calculate the smoothed weight allocation coefficients that dynamically change with the environmental state. The smoothing momentum update rate is calculated based on the target human voice confidence level, using the following formula: ,in, This represents the smoothing momentum update rate, which determines the weight of the new noise component acquired in the current frame in the overall fusion update noise model. The value is limited to 0 to 1. The larger the value, the faster and more aggressive the noise model tracking update. It is calculated by adding and combining the basic smoothing component and the nonlinear adaptive damping component. The base smoothing coefficient is used as the basic rate index and minimum requirement for smoothing the update of the environmental noise spectrum. It is determined by comparing long-term recording tests of white noise environment in a stable state for up to 24 hours to find the best constant that can smoothly track the slowly changing background sound of the environment. It is fixed at 0.02. This represents the confidence level of the target human voice, which is the judgment value calculated and output by the pre-collaborative evaluation step, representing the probability of the target speech being present. For example, when C is calculated to be 0.9, it indicates that almost all of the current signal segment is valid target speech. This represents the adaptive damping coefficient, used to further suppress the update rate of the noise spectrum when the confidence value is high, indicating speech dominance. By analyzing and comparing a large number of audio segments containing high signal-to-noise ratio target speech, ensuring that the fluctuation range of the noise spectrum update error in strong speech segments does not exceed 5%, this coefficient is calculated and set to 0.01. The adaptive damping coefficient is used to protect speech features from being covered. The square root of the confidence value squared is added to the internally contained value 1 to construct a smooth, non-linear damping attenuation denominator curve. The attenuation coefficient value calculated by this formula is multiplied by the current time-instance power energy matrix obtained in the previous steps according to the corresponding positions, and the smoothing momentum update rate is multiplied by the current frame frequency domain. The power spectrum parameters are multiplied to obtain the newly added noise component. The complementary value of the update rate coefficient is calculated, which is the difference between the constant 1 and the update rate. The difference between 1 and the shifted update rate is multiplied with the background noise spectrum parameters of the previous frame to obtain the historically retained noise component. The feature matrices of these two parts are added and fused element by element according to row and column positions. The newly added noise component and the historically retained noise component are summed to generate the current frame's environmental background noise spectrum. The series of noise matrices generated after each short-time processing frame is collected into a sequence array and combined into a complete time-frequency matrix structure according to the processing time sequence. The environmental background noise spectrum is obtained by integrating the current frame's environmental background noise spectrum based on each frame.

[0069] In this embodiment, the steps for denoising the initial enhancement signal based on the ambient background noise spectrum and the target human voice confidence level to obtain a clean speech signal are as follows: The frequency band signal-to-noise ratio parameter is calculated using the ambient background noise spectrum; the gain function of the Wiener filter is modulated using the target human voice confidence level; a time-frequency masking matrix is ​​calculated using the Wiener filter; the time-frequency masking matrix is ​​multiplied with the initial enhancement signal to obtain the filtered enhancement spectrum; the filtered enhancement spectrum is inverted to the time domain using an inverse short-time Fourier transform; the boundary truncation effect is eliminated using an overlap-addition algorithm to generate a clean speech signal; and the clean speech signal is transmitted to a speech recognition engine for parsing.

[0070] Specifically, the initial enhanced signal is denoised based on the ambient background noise spectrum and the target human voice confidence level. The total power spectrum matrix of the current signal carrying interference noise is divided by the estimated updated ambient background noise power spectrum value at the corresponding frequency position. The prior signal-to-noise ratio (SNR) and the posterior SNR quantized data after smoothing by the decision-guided method are calculated in each sub-band. The frequency band SNR parameter is calculated using the ambient background noise spectrum. A dynamically changing threshold mapping function rule is constructed based on the target confidence probability derived from the above calculations. For example, when the confidence level is found to be higher than 0.8, it is determined that the voice is in a safe environment. The Wiener filter gain threshold is raised and limited to 0.2 to prevent over-filtering from damaging the fullness of normal speech. When the confidence level is found to be below 0.3, it is determined to be in a pure noise segment, and the gain threshold is lowered to 0.01 for strong suppression filtering. The Wiener filter gain function is modulated using the target human voice confidence level. The above signal-to-noise ratio data is substituted into the minimum mean square error filtering formula after adaptive modulation. For each basic unit domain determined by time and frequency coordinates, an energy retention ratio value between 0 and 1 is calculated. The nanofilter calculates the time-frequency masking matrix. This masking matrix grid is then multiplied by the complex spectral points containing signal amplitude and phase according to the grid intersection coordinates. This suppresses the energy peaks of frequencies identified as noise while preserving the energy characteristics of the speech bands. The time-frequency masking matrix is ​​multiplied by the initial enhancement signal to obtain the filtered enhancement spectrum. The basic algorithm of fast inverse discrete Fourier transform is called to integrate the complex sequence, converting the multidimensional frequency domain data back to the discrete time waveform point sequence axis. The filtered enhancement spectrum is then inverted to the time domain using an inverse short-time Fourier transform. For redundant data portions where adjacent segments overlap due to manual settings during frame segmentation, the amplitudes of data points within the overlapping waveform intervals are numerically added to restore the continuous and true physical signal waveform curve. The overlap-addition algorithm eliminates boundary truncation effects, generating a clean speech signal. The processed speech signal is then pushed to the backend semantic understanding and text conversion software program for subsequent operations via the transmission bus interface configured with the corresponding read transmission protocol. Finally, the clean speech signal is transmitted to the speech recognition engine for parsing.

[0071] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A deep learning-based method for filtering non-stationary noise in robots, characterized in that, Includes the following steps: Acquire an audio signal, perform frequency domain transformation on the audio signal, and obtain a multi-channel complex spectrum; The target direction is obtained by performing orientation estimation on the multi-channel complex spectrum; Beamforming is performed on the multi-channel complex spectrum based on the target direction to obtain a preliminary enhanced signal; The voiceprint embedding features of the preliminary enhanced signal are extracted, and human voice activity analysis is performed on the preliminary enhanced signal to obtain human voice probability parameters; The voiceprint embedding features are compared with the registered voiceprints to obtain the voiceprint matching similarity. By fusing the human voice probability parameters with the voiceprint matching similarity, a target human voice confidence score is generated. Background noise is estimated based on the target human voice confidence level to obtain the environmental background noise spectrum; The initial enhanced signal is denoised based on the ambient background noise spectrum and the target human voice confidence level to obtain a clean speech signal.

2. The deep learning-based method for filtering non-stationary noise in robots according to claim 1, characterized in that, The specific steps for acquiring an audio signal and performing frequency domain transformation on the audio signal to obtain a multi-channel complex spectrum are as follows: Multiple raw audio data streams are acquired using a microphone array, filtered using a low-pass filter to obtain filtered audio data, and the time series of the filtered audio data is extracted. The time series is then used to construct an audio signal, which is input into a pre-emphasis filter for high-frequency boosting to obtain a pre-emphasis time-domain signal. The pre-emphasis time-domain signal is then segmented into frames to obtain multiple overlapping data frames. The multiple overlapping data frames are then windowed and smoothed using a window function to obtain a windowed audio frame sequence. Finally, a short-time Fourier transform is performed on the windowed audio frame sequence to generate a multi-channel complex spectrum.

3. The deep learning-based method for filtering non-stationary noise in robots according to claim 1, characterized in that, The specific steps for directional estimation of the multi-channel complex spectrum to obtain the target direction are as follows: The cross-power spectral density parameters of each channel in the multi-channel complex spectrum are extracted, and the cross-power spectral density parameters are normalized using a phase transformation function to obtain the cross-correlation spectral parameters. The cross-correlation spectrum parameters are subjected to inverse Fourier transform to obtain a spatial time delay matrix. The peak energy distribution of the spatial time delay matrix is ​​analyzed, the maximum peak parameter is extracted, the maximum peak parameter is converted into spatial angular coordinates, the horizontal azimuth angle corresponding to the spatial angular coordinates is extracted, and the horizontal azimuth angle is input to the signal classification model for spatial spectrum search to obtain a high-resolution spatial spectrum. The direction vectors of the extreme points in the high-resolution spatial spectrum are extracted and determined as the target directions.

4. The deep learning-based method for filtering non-stationary noise in robots according to claim 1, characterized in that, The specific steps for beamforming the multi-channel complex spectrum based on the target direction to obtain a preliminary enhanced signal are as follows: An image sensor is used to acquire an environmental image. A neural network model is used to perform face recognition on the environmental image to obtain the face position coordinates. The face position coordinates are mapped to an acoustic coordinate system to obtain an auxiliary reference direction. The spatial angle between the target direction and the auxiliary reference direction is calculated. The target direction is calibrated using the spatial angle value to obtain a calibrated target direction. The noise covariance matrix of the multi-channel complex spectrum is extracted. The steering vector parameters are extracted based on the calibrated target direction. The noise covariance matrix and the steering vector parameters are solved using a distortion-free response algorithm to obtain a beamforming weight matrix. The beamforming weight matrix is ​​then multiplied by the multi-channel complex spectrum to generate a preliminary enhancement signal.

5. The deep learning-based method for filtering non-stationary noise in robots according to claim 1, characterized in that, The specific steps for extracting the voiceprint embedding features of the preliminary enhanced signal and performing voice activity analysis on the preliminary enhanced signal to obtain voice probability parameters are as follows: The initial enhancement signal is divided into multiple short-time analysis frames, and acoustic feature parameters of each short-time analysis frame are extracted. The acoustic feature parameters are input into a preset neural network for feature mapping, and a high-dimensional voiceprint representation sequence is output. The high-dimensional voiceprint representation sequence is reduced in dimensionality using a pooling function to generate voiceprint embedding features. The initial enhancement signal is input into the frequency domain layer of the human voice detection model to extract local feature maps. Logistic regression is performed on the local feature map using a fully connected layer to calculate the current frame probability of the preliminary enhanced signal in each of the short-time analysis frames. Based on the time series, all the current frame probabilities are spliced ​​and integrated to construct the human voice probability parameters.

6. The deep learning-based method for filtering non-stationary noise in robots according to claim 1, characterized in that, The specific steps for comparing the embedded voiceprint features with the registered voiceprint to obtain the voiceprint matching similarity are as follows: Extract the target user's identity information, load the registered voiceprint based on the identity information, calculate the vector inner product of the voiceprint embedding feature and the registered voiceprint, calculate the feature norm of the voiceprint embedding feature and the reference norm of the registered voiceprint, and divide the vector inner product by the product of the feature norm and the reference norm to obtain the cosine distance value. Load a pre-set interference model library, extract the global interference centroid parameter of the interference model library, calculate the Euclidean distance attenuation value between the voiceprint embedding feature and the global interference centroid parameter, and use the Euclidean distance attenuation value to negatively compensate the cosine distance value to obtain the voiceprint matching similarity.

7. The deep learning-based method for filtering non-stationary noise in robots according to claim 5, characterized in that, The specific steps for generating the target voice confidence score by fusing the human voice probability parameters with the voiceprint matching similarity are as follows: Extract the deviation angle between the target direction and the central axis of the preset area, extract the current frame probability from the human voice probability parameters, construct a collaborative confidence evaluation model, and jointly solve the model by inputting the deviation angle, the current frame probability, and the voiceprint matching similarity to generate the target human voice confidence score, as shown in the formula: ; in, This indicates the confidence level of the target human voice. Indicates the probability of the current frame. This indicates the similarity of the voiceprint matching. Indicates the deviation angle, Represents the angle sensitivity coefficient. This represents the collaborative gain coefficient, which is used to perform probability compensation on the confidence level of the target human voice.

8. The deep learning-based method for filtering non-stationary noise in robots according to claim 1, characterized in that, The specific steps for estimating background noise based on the target human voice confidence level to obtain the environmental background noise spectrum are as follows: Extract the background noise spectrum parameters of the previous frame from the historical time period, calculate the frequency domain power spectrum parameters of the preliminary enhanced signal in the current frame at the current time, and calculate the translational motion update rate based on the target human voice confidence level, using the following formula: ,in, This represents the translational sliding velocity update rate. Represents the basic smoothing coefficient. This indicates the confidence level of the target human voice. Indicates the adaptive damping coefficient; The adaptive damping coefficient is used to protect speech features from being covered. The current newly added noise component is obtained by multiplying the translational speed update rate by the current frame frequency domain power spectrum parameter. The difference between 1 and the translational speed update rate is multiplied by the previous frame background noise spectrum parameter to obtain the historically retained noise component. The current newly added noise component and the historically retained noise component are summed to generate the current frame ambient background noise spectrum. The ambient background noise spectrum is obtained by integrating the current frame ambient background noise spectra of each frame.

9. The deep learning-based method for filtering non-stationary noise in robots according to claim 1, characterized in that, The specific steps for denoising the preliminary enhanced signal based on the environmental background noise spectrum and the target human voice confidence level to obtain a clean speech signal are as follows: The frequency band signal-to-noise ratio parameter is calculated using the ambient background noise spectrum, the gain function of the Wiener filter is modulated using the target human voice confidence, and the time-frequency masking matrix is ​​calculated using the Wiener filter. The time-frequency masking matrix is ​​multiplied with the preliminary enhancement signal to obtain the filtered enhancement spectrum. The filtered enhancement spectrum is then inverted to the time domain by performing an inverse short-time Fourier transform. The boundary truncation effect is eliminated by using an overlap-add algorithm to generate a clean speech signal. The clean speech signal is then transmitted to the speech recognition engine for parsing.

Images