Mobile device and method for self-noise cancellation thereof

Abstract

The application provides a mobile device and a self-noise elimination method thereof, and relates to the technical field of noise elimination. The method comprises the following steps: collecting motion state data of the mobile device in real time; inputting the motion state data into a noise characteristic pre-judgment model to obtain noise characteristic parameters output by the noise characteristic pre-judgment model; the noise characteristic parameters comprise a target amplitude, a target frequency and a target phase of self-noise generated by the mobile device; based on the noise characteristic parameters, a reverse cancellation signal is generated, and a reverse cancellation component of the mobile device is controlled to output the reverse cancellation signal. The application actively cancels self-noise from the source by hardware, reduces the interference of complex multi-source self-noise in real time and in advance, lays a solid foundation for high-quality voice interaction of the mobile device in a harsh operating environment, realizes efficient and high-quality self-noise elimination, and ensures efficient and high-quality voice interaction.

Description

Technical Field

[0001] This invention relates to the field of noise reduction technology, and in particular to a mobile device and its self-noise cancellation method. Background Technology

[0002] With the continuous development of artificial intelligence technology, mobile devices (such as industrial inspection robots, home companion robots, and quadrupedal robot dogs) have been widely used in various practical scenarios. Voice interaction, as the most natural and convenient form of human-computer interaction, has become an indispensable core function of mobile devices. However, unlike smart devices in fixed environments (such as smart speakers and smart TVs), mobile devices generate significant self-noise during operation and movement due to the operation of internal joint motors, reducer friction, and the operation of other electronic components (such as radar). This self-noise highly overlaps with ambient speech in time and space, greatly masking normal speech signals and severely affecting the voice interaction and command recognition capabilities of mobile devices. Therefore, it is necessary to eliminate the self-noise in the voice interaction of mobile devices.

[0003] Currently, noise reduction for devices typically employs static noise reduction algorithms (such as single-channel spectral subtraction, Wiener filtering, or fixed beamforming) to post-process the audio stream captured by the microphone; or it utilizes a reference microphone to collect noise and perform conventional adaptive active noise reduction. However, in practical applications of mobile devices, noise reduction strategies generally remain at the passive noise reduction level of "first collecting sound, then filtering," heavily relying on the feedback of the sound signal. However, during movement, the posture and speed of mobile devices are often undergoing drastic dynamic changes, resulting in self-noise exhibiting high dynamic fluctuations in frequency, amplitude, and phase. Passive sound acquisition and static noise reduction models cannot match these drastically fluctuating self-noise characteristics in real time, exhibiting significant lag. When facing extreme dynamic conditions of high-speed, strongly coupled mobile devices, this often leads to noise reduction parameters failing to adapt in real time, a sharp drop in self-noise reduction effectiveness, or even severe distortion of the normal speech signal.

[0004] Therefore, improving the self-noise cancellation effect of mobile devices and avoiding voice signal distortion are technical problems that urgently need to be solved. Summary of the Invention

[0005] This invention provides a mobile device and a self-noise cancellation method thereof, which solves the defect of poor self-noise cancellation effect of mobile devices in the prior art and achieves efficient and high-quality self-noise cancellation.

[0006] This invention provides a method for self-noise cancellation in a mobile device, comprising:

[0007] The motion state data of the mobile device is collected in real time; the motion state data is correlated with the self-noise of the mobile device.

[0008] The motion state data is input into the noise feature prediction model to obtain the noise feature parameters output by the noise feature prediction model; the noise feature parameters include the target amplitude, target frequency, and target phase of the self-noise generated by the mobile device; the noise feature prediction model is trained based on the sample motion state data;

[0009] Based on the noise characteristic parameters, a reverse cancellation signal is generated, and the reverse cancellation component of the mobile device is controlled to output the reverse cancellation signal.

[0010] According to the present invention, a method for self-noise cancellation of a mobile device, wherein inputting the motion state data into a noise feature prediction model to obtain noise feature parameters output by the noise feature prediction model includes:

[0011] The motion state data is denoised using a Kalman filter algorithm to obtain denoised motion state data. The process noise covariance and / or observation noise covariance of the Kalman filter algorithm are dynamically adjusted based on the motion intensity coefficient of the mobile device, which is used to characterize the stability of the current motion state of the mobile device.

[0012] The noise-reduced motion state data is input into the noise feature prediction model to obtain the noise feature parameters output by the noise feature prediction model.

[0013] According to the present invention, a method for noise cancellation of a mobile device is provided, wherein the motion state data includes the acceleration of each joint of the mobile device and the three-dimensional linear acceleration of the fuselage;

[0014] The exercise intensity coefficient is determined based on the following method:

[0015] The first coefficient is determined based on the ratio of the maximum joint acceleration among the aforementioned joint accelerations to the maximum rated joint acceleration of the moving device.

[0016] A second coefficient is determined based on the ratio of the linear acceleration of the mobile device's fuselage to the maximum rated linear acceleration of the mobile device's fuselage; the linear acceleration of the fuselage is determined based on the three-dimensional linear acceleration of the fuselage.

[0017] The exercise intensity coefficient is obtained by weighted summation of the first coefficient and the second coefficient.

[0018] According to a self-noise cancellation method for a mobile device provided by the present invention, the reverse cancellation signal includes a reverse electromagnetic signal, and the reverse cancellation component includes a cancellation coil disposed on the radar of the mobile device;

[0019] The reverse cancellation component controlling the moving device outputs the reverse cancellation signal, including:

[0020] Controlling the cancelling coil to generate a reverse electromagnetic field to output the reverse electromagnetic signal; and / or,

[0021] The reverse cancellation signal includes a reverse acoustic wave signal, and the reverse cancellation component includes an audio output component disposed on the mobile device;

[0022] The reverse cancellation component controlling the moving device outputs the reverse cancellation signal, including:

[0023] Control the audio output component to generate a reverse sound wave to output the reverse sound wave signal; and / or,

[0024] After inputting the motion state data into the noise feature prediction model and obtaining the noise feature parameters output by the noise feature prediction model, the method further includes:

[0025] Based on the noise characteristic parameters, the target displacement of the reverse mechanical vibration is determined;

[0026] The calculation deviation is determined based on the target displacement and the actual displacement of the piezoelectric ceramic component located on the moving device.

[0027] The control voltage is determined based on the calculated deviation, and the piezoelectric ceramic component is driven to generate reverse mechanical vibration based on the control voltage.

[0028] According to a self-noise cancellation method for a mobile device provided by the present invention, after the reverse cancellation component controlling the mobile device outputs the reverse cancellation signal, it further includes:

[0029] Acquire the audio signal collected by the voice signal acquisition component of the mobile device;

[0030] The audio signal is subjected to spatial filtering and / or speech separation processing to eliminate self-noise signals in the audio signal and obtain the target speech signal.

[0031] According to the present invention, a self-noise cancellation method for a mobile device is provided, wherein the speech separation processing is performed as follows:

[0032] Audio features are extracted from the frequency domain signal of the audio signal to obtain an audio feature vector;

[0033] The audio feature vector is input into the speech separation model to obtain a new amplitude spectrum output by the speech separation model; the speech separation model is used to eliminate self-noise signals in the audio signal.

[0034] Based on the phase spectrum of the frequency domain signal and the new amplitude spectrum, the audio signal after speech separation processing is determined.

[0035] According to a noise cancellation method for a mobile device provided by the present invention, before inputting the audio feature vector into a speech separation model to obtain a new amplitude spectrum output by the speech separation model, the method further includes:

[0036] Based on the motion state data, the motion intensity of the mobile device is determined;

[0037] Based on the motion intensity, the noise suppression threshold of the speech separation model is adaptively adjusted; the noise suppression threshold is used to determine the amount of noise suppression and the amount of speech preservation.

[0038] According to the present invention, a self-noise cancellation method for a mobile device is provided, wherein the motion state data includes the joint rotation speed of the mobile device and the radar power of the mobile device; the step of determining the motion intensity of the mobile device based on the motion state data includes:

[0039] A third coefficient is determined based on the ratio of the joint rotation speed to the maximum joint rotation speed of the moving device;

[0040] A fourth coefficient is determined based on the ratio of the radar power to the maximum radar power of the mobile device;

[0041] The motion intensity of the mobile device is obtained by weighted summation of the third and fourth coefficients.

[0042] According to the present invention, a self-noise cancellation method for a mobile device is provided, wherein the spatial filtering method is as follows:

[0043] The direction of arrival (DOA) of the audio signal is estimated to obtain the current speech direction angle;

[0044] If the difference between the current speech direction angle and the previous speech direction angle is greater than or equal to a preset angle threshold, or if the magnitude of the residual of the beam weights is greater than or equal to a preset error threshold, the current beam weights are updated. The initial value of the beam weights is a minimum variance distortionless response (MVDR) beam weight determined based on the initial speech direction angle and the initial noise reference signal. The residual of the beam weights is determined based on the difference between the desired signal and the actual signal. The desired signal is determined based on the current speech direction angle and the audio signal. The actual signal is determined based on the current beam weights and the audio signal.

[0045] Based on the latest beam weights, the audio signal is weighted to obtain a spatially filtered audio signal.

[0046] According to a method for noise cancellation of a mobile device provided by the present invention, the motion state data includes at least one of the joint motion state data of the mobile device, the posture state data of the mobile device, and the radar operating state data of the mobile device.

[0047] The present invention also provides a noise cancellation device for a mobile device, comprising:

[0048] The data acquisition module is used to collect motion state data of the mobile device in real time; the motion state data is correlated with the self-noise of the mobile device.

[0049] The feature prediction module is used to input the motion state data into the noise feature prediction model to obtain the noise feature parameters output by the noise feature prediction model; the noise feature parameters include the target amplitude, target frequency and target phase of the self-noise generated by the mobile device; the noise feature prediction model is trained based on the sample motion state data;

[0050] The signal generation module is used to generate a reverse cancellation signal based on the noise characteristic parameters, and to control the reverse cancellation component of the mobile device to output the reverse cancellation signal.

[0051] The present invention also provides a mobile device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the self-noise cancellation method of any of the mobile devices described above.

[0052] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the self-noise cancellation method of the mobile device as described above.

[0053] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the self-noise cancellation method of any of the mobile devices described above.

[0054] The mobile device and its self-noise cancellation method provided by this invention collect motion state data of the mobile device in real time, thereby ensuring real-time prediction of noise feature parameters and real-time output of reverse cancellation signals, thus improving the self-noise cancellation effect and ensuring the acquisition of pure speech signals. The motion state data is input into a noise feature prediction model to obtain the noise feature parameters output by the model. Since there is a correlation between the motion state data and the self-noise of the mobile device, noise feature parameters can be predicted based on the motion state data. This successfully transforms the noise cancellation strategy from "passive adaptation" to "active prediction," solving the problem of dynamic noise adaptation lag, thereby improving the self-noise cancellation effect and ensuring the acquisition of pure speech signals. Voice signals; based on noise characteristic parameters, an inverse cancellation signal is generated, and the inverse cancellation component of the mobile device is controlled to output the inverse cancellation signal. This actively cancels self-noise at the source through hardware, overcoming the shortcomings of existing noise cancellation algorithms that are independent of the device's motion control unit. It establishes a close relationship between the noise cancellation strategy and the motion strategy. Even when facing extreme dynamic conditions such as extremely high speed and strong coupling of the mobile device, it can still accurately track the dynamic drift of the noise source, reducing complex and multi-source noise interference in real time and in advance. This lays a solid foundation for high-quality voice interaction of mobile devices in harsh operating environments, that is, to achieve efficient and high-quality self-noise cancellation and ensure efficient and high-quality voice interaction. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0056] Figure 1 This is one of the flowcharts illustrating the self-noise cancellation method for mobile devices provided by the present invention.

[0057] Figure 2 This is one of the structural schematic diagrams of the mobile device provided by the present invention.

[0058] Figure 3 This is the second structural schematic diagram of the mobile device provided by the present invention.

[0059] Figure 4 This is the second flowchart illustrating the self-noise cancellation method for mobile devices provided by the present invention.

[0060] Figure 5This is the third structural schematic diagram of the mobile device provided by the present invention.

[0061] Figure 6 This is a schematic diagram of the structure of the self-noise cancellation device for mobile devices provided by the present invention.

[0062] Figure 7 This is the fourth structural schematic diagram of the mobile device provided by the present invention. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0064] In practical applications of mobile devices such as mobile robots or robotic dogs, due to their complex structure and constant movement, their self-noise interference exhibits extreme characteristics of being "multi-source, dynamic, and strongly coupled," specifically manifested in the following three dimensions. First, the noise sources are extremely complex, including not only mechanical vibration noise from joint motors and friction noise from reducers, but also high-frequency electromagnetic radiation noise generated by radar (such as lidar and millimeter-wave radar) and airflow disturbance noise. In actual measurements, some radar noise peaks can even reach 60dB-80dB, a level that directly masks the normal human speech signal of approximately 40dB-60dB. Second, the noise and speech signal are highly spatiotemporally superimposed. During device movement, even minor changes in the device's posture (such as turning or climbing) can cause drastic dynamic fluctuations in the frequency and amplitude of self-noise. Because the self-noise and the target speech completely overlap in time and space, traditional noise reduction algorithms for fixed scenarios are simply inadequate. Third, the sound recording system is highly susceptible to deterioration due to motion. The microphone array will generate strong wind noise as the device moves rapidly, and the mechanical vibration of the device will be directly transmitted to the microphone as structural noise, further deteriorating the signal-to-noise ratio of the voice signal.

[0065] While existing technologies include single-channel noise suppression (such as spectral subtraction and Wiener filtering), fixed beamforming (such as a fixed 4-microphone array), and deep learning-based signal separation, these solutions are all developed for fixed scenarios (such as smart speakers) or simple mobile scenarios (such as mobile phone calls). When faced with the complex motion states of mobile robots / robot dogs, the following significant shortcomings are exposed.

[0066] The self-noise dynamic adaptability is extremely poor. Existing technologies heavily rely on static noise models and lack real-time parameter update mechanisms. For example, when the joint rotation speed of a robot dog instantly increases from 1 rad / s to 10 rad / s, the mechanical noise amplitude can increase by 3-5 times. Existing static models cannot match this millisecond-level surge in real time, directly leading to a sharp drop in noise reduction effect or speech distortion.

[0067] The noise reduction strategy is severely disconnected from the device's motion state. Existing noise reduction algorithms operate independently of the device's motion control unit. When the robot turns, causing a shift in the direction of the self-noise source, or when the robot dog climbs a slope, causing a drift in the joint noise spectrum, the noise reduction system lacks underlying motion state feedback and cannot synchronously adjust the suppression strategy, resulting in a large amount of residual noise in the new direction and frequency band.

[0068] The self-noise of mobile devices is a complex superposition of mechanical vibration, electromagnetic radiation, and wind noise. Existing single-dimensional designs (such as Wiener filtering, which only targets steady-state acoustic noise, and electromagnetic shielding, which only targets electromagnetic interference) lack multi-dimensional synergistic physical mechanisms, and a single technical means simply cannot cover all types of coupled noise.

[0069] The system lacks robustness in extreme scenarios. When the robot is in high-speed motion (e.g., joint rotation speed > 8 rad / s) or when the radar is scanning at high power, the self-noise intensity often exceeds 80 dB. Existing noise reduction models have fixed suppression thresholds, making it impossible to achieve a balance between "strong noise suppression" and "weak speech detail preservation," which easily leads to the over-filtering of normal speech signals or the leakage of strong noise.

[0070] To address the aforementioned problems, this invention breaks away from the traditional passive approach of "static noise reduction" and "acquisition followed by filtering," proposing a technical system of "noise prediction, multi-source separation, and dynamic adaptation" to achieve accurate extraction of speech signals in mobile states. Specifically, the invention proposes the following embodiments. The following is a detailed explanation... Figures 1-5 The present invention describes a method for noise cancellation in a mobile device.

[0071] Figure 1 This is one of the flowcharts illustrating the self-noise cancellation method for mobile devices provided by the present invention, such as... Figure 1 As shown, the self-noise cancellation method of the mobile device includes the following steps 110, 120 and 130.

[0072] Step 110: Collect motion status data of the mobile device in real time.

[0073] Here, "mobile device" refers to a movable device, primarily meaning an equipment whose internal mechanical parts or electronic components generate dynamic operating noise during operation and movement. Examples include mobile robots (such as industrial inspection robots and home companion robots) or robotic dogs (such as quadrupedal robotic dogs). Self-noise refers to the interference sounds generated by the mobile device itself, specifically including mechanical vibration noise from joint motors, friction noise from reducers, electromagnetic radiation noise from radar operation, and airflow disturbance noise.

[0074] The motion state data is correlated with the self-noise of the mobile device. For example, the self-noise of the mobile robot / robot dog (such as joint mechanical noise, radar electromagnetic noise, etc.) is strongly correlated with the device's motion state (joint rotation speed, radar power, etc.). For instance, for every 2 rad / s increase in joint rotation speed, the mechanical noise amplitude increases by approximately 1 dB, meaning that an increase in radar transmission power directly leads to an increase in the electromagnetic radiation noise amplitude.

[0075] In one embodiment, motion state data refers to a set of data characterizing the current physical kinematics of the mobile device and the operating characteristics of its electronic components.

[0076] The motion state data may include, but is not limited to, at least one of the following: joint motion state data of the mobile device, posture state data of the mobile device, and radar operating state data of the mobile device, etc.

[0077] The joint motion status data may include, but is not limited to, at least one of the following: joint angle, instantaneous joint rotational speed (joint rotational speed), joint acceleration, etc. The attitude status data may include, but is not limited to, at least one of the following: yaw angle, pitch angle, roll angle, fuselage three-dimensional linear acceleration, etc. The radar operating status data may include, but is not limited to, at least one of the following: radar start / stop status, radar transmit power (radar power), radar operating frequency, etc.

[0078] Furthermore, the motion state data is timestamped for sorting. Data with a timestamp deviation ≤3ms is considered to be data from the same moment, while data with a deviation greater than 3ms is supplemented by linear interpolation to ensure data temporal consistency and achieve time synchronization of multi-source data. For example, the timestamp generation is based on the high-precision clock of the mobile device's main control module, generating a timestamp every 1ms, which is appended in real time when the sensor collects data.

[0079] Furthermore, a dual-link transmission of motion state data is employed to achieve time synchronization of multi-source data. For example, the dual-link transmission includes two links: one is the transmission of encoder (joint angle) and IMU (Inertial Measurement Unit) data via CAN (Controller Area Network) bus, and the other is the transmission of radar status data via Ethernet.

[0080] For example, such as Figure 2 As shown, the mobile device (robot dog) includes a millimeter-wave radar, an inertial measurement unit (IMU), and a high-precision encoder. The millimeter-wave radar's radar status feedback unit provides feedback on core parameters such as start / stop status, transmit power, and operating frequency. The IMU integrates accelerometer and gyroscope data to output the device's attitude angle and motion acceleration in real time. The high-precision encoder collects joint rotation pulse signals and converts them into rotational speed. In other words, the mobile device can capture its motion state in real time through multiple sensors to obtain motion state data, such as through a multi-sensor fusion module (encoder + IMU + radar status sensor) to collect motion state data in real time.

[0081] It should be noted that motion state data is collected in real time, and a reverse cancellation signal is output in real time to achieve real-time dynamic prediction and real-time noise cancellation, such as millisecond-level dynamic prediction and cancellation, thereby improving the noise cancellation effect and ensuring that a clean speech signal is collected.

[0082] It should be understood that by extracting the kinematic and operational status data of the mobile device in real time across the physical domain through multiple sensors (such as encoders, IMUs, and radar status sensors), a pure reference source that is not affected by ambient sound is provided for the subsequent noise reduction system. This breaks the limitation of traditional noise reduction that relies solely on acoustic feedback and establishes a tracking anchor point for the dynamic changes of self-noise from the source.

[0083] Step 120: Input the motion state data into the noise feature prediction model to obtain the noise feature parameters output by the noise feature prediction model.

[0084] The noise feature prediction model is trained based on sample motion state data. In one embodiment, this noise feature prediction model is a deep learning network model (such as a CNN-LSTM hybrid network architecture) used to establish a nonlinear mapping relationship between multidimensional physical motion states and acoustic / electromagnetic noise features. After the input motion state data is processed by the network to extract spatial features and long-term temporal correlation features, it regresses and outputs three-dimensional core parameters, i.e., noise feature parameters. In one embodiment, this noise feature prediction model is trained and converged by pre-collecting a large number of motion-noise pairing datasets (i.e., sample motion state data and their corresponding real noise feature parameter labels).

[0085] Furthermore, the feature vectors of the motion state data are standardized (i.e., mapped to the [0, 1] interval) and then input into the noise feature prediction model. For example, the input feature vector is 12-dimensional, including: joint rotation angle, joint rotation speed, joint acceleration, yaw angle, pitch angle, roll angle, fuselage X-axis acceleration, fuselage Y-axis acceleration, fuselage Z-axis acceleration, radar start / stop status, radar transmit power, and radar operating frequency.

[0086] In one embodiment, the noise feature prediction model includes a feature extraction layer and a prediction layer connected in sequence. Further, the feature extraction layer includes a CNN (Convolutional Neural Network) feature extraction layer and an LSTM (Long Short-Term Memory) temporal capture layer connected in sequence, and the prediction layer includes a fully connected layer.

[0087] The CNN feature extraction layer is primarily used for spatial feature extraction. In one embodiment, the CNN feature extraction layer comprises three convolutional layers, which extract locally correlated features (such as the correlation between joint rotation speed and acceleration, and the correlation between radar power and frequency) from the feature vector through convolutional kernels. Furthermore, each layer is paired with a Batch Normalization (BN) layer and a ReLU activation function to prevent overfitting. For example, the output of the three convolutional layers is flattened to convert the 12×64 feature map into a 768-dimensional vector, and then compressed into a 128-dimensional spatial feature vector by a fully connected layer, which serves as the input to the LSTM temporal capture layer. For example, the compression formula is as follows:

[0088] ;

[0089] In the formula, Represents the compressed spatial feature vector; This represents the weight matrix of a fully connected layer, which is a 128×768 weight matrix; Indicates an expand operation; This represents the feature vector output by three convolutional layers; This represents a 128-dimensional bias vector.

[0090] The LSTM temporal capture layer is primarily used for capturing long-term temporal correlations. In one embodiment, the LSTM temporal capture layer includes two bidirectional LSTM units, with the core function of capturing long-term temporal correlations of noisy features. For example, each LSTM layer contains 256 LSTM units (each LSTM unit includes a gating unit), with an input dimension of 128 dimensions (the spatial feature vector output by the CNN) and a time step of 50 (i.e., each input sample contains 128-dimensional features across 50 consecutive time points). The forward LSTM captures future temporal features, the backward LSTM captures historical temporal features, and the output dimension is 2 × 256 = 512 dimensions.

[0091] The fully connected layer is primarily used for feature regression. For example, the 512-dimensional feature vector output from the LSTM temporal capture layer is compressed to 3 dimensions, corresponding to the noise amplitude (dB), frequency (Hz), and phase (rad), respectively. A linear activation function is used to ensure accurate feature regression. For instance, the processing formula is as follows:

[0092] ;

[0093] In the formula, , Indicates the target amplitude. Indicates the target frequency. Indicates the target phase; This represents the weight matrix of the fully connected layer, which is a 3×512 weight matrix that is iteratively optimized through model training. This represents the 512-dimensional hidden state vector output by the LSTM timing capture layer; This represents a 3D bias vector.

[0094] For example, the mean squared error (MSE) loss function is used to calculate the mean squared difference between the predicted value and the true value. The Adam optimizer is used with a learning rate of 0.001, a decay coefficient of 0.9, a batch size of 32, and 100 iterations. The training is carried out until the loss value stabilizes below 0.02 and the prediction error on the test set is ≤5%.

[0095] The noise characteristic parameters include the target amplitude, target frequency, and target phase of the self-noise generated by the mobile device. The target amplitude represents the predicted energy intensity of the self-noise at the current moment; the target frequency represents the predicted vibration or radiation frequency of the self-noise at the current moment, such as the high-frequency harmonic frequency of a joint motor or the operating frequency of a radar; and the target phase represents the predicted initial phase angle of the self-noise waveform at the current moment.

[0096] Furthermore, the Kalman filter algorithm is used to denoise the motion state data to obtain denoised motion state data. The denoised motion state data is then input into the noise feature prediction model to obtain the noise feature parameters output by the noise feature prediction model.

[0097] It should be understood that the embodiments of the present invention achieve millisecond-level dynamic prediction of self-noise characteristics. During the movement of a mobile device (such as turning or climbing), its self-noise characteristics undergo high dynamic fluctuations, which traditional static models cannot match in real time. Therefore, by directly outputting physical-level amplitude, frequency, and phase through model inference, the noise pattern of the next millisecond can be known in advance, successfully transforming the noise reduction strategy from "passive adaptation" to "active prediction," thus solving the technical problem of lagging dynamic noise adaptation.

[0098] Step 130: Based on the noise characteristic parameters, generate a reverse cancellation signal and control the reverse cancellation component of the mobile device to output the reverse cancellation signal.

[0099] Here, the reverse cancellation signal is a signal constructed based on the principle of wave interference cancellation. Specifically, based on the target amplitude, target frequency, and target phase, a reverse cancellation signal is generated that has the same amplitude and frequency as the original noise, but a phase difference of π (i.e., 180 degrees). For example, the reverse cancellation signal includes a reverse electromagnetic signal or a reverse acoustic wave signal.

[0100] Here, the reverse cancellation component is a hardware unit that performs cancellation at the physical, acoustic, or electromagnetic levels. For example, when the reverse cancellation signal is a reverse mechanical vibration signal, the reverse cancellation component can be a piezoelectric ceramic component (piezoelectric ceramic actuator) used to output reverse mechanical vibration to cancel the transmission of body vibration to the microphone structure; when the reverse cancellation signal is a reverse electromagnetic signal, the reverse cancellation component can be a cancellation coil used to generate a reverse electromagnetic field to attenuate radar electromagnetic radiation noise; when the reverse cancellation signal is a reverse acoustic signal, the reverse cancellation component can be a loudspeaker.

[0101] It should be understood that by significantly advancing the noise reduction process, multi-dimensional inverse cancellation components directly intervene at the physical and electromagnetic levels, weakening the noise intensity at the source of audio acquisition components (such as microphones) (e.g., reducing mechanical transmission noise by more than 80% and canceling electromagnetic noise by more than 40%). This significantly reduces the computational burden on subsequent speech separation algorithms and effectively solves the technical problem of weak multi-source noise synthesis and suppression capabilities. Specifically, it uses deep learning to predict noise characteristic parameters to generate inverse cancellation signals to reduce noise at the source, breaking the traditional passive noise reduction mode of "acquisition first, then filtering." By advancing the noise reduction process, it significantly reduces the pressure on subsequent signal processing and can update noise characteristic parameters in real time according to noise changes, shifting from "passive adaptation" to "active prediction," thus solving the dynamic noise adaptation problem.

[0102] The self-noise cancellation method for mobile devices provided in this invention collects motion state data of the mobile device in real time, thereby ensuring real-time prediction of noise feature parameters and real-time output of inverse cancellation signals, thus improving the self-noise cancellation effect and ensuring the acquisition of clean speech signals. The motion state data is input into a noise feature prediction model to obtain the noise feature parameters output by the model. Since there is a correlation between the motion state data and the self-noise of the mobile device, noise feature parameters can be predicted based on the motion state data. This successfully transforms the noise cancellation strategy from "passive adaptation" to "active prediction," solving the problem of dynamic noise adaptation lag, thereby improving the self-noise cancellation effect and ensuring clean speech signals. The system generates an inverse cancellation signal based on noise characteristic parameters and controls the inverse cancellation component of the mobile device to output the inverse cancellation signal. This actively cancels self-noise at the source through hardware, overcoming the shortcomings of existing noise cancellation algorithms that are independent of the device's motion control unit. It establishes a close relationship between the noise cancellation strategy and the motion strategy. Even when facing extreme dynamic conditions such as extremely high speed and strong coupling of the mobile device, it can still accurately track the dynamic drift of the noise source and reduce complex and multi-source noise interference in real time and in advance. This lays a solid foundation for high-quality voice interaction of the mobile device in harsh operating environments, namely, achieving efficient and high-quality self-noise cancellation and ensuring efficient and high-quality voice interaction.

[0103] Based on any of the above embodiments, in this method, step 120 includes:

[0104] The motion state data is denoised using the Kalman filter algorithm to obtain denoised motion state data.

[0105] The noise-reduced motion state data is input into the noise feature prediction model to obtain the noise feature parameters output by the noise feature prediction model.

[0106] Here, the Kalman filter algorithm is an algorithm that uses the state equation of a linear system to optimally estimate the system state based on the system's input and output observation data. Since sensors on mobile devices (such as encoders and IMUs) inevitably contain random errors when collecting motion state data (e.g., encoders have ±5% physical angle error, and IMUs have ±10% acceleration error), directly inputting the raw data containing these errors into the noise feature prediction model would lead to severe deviations in the output noise feature parameters. Therefore, the Kalman filter algorithm is used to achieve data noise reduction through an iterative process of "state prediction - observation update".

[0107] The process noise covariance and / or observation noise covariance of the Kalman filter algorithm are dynamically adjusted based on the motion intensity coefficient of the mobile device, which is used to characterize the stability of the current motion state of the mobile device.

[0108] In the Kalman filter algorithm, there are two core covariances: process noise covariance (usually denoted as Q) and observation noise covariance (usually denoted as R). The process noise covariance typically reflects the uncertainty of the system state transition model itself or the variance of external unknown disturbances; the observation noise covariance typically reflects the variance of the sensor's inherent measurement error.

[0109] Here, the motion intensity coefficient is a quantitative indicator that characterizes the stability of the current physical motion state of the mobile device. In one embodiment, the value range of the motion intensity coefficient is usually normalized to [0, 1]. The closer the value is to 1, the more intense the current motion of the mobile device (such as running at high speed, making sharp turns, etc.), and the closer the value is to 0, the more stable the motion (such as being stationary, traveling in a straight line at a constant speed, etc.).

[0110] It should be noted that in conventional Kalman filtering algorithms, the process noise covariance Q and observation noise covariance R are usually set as fixed constants. However, in the complex application scenarios of mobile devices, fixed parameters cannot adapt to the changing motion states. Therefore, this embodiment of the invention specifies that the process noise covariance and / or observation noise covariance of the Kalman filtering algorithm are dynamically adjusted according to the motion intensity coefficient of the mobile device.

[0111] For example, taking joint rotation speed as an example, the Kalman filtering process is divided into two stages: state prediction and observation update. The iteration period is consistent with the sensor sampling period (e.g., 1ms). First, the state prediction stage is as follows:

[0112] ;

[0113] ;

[0114] In the formula, This represents the predicted state quantity at time k based on time k-1; Represents the state transition matrix; This represents the filtered state quantity at time k-1; Represents the control input matrix (when there is no external control). ), For control input (when there is no control) ); This represents the predicted state covariance matrix at time k; This represents the state covariance matrix at time k (reflecting the uncertainty of the state estimate). Represents the process noise covariance;

[0115] Secondly, the observation update phase is as follows:

[0116] ;

[0117] ;

[0118] ;

[0119] In the formula, Indicates the Kalman gain at time k; Represents the observation matrix; Represents the observation noise covariance; This represents the filtered state quantity at time k; This represents the sensor observation value at time k; Represents the state covariance matrix at time k; It is an identity matrix.

[0120] In one specific embodiment, the motion state data includes the acceleration of each joint of the mobile device and the three-dimensional linear acceleration of the fuselage. The motion intensity coefficient is obtained by weighting and calculating the joint acceleration and the three-dimensional linear acceleration of the fuselage.

[0121] In one embodiment, a motion intensity coefficient threshold (e.g., 0.5) is preset. When the motion intensity coefficient is greater than or equal to the motion intensity coefficient threshold (determined as motion intensity), the values ​​of process noise covariance Q and / or observation noise covariance R are actively increased to improve the response speed. When the motion intensity coefficient is less than the motion intensity coefficient threshold (determined as motion stability), the values ​​of process noise covariance Q and / or observation noise covariance R are decreased.

[0122] In another embodiment, a positive correlation mathematical mapping function is constructed between the process noise covariance Q and / or the observation noise covariance R and the motion intensity coefficient. As the motion intensity coefficient gradually increases, the Q or R value increases proportionally; as the motion intensity coefficient gradually decreases, the Q or R value decreases proportionally.

[0123] Furthermore, before using the Kalman filter algorithm to denoise the motion state data, time synchronization of the motion state data can also be performed. Specifically, a "timestamp marking + dual-link transmission" mechanism is adopted. The timestamp is generated based on the high-precision clock of the mobile device's main control module. Encoder and IMU data are transmitted via a CAN bus link, and radar status data is transmitted via an Ethernet link. The data receiving end sorts the data based on the timestamp. Data with timestamp deviations within a preset minimum range (e.g., ≤3ms) is considered as synchronized data at the same moment. Data with deviations exceeding this range is supplemented by linear interpolation, thereby ensuring strict consistency in timing of multi-source data.

[0124] In one specific embodiment, the high-precision, smooth, and phase-shift-free motion state data obtained after Kalman filtering adaptive noise reduction is used as a multi-dimensional feature vector and input into a noise feature prediction model composed of a deep learning network architecture for forward inference calculation, thereby outputting high-accuracy noise feature parameters.

[0125] It should be understood that by dynamically binding the process noise covariance and / or observation noise covariance to the motion stunt coefficient, the Kalman filter algorithm is endowed with adaptive adjustment capabilities in complex moving scenarios. When the moving device moves smoothly, the process noise covariance is reduced to improve the steady-state smoothing accuracy of the data; when the moving device moves violently, the system state changes extremely rapidly, and the process noise covariance is increased to improve the transient response speed of the filter. This dynamic adjustment mechanism effectively prevents the filter divergence or response hysteresis phenomenon caused by traditional filters when the equipment suddenly moves violently, and significantly compresses the error range of sensor data.

[0126] It should be understood that this provides a high-quality input data source for the noise feature prediction model. Since the noise feature prediction model is extremely sensitive to noise in the input features, high-quality denoised data can prevent the model from amplifying errors due to random fluctuations in the input features during inference, ensuring the absolute accuracy of the self-noise feature parameters output by the noise feature prediction model in both the time and frequency domains.

[0127] The self-noise cancellation method for mobile devices provided in this invention embeds an adaptive Kalman filter noise reduction mechanism dynamically adjusted by the motion intensity coefficient before inputting motion state data into the noise feature prediction model. This not only eliminates random errors in the underlying hardware of different types of sensors, but more importantly, it overcomes the technical defect of traditional fixed-parameter filtering algorithms that are prone to failure when faced with extreme conditions such as high-frequency maneuvering and violent acceleration and deceleration of mobile devices. This achieves a perfect balance between filtering response speed and steady-state accuracy, ensuring the absolute accuracy and robustness of the entire self-noise cancellation in the front-end data acquisition stage. This provides an extremely reliable data base for the subsequent generation of accurate reverse cancellation signals, thereby improving the self-noise cancellation effect and ensuring the acquisition of pure speech signals.

[0128] Based on any of the above embodiments, in this method, the motion state data includes the joint accelerations and three-dimensional linear accelerations of the mobile device. Accordingly, the motion intensity coefficient is determined based on the following method:

[0129] The first coefficient is determined based on the ratio of the maximum joint acceleration among the aforementioned joint accelerations to the maximum rated joint acceleration of the moving device.

[0130] A second coefficient is determined based on the ratio of the linear acceleration of the mobile device's fuselage to the maximum rated linear acceleration of the mobile device's fuselage; the linear acceleration of the fuselage is determined based on the three-dimensional linear acceleration of the fuselage.

[0131] The exercise intensity coefficient is obtained by weighted summation of the first coefficient and the second coefficient.

[0132] It should be noted that robots or robot dogs typically have multiple joints, therefore motion data includes accelerations from multiple joints. The three-dimensional linear accelerations of the robot body include the X-axis acceleration, Y-axis acceleration, and Z-axis acceleration.

[0133] Here, joint acceleration refers to the angular acceleration of each moving joint of the mobile device at the current moment. It directly reflects the degree of drastic change in the instantaneous output torque of the joint motor and is the direct physical source of mechanical friction noise and high-frequency howling of the reducer. Maximum joint acceleration refers to the largest absolute value of acceleration among all joints of the mobile device at the same sampling moment. Maximum rated joint acceleration refers to the limit acceleration physical quantity that the motor can withstand, set in the factory hardware parameters of the mobile device.

[0134] Here, the fuselage linear acceleration is the global acceleration amplitude determined by vector synthesis (e.g., calculating the square root of the sum of squares) of the three-dimensional linear accelerations of the fuselage. The maximum rated linear acceleration of the fuselage is a limiting physical parameter set according to the overall motion performance of the mobile device.

[0135] In one embodiment, the maximum rated acceleration of the joint is preset based on the joint parameters of the moving device. The maximum rated linear acceleration of the fuselage is preset based on the motion performance of the moving device, for example, 5 m / s². 2 .

[0136] Here, the first coefficient is normalized to characterize the degree to which the motion limit of the local mechanical structure of the mobile device is approximated. Based on this, it is possible to accurately and sensitively capture instantaneous violent movements occurring in local parts of the mobile device (such as the hip joint of a leg). By extracting the maximum value for calculation, it is ensured that the fastest response can be made to abnormal mechanical noise fluctuations of any single joint, avoiding the feature overload caused by multi-joint averaging calculations.

[0137] Here, the second coefficient is used to characterize the intensity of the overall attitude and spatial displacement of the mobile device. Based on this, the global motion state of the mobile device is comprehensively quantified. The intense motion of the entire fuselage (such as running, jumping, and sudden braking) is the main cause of structural vibration transmission and rapid changes in external wind noise. The introduction of the second coefficient enables accurate assessment of the dynamic fluctuations of noise at the macroscopic spatial level.

[0138] In one embodiment, joint movement has a greater impact on noise, therefore the weight of the first coefficient is greater than the weight of the second coefficient. Specifically, since the mechanical engagement and high-speed rotation of the joint motors have a much greater impact on noise intensity than the air disturbances generated by the overall movement of the machine body in terms of generating device self-noise, the local mechanical weight is usually set to be greater than the global attitude weight when setting the weight coefficients, so as to ensure that the numerical change curve of the motion intensity coefficient can perfectly match the acoustic change curve of the mechanical self-noise.

[0139] For example, the formula for calculating the exercise intensity coefficient is as follows:

[0140] ;

[0141] In the formula, Indicates the intensity coefficient of exercise. This indicates the weight of the first coefficient. Usually greater than 0.5, This indicates the weight of the second coefficient. This represents the maximum joint acceleration. This indicates the maximum rated acceleration of the joint. This indicates the fuselage's X-axis acceleration. This indicates the fuselage's Y-axis acceleration. This indicates the Z-axis acceleration of the fuselage. This indicates the maximum rated linear acceleration of the fuselage.

[0142] It should be understood that by using weighted summation, the local and global motion characteristics are scientifically integrated, outputting a motion intensity coefficient that accurately represents the stability of the current motion state of the device. This provides a standardized, dimensionless, and continuous adjustment factor for the process noise covariance and observation noise covariance of the Kalman filter algorithm.

[0143] The self-noise cancellation method for mobile devices provided in this invention provides a specific method for determining the motion intensity coefficient. This method overcomes the shortcomings of existing technologies that rely solely on a single sensor to comprehensively assess motion status. By comprehensively considering both "local displacements that induce high-frequency mechanical noise" and "global displacements that induce low-frequency structural vibrations," and by assigning a weighting mechanism that conforms to acoustic laws, a highly scene-adaptable multi-source heterogeneous data evaluation model is constructed. This allows the output motion intensity coefficient to most realistically and sensitively reflect the transient physical deterioration of device self-noise, thus providing a solid and accurate mathematical criterion for the dynamic adaptive adjustment of the underlying Kalman filter. This significantly improves the input data quality of the subsequent noise feature prediction model, further enhancing the self-noise cancellation effect and ensuring the acquisition of clean speech signals.

[0144] Based on any of the above embodiments, in this method, the reverse cancellation signal includes a reverse electromagnetic signal, and the reverse cancellation component includes a cancellation coil of a radar disposed on the mobile device; controlling the reverse cancellation component of the mobile device to output the reverse cancellation signal includes:

[0145] The cancelling coil is controlled to generate a reverse electromagnetic field in order to output the reverse electromagnetic signal.

[0146] It should be noted that, due to the "multi-source, strongly coupled" characteristics of self-noise in mobile devices (such as mobile robots or robotic dogs) (including high-frequency electromagnetic radiation noise from electronic components such as radar, mechanical vibration noise from joint motors, and acoustic noise propagating through the air), traditional pure software post-filtering algorithms are difficult to cope with extremely high-intensity multi-source coupled interference. Therefore, this invention constructs a hardware-level active pre-cancellation scheme that includes three parallel or combined methods: electromagnetic cancellation, acoustic cancellation, and mechanical vibration cancellation. Specifically, it employs a dual-path design of "electrical signal cancellation + physical vibration cancellation," targeting both electromagnetic and mechanical noise respectively, reducing noise intensity by more than 40% at the source.

[0147] Here, the reverse electromagnetic signal refers to a digital or analog electrical signal output by a noise characteristic prediction model that has the same amplitude as the radar electromagnetic radiation noise but is 180 degrees (π) out of phase.

[0148] In one specific embodiment, the cancellation coil is a hardware divergence component deployed inside or around the shield of a mobile device radar (such as lidar or millimeter-wave radar).

[0149] Specifically, the generated reverse electromagnetic signal is applied to the cancellation coil via a control circuit, causing the cancellation coil to generate a reverse electromagnetic field in space according to the law of electromagnetic induction. This reverse electromagnetic field is then physically vector-superimposed on the original electromagnetic radiation field generated by the radar during operation within the spatial region.

[0150] For example, the time-domain signal of radar electromagnetic noise can be modeled as a sinusoidal signal, as shown below:

[0151] ;

[0152] In the formula, express The radar electromagnetic noise signal at any given moment; Indicates the amplitude of radar electromagnetic noise; This indicates the frequency of radar electromagnetic noise (usually the same as the radar's operating frequency). Indicates the phase of radar electromagnetic noise;

[0153] Based on the above, reverse electromagnetic signal The formula for generating is shown below:

[0154] ;

[0155] Furthermore, the total signal after superposition This achieves electromagnetic noise cancellation.

[0156] For example, electromagnetic noise cancellation is performed in hardware. First, a reverse electromagnetic signal is generated: noise characteristic parameters are received by a DSP (Digital Signal Processing) chip, and a digital domain reverse signal is generated using direct digital frequency synthesis technology; second, the digital signal is converted into an analog signal by a 16-bit D / A converter; then, the analog signal is amplified by an operational amplifier and transmitted to a cancellation coil built into the radar shield. The coil generates a reverse electromagnetic field to reduce the intensity of electromagnetic noise radiated from the radar to the microphone area from 60dB-80dB to 36dB-48dB, with a cancellation efficiency of ≥40%.

[0157] It should be understood that this invention achieves source-level hardware-level suppression of self-noise from high-frequency electromagnetic radiation. Electromagnetic interference generated during radar operation is a type of non-acoustic noise that is extremely difficult to eliminate using conventional acoustic microphones. This embodiment of the invention, through the superposition of spatially reversed electromagnetic fields, directly weakens the intensity of electromagnetic noise radiated from the radar to the microphone area along the electromagnetic wave propagation path, greatly improving the purity of the analog signal acquired by the microphone.

[0158] In this method, the reverse cancellation signal includes a reverse acoustic wave signal, and the reverse cancellation component includes an audio output component disposed on the mobile device; controlling the reverse cancellation component of the mobile device to output the reverse cancellation signal includes:

[0159] The audio output component is controlled to generate an inverted sound wave to output the inverted sound wave signal.

[0160] Here, the inverted acoustic signal refers to the phase-inverted audio electrical signal generated based on predicted noise characteristic parameters (such as joint friction, airborne acoustic noise generated by reducer operation). The audio output component is typically a speaker or miniature sound unit located near the mobile device body or pickup array.

[0161] In one specific embodiment, the main control unit drives the audio output component (such as a speaker) to work, causing its diaphragm to vibrate and release a reverse sound wave into the air that is opposite in phase to the ambient noise, thereby forming an acoustic interference cancellation region near the microphone array pickup area.

[0162] For example, the acoustic signal modeling of mechanical vibration noise is as follows:

[0163] ;

[0164] In the formula, express Acoustic signals of constant mechanical vibration noise; This indicates the acoustic noise amplitude of the acoustic signal; This represents the acoustic noise frequency of the acoustic signal (usually the vibration frequency, which is positively correlated with the joint rotation speed). , Indicates the harmonic order. Take 1-5, (Indicates joint rotation speed); This indicates the acoustic noise phase of the acoustic signal;

[0165] Based on the above, the reverse acoustic signal The formula for generating it is as follows:

[0166] ;

[0167] Furthermore, by releasing a reverse sound wave through an audio output component (such as a speaker), which superimposes and cancels out the noise sound wave, the acoustic noise amplitude at the audio acquisition component (such as a microphone) is reduced by more than 30%.

[0168] It should be understood that the embodiments of the present invention construct an active noise control barrier in the acoustic dimension. By directly neutralizing the low- and mid-frequency mechanical operating noise in the air medium through the audio output component, the self-noise level reaching the microphone diaphragm is physically reduced, effectively alleviating the noise reduction pressure of subsequent digital signal processing.

[0169] In this method, after step 120 above, the method further includes:

[0170] Based on the noise characteristic parameters, the target displacement of the reverse mechanical vibration is determined;

[0171] The calculation deviation is determined based on the target displacement and the actual displacement of the piezoelectric ceramic component located on the moving device.

[0172] The control voltage is determined based on the calculated deviation, and the piezoelectric ceramic component is driven to generate reverse mechanical vibration based on the control voltage.

[0173] It should be noted that the mechanical vibrations of the fuselage are directly transmitted to the audio acquisition components (such as microphones) through the solid structure, creating structural noise that is extremely difficult to eliminate. Therefore, determining the target displacement is necessary to completely counteract the transient mechanical vibrations of the fuselage, requiring precise physical deformation or displacement of the components to be counteracted.

[0174] In one specific embodiment, the target displacement is determined directly by converting the vibration amplitude and frequency parameters output by the noise feature prediction model. For example, the formula for determining the target displacement is as follows:

[0175] ;

[0176] In the formula, express The target displacement at any given moment; This indicates the vibration amplitude (positively correlated with joint acceleration). , , (Indicates joint acceleration); Indicates the vibration frequency (consistent with the joint rotation speed); Indicates the vibration phase.

[0177] Here, the piezoelectric ceramic component is a solid actuator based on the piezoelectric effect, which produces extremely precise micron-level mechanical deformation when a voltage is applied.

[0178] In one specific embodiment, the control process employs negative feedback closed-loop control logic, such as a conventional PID (Proportional-Integral-Derivative) control algorithm. Specifically, the actual displacement of the piezoelectric ceramic component is acquired in real time (which can be obtained through a high-precision displacement sensor), and the difference between this displacement and the theoretically predicted target displacement is calculated to obtain the deviation. Subsequently, the controller calculates the control voltage for correction based on this deviation in real time and applies this control voltage to both ends of the piezoelectric ceramic component. The piezoelectric ceramic component generates precise reverse mechanical vibration under high-frequency voltage drive.

[0179] For example, the formula for determining the control voltage is as follows:

[0180] ;

[0181] In the formula, express Control voltage at any given time; This represents the proportionality coefficient, which determines the strength and speed of the response; Indicates the calculation deviation. , express The actual displacement at any given moment; The coefficient of the integral term determines the degree to which residual errors are eliminated. Indicates to Integral; Represents the coefficients of the differential term;

[0182] Furthermore, by using reverse mechanical vibration, the vibration acceleration at the audio acquisition component (such as a microphone) can be reduced from 10 m / s². 2 Reduced to 2m / s 2 The following results in a reduction of over 80% in the transmission of structural vibrations.

[0183] It should be understood that this achieves high-precision active isolation of solid-borne sound (structural noise). Because structural vibrations propagate rapidly and attenuate slowly, conventional acoustic filtering is ineffective. This step introduces closed-loop displacement control to ensure precise anti-superposition of the reverse vibration generated by the piezoelectric ceramic with the body vibration, thus rigidly suppressing the physical vibration acceleration at the microphone (e.g., from 10 m / s²). 2 Reduced to 2m / s 2 The following measures reduce structural vibration transmission by more than 80%, fundamentally cutting off the intrusion path of structural noise.

[0184] For example, such as Figure 3 As shown, the mobile device (robot dog) includes a cancellation coil, a piezoelectric ceramic actuator, and a speaker. The cancellation coil generates a reverse electromagnetic signal, using the principle of electromagnetic field superposition to reduce noise; the piezoelectric ceramic actuator outputs reverse mechanical vibration to counteract the vibration transmitted from the robot body; and the speaker releases a reverse sound wave, which superimposes and cancels out the noise sound wave, reducing the acoustic noise amplitude at the microphone.

[0185] The self-noise cancellation method for mobile devices provided in this invention achieves active noise cancellation through the aforementioned approach. Compared to traditional noise reduction schemes that require waiting for heavily noisy, dirty data to fully enter the processing chip before algorithm calculations can be performed, resulting in huge computational overhead and physical limitations, this invention moves the noise cancellation defense line forward to the periphery of physical sensors (air medium, solid structure of the fuselage, electromagnetic radiation space). By using the parallel or collaborative operation of cancellation coils, audio output components, and piezoelectric ceramic components with closed-loop feedback, it effectively reduces high-frequency electromagnetic interference from radar, acoustic noise from joint operation, and solid-borne noise from the fuselage. This cross-physical-domain active cancellation mechanism not only significantly reduces the computational burden on subsequent digital signal processing chips but also cuts off self-noise at the data acquisition starting point, where the signal-to-noise ratio is most likely to deteriorate. This fundamentally ensures that the mobile device can still provide a high-purity basic audio signal under extreme conditions such as multi-component concurrency and high-intensity movement, thereby further improving the self-noise cancellation effect and ensuring the acquisition of pure voice signals.

[0186] Based on any of the above embodiments Figure 4 This is a second schematic flowchart of the self-noise cancellation method for mobile devices provided by the present invention, as shown below. Figure 4 As shown, the self-noise cancellation method of the mobile device includes steps 110, 120, 130, 140 and 150.

[0187] Step 140: Acquire the audio signal collected by the voice signal acquisition component of the mobile device.

[0188] Here, the voice signal acquisition component is typically a microphone array configured on a mobile device.

[0189] In one specific embodiment, to facilitate subsequent spatial filtering algorithms, the voice signal acquisition component can employ a distributed microphone array layout, comprising a main microphone array (such as a circular or linear four-microphone array, used to prioritize acquiring user voice signals) deployed at the head of the mobile device, and auxiliary microphone arrays deployed at self-noise sources such as the device's joints and radar casing (used to accurately acquire reference self-noise). For example, Figure 5 As shown, a dual-array structure of "head main array + fuselage auxiliary array" is adopted to realize the functional division of "voice priority acquisition + accurate self-noise reference". The fuselage auxiliary array can be deployed at self-noise sources such as hip joint, elbow joint, and radar shell. The head main array acquires interactive voice signals, and the fuselage auxiliary array acquires self-noise reference signals.

[0190] Here, the audio signal refers to the digital signal generated by the voice signal acquisition component after receiving sound wave vibrations in the air and performing analog-to-digital conversion (A / D conversion). It should be noted that, since hardware-level reverse physical / electromagnetic / acoustic cancellation has been performed, the audio signal acquired by the current microphone array already has a high initial signal-to-noise ratio at the physical source. This audio signal is actually a mixed signal containing the target user's voice and residual self-noise.

[0191] It should be understood that the voice signal acquisition component converts the pre-purified ambient sound field into a digital basic data stream that can be used for subsequent digital signal processors and artificial intelligence chips, providing a high-quality input source with a good signal-to-noise ratio baseline for subsequent algorithm-level noise reduction.

[0192] Step 150: Perform spatial filtering and / or speech separation processing on the audio signal to eliminate self-noise signals in the audio signal and obtain the target speech signal.

[0193] Here, the core principle of spatial filtering is to utilize the inherent geometrical directional differences between the target speech signal and the self-noise signal along their spatial propagation paths. The target speech signal typically originates from a specific angular region in front of or around the mobile device, while the self-noise signal is fixed to physical components such as body joints, motors, or radar. Spatial filtering techniques (such as beamforming algorithms) perform specific weighted summations on audio signals collected from multiple microphones, creating high gain in the direction of arrival of the target speech while simultaneously creating deep attenuation (a dip) in the direction of the self-noise source, thereby stripping away the self-noise in the physical spatial dimension.

[0194] Here, self-noise refers to residual mechanical vibration noise, electromagnetic noise, and wind noise mixed in the audio signal; the target speech signal refers to the pure user speech command signal extracted after spatial and frequency domain purification.

[0195] Furthermore, spatial filtering is applied to the audio signal, followed by speech separation processing. Based on this, when the direction of some residual noise coincides with the direction of speech, or when the frequency distribution of the noise severely overlaps with the frequency of the speech signal (such as high-frequency consonants), spatial filtering cannot remove it. In this case, speech separation processing is employed. A deep learning network model is used to perform high-dimensional feature extraction and mask prediction on the spectral characteristics of the audio signal (such as amplitude spectrum, Mel-frequency cepstral coefficients, and temporal dynamic features). Utilizing the spectral differences between speech, which has a distinct harmonic structure, and the noise, which tends towards random or fixed mechanical frequencies, the mixed signal is forcibly separated at the frequency domain level.

[0196] Furthermore, the two processing methods mentioned above can be dynamically scheduled based on the current motion status data of the mobile device. For example, in low-speed motion or stationary states, only lightweight spatial filtering is triggered; in extreme conditions such as high-speed motion or high-power radar scanning, a full-process cascaded scheme of spatial filtering + speech separation processing is triggered concurrently.

[0197] It should be understood that spatial filtering can resolve interference from directional noise, while speech separation processing can resolve complex nonlinear noise interference caused by spectral overlap. The combination of these two methods, or their independent application, can achieve millisecond-level precise separation of residual mixed signals, eliminating residual self-noise that slips through the hardware cancellation stage.

[0198] The self-noise cancellation method for mobile devices provided in this invention uses a complete three-dimensional noise reduction system consisting of active cancellation, spatial filtering, and speech separation. Compared to the single-dimensional noise reduction design (single beamforming or single noise reduction model) in the prior art, the multi-cascade processing mechanism of this invention can achieve complementary advantages of each dimension. Front-end physical cancellation ensures that no signal overload occurs, spatial filtering effectively locks the sound location of the target user, and back-end intelligent separation solves the problem of eliminating high-frequency complex coupling noise. Ultimately, it can accurately and losslessly restore and extract the target speech signal with a high signal-to-noise ratio from a highly dynamic and fluctuating background with strong self-noise, greatly improving the recognition rate and human-computer interaction experience of the mobile device's speech recognition system, that is, further improving the self-noise cancellation effect and ensuring the acquisition of pure speech signals.

[0199] Based on any of the above embodiments, the speech separation processing in this method is as follows:

[0200] Audio features are extracted from the frequency domain signal of the audio signal to obtain an audio feature vector;

[0201] The audio feature vector is input into the speech separation model to obtain a new amplitude spectrum output by the speech separation model;

[0202] Based on the phase spectrum of the frequency domain signal and the new amplitude spectrum, the audio signal after speech separation processing is determined.

[0203] Here, the frequency domain signal of an audio signal refers to the frequency domain representation obtained after a series of preprocessing and mathematical transformations of the time-domain audio waveform.

[0204] In one specific embodiment, the acquired audio signal is pre-emphasized to enhance high-frequency speech components and compensate for microphone attenuation, or framed and windowed, such as by overlaying a Hanning window to prevent spectral leakage; then a Fast Fourier Transform (FFT) is performed to convert it into a complex frequency domain signal containing amplitude and phase spectra.

[0205] For example, the pre-emphasis processing is as follows:

[0206] ;

[0207] In the formula, This represents the pre-emphasized audio signal at time t; Represents the audio signal at time t; Indicates the pre-emphasis coefficient; This represents the audio signal at time t-1.

[0208] For example, the framing and windowing process is as follows:

[0209] ;

[0210] In the formula, This represents the window function weight for the nth sampling point; This indicates the frame length, which is the number of sampling points included in a frame.

[0211] For example, the frequency domain transformation method is as follows:

[0212] ;

[0213] In the formula, express The frequency domain signal at a given time; This represents the pre-emphasis processed audio signal at the nth sampling point; This represents the window function weight for the nth sampling point; express The amplitude spectrum at any given time; express The phase spectrum at time (this phase spectrum is directly reused in the output stage).

[0214] Here, audio feature extraction refers to extracting the core physical parameters from the frequency domain signal that can effectively distinguish between human voices and machine noise.

[0215] In one specific embodiment, the extracted audio feature vector can be composed of two heterogeneous feature fusions. The first part is perceptual features, such as 39-dimensional Mel-Frequency Cepstral Coefficients (MFCCs) and their first and second-order differences, mainly used to characterize the acoustic perceptual properties of human speech; the second part is temporal dynamic physical features (such as 6-dimensional features: spectral centroid, spectral bandwidth, spectral flux, spectral entropy, peak factor, and zero-crossing rate), mainly used to capture the mechanical and physical fingerprint of mobile devices. For example, the spectral centroid can be used to sensitively locate the high-frequency howling of gears, and the spectral bandwidth can be used to effectively isolate broadband wind noise interference.

[0216] For example, regarding MFCC feature extraction, its Mel filter bank includes 24 triangular filters, with the center frequency f distributed according to the Mel scale:

[0217] ;

[0218] Accordingly, the logarithmic energy of each filter output is:

[0219] ;

[0220] In the formula, Indicates the first Logarithmic energy of the filter output; , They represent different frequencies; express Sound energy (power) at a specific frequency; Indicates the first The frequency response of a Mel filter;

[0221] Furthermore, regarding logarithmic energy Perform a 13-point DCT (Discrete Cosine Transform) to obtain 13-dimensional MFCC coefficients C(n) (n=1-13); subsequently, calculate the 13-dimensional first-order difference ΔC(n) and the 13-dimensional second-order difference ΔC(n). 2 C(n).

[0222] For example, regarding spectral feature extraction, the 6-dimensional time-domain dynamic features include the spectral centroid, spectral bandwidth, spectral flux, spectral entropy, peak factor, and zero-crossing rate. The spectral centroid is determined as follows:

[0223] ;

[0224] In the formula, Indicates the centroid of the spectrum; This indicates the frequency index number (from 0 to 256, corresponding to the 257-dimensional amplitude spectrum output earlier). express Sound energy (power) at a specific frequency.

[0225] The method for determining the spectrum bandwidth is as follows:

[0226] ;

[0227] In the formula, Indicates the spectrum bandwidth; This indicates the frequency index number (from 0 to 256, corresponding to the 257-dimensional amplitude spectrum output earlier). Indicates the centroid of the spectrum; express Sound energy (power) at a specific frequency.

[0228] Furthermore, the audio feature vectors are standardized and then input into the speech separation model.

[0229] It should be understood that a heterogeneous feature fusion mechanism combining biomimetic perception and physical statistics has been constructed. Traditional single MFCC features have low discriminative power when facing the pure mechanical self-noise of mobile robots. The feature vectors extracted in this embodiment provide multi-dimensional and strong discriminative information for subsequent deep learning models, greatly improving the model's accuracy in defining the boundary between human voices and machine mechanical sounds.

[0230] The speech separation model is used to eliminate self-noise signals in the audio signal.

[0231] In one embodiment, the speech separation model is a deep learning neural network that has been pre-trained and converged on a large dataset of noisy speech.

[0232] In one embodiment, the speech separation model is a Transformer architecture model comprising an encoder, a multi-head attention mechanism, and a decoder. The extracted audio feature vector (further, the amplitude spectrum of the frequency domain signal) is input into the speech separation model. The speech separation model predicts the energy distribution ratio of speech and noise in the current frame signal through internal nonlinear network layers (such as the multi-head attention mechanism focusing on the harmonic structure of the speech spectrum and weakening the random characteristics of noise). This energy distribution ratio is then used to perform a dot product filtering on the input original amplitude spectrum, thereby stripping away the amplitude energy belonging to the "self-noise signal," ultimately outputting a clean new amplitude spectrum containing only the target speech energy (i.e., the estimated speech amplitude spectrum).

[0233] Furthermore, the encoder includes four CNN layers, each with a "CNN-BN-GELU" structure, and performs positional encoding on the feature maps to add temporal information. Furthermore, the decoder includes two deconvolutional layers.

[0234] It should be understood that the speech separation model can capture the harmonic continuity of the speech signal over a long time series. Even if the frequencies of some residual self-noise seriously overlap with the frequencies of the speech signal, the speech separation model can accurately remove them through contextual features, thus solving the problem that traditional filters are helpless in the face of non-stationary high-frequency noise.

[0235] Here, the phase spectrum of the frequency domain signal refers to the original noisy phase information that was directly extracted and bypassed during the previous FFT transformation.

[0236] Since the phase of sound mainly contains the temporal alignment information of the signal, the specific process of determining the audio signal after speech separation is as follows: the pure new amplitude spectrum output by the speech separation model is combined with the original phase spectrum retained by the bypass in polar coordinates, and then an inverse fast Fourier transform is performed to restore it to a waveform signal in the time domain, thus obtaining the final pure audio signal.

[0237] It should be understood that this invention proposes an extremely computationally efficient amplitude-phase decoupling and reconstruction scheme. In digital audio processing, the distribution of phase information is extremely random. Forcibly predicting or separating the phase using a speech separation model not only consumes enormous computing power but also results in the output sound having a strong machine-synthesized electronic sound or artifacts. This invention only allows the amplitude spectrum to be purified by the speech separation model, while directly reusing the original phase spectrum to the output stage. This ensures high purity of the output speech while preserving the naturalness of the original sound timbre to the greatest extent, and significantly reduces the computing power overhead of the edge computing main control chip in mobile devices, perfectly meeting the dual stringent requirements of low power consumption and high sound quality in mobile devices.

[0238] The self-noise cancellation method for mobile devices provided in this invention fully utilizes the powerful high-dimensional feature nonlinear mapping capability of deep neural networks to eliminate complex mechanical and electromagnetic residual noise through the above-mentioned method. Furthermore, by preserving the original phase spectrum through a bypass design, it avoids the computational bottleneck and distortion risk of the speech separation model in phase prediction, thereby achieving an optimal balance between noise reduction depth and edge computing power consumption. This ensures the real-time performance (such as millisecond-level low latency) and high fidelity of the output results of speech separation processing on mobile devices.

[0239] Based on any of the above embodiments, before inputting the audio feature vector into the speech separation model to obtain the new amplitude spectrum output by the speech separation model, the method further includes:

[0240] Based on the motion state data, the motion intensity of the mobile device is determined;

[0241] Based on the motion intensity, the noise suppression threshold of the speech separation model is adaptively adjusted; the noise suppression threshold is used to determine the amount of noise suppression and the amount of speech preservation.

[0242] Here, motion intensity is a comprehensive indicator used to quantitatively assess the overall motion intensity and self-noise generation potential of a mobile device in real time. The motion intensity is calculated based on the physical quantity in the motion state data that has the strongest correlation with self-noise intensity.

[0243] It should be understood that the embodiments of the present invention construct a real-time physical quantification index that can accurately reflect the potential intensity of self-noise in a mobile device. By converting the physical data (such as rotational speed and power) of the underlying mechanical and electromagnetic domains into dimensionless motion intensity values ​​that can be recognized by the upper-layer applications, a stable and reliable judgment basis is provided for the adaptive adjustment of parameters of the subsequent speech separation model.

[0244] Here, the noise suppression threshold is a key hyperparameter within the speech separation model. This threshold directly affects the decision-making process of the speech separation model in masking the amplitude spectrum, determining the degree of energy suppression of suspected noise during separation. In short, the noise suppression threshold is used to balance the amount of noise suppression (how much noise is eliminated) with the amount of speech preservation (how much speech detail is retained). A higher noise suppression threshold results in more thorough noise suppression, but also a higher risk of losing speech details; a lower threshold results in more complete preservation of speech details, but also a higher risk of residual noise.

[0245] Here, adaptive adjustment means no longer using a fixed, unchanging noise suppression threshold, but dynamically and stepwise adjusting the noise suppression threshold based on the real-time calculated motion intensity, through instructions issued by the main control module.

[0246] In one embodiment, three threshold levels—high, medium, and low—can be set for threshold switching. Specifically, when the motion intensity is less than or equal to 0.3 (e.g., joint rotation speed ≤ 3.6 rad / s, radar power ≤ 15 W), the mobile device is determined to be in a low-noise state, and the noise suppression threshold is adaptively lowered to a lower value (e.g., 0.1, based on the spectral amplitude) to maximize the preservation of speech details such as weak consonants. When the motion intensity is between 0.3 and 0.7 (e.g., joint rotation speed 3.6-8.4 rad / s, radar power 15-35 W), the mobile device is determined to be in a medium-noise state, and the noise suppression threshold is set to a balanced value (e.g., 0.3) to achieve a balance between noise suppression and speech preservation. When the motion intensity is greater than or equal to 0.7 (e.g., joint rotation speed ≥ 8.4 rad / s, radar power ≥ 35 W), the mobile device is determined to be in a high-noise state, and the noise suppression threshold is adaptively increased to a higher value (e.g., 0.5) to prioritize the suppression of extremely high-intensity self-noise and prevent strong noise from completely masking the target speech signal.

[0247] It should be understood that a gradient-based noise reduction mechanism was constructed to enhance the speech separation model. By dynamically hard-correlating the noise suppression threshold of the speech separation model with the physical motion intensity of the mobile device, the speech separation model can intelligently switch its focus under different operating conditions. In low-noise scenarios, it prioritizes fidelity; in high-noise scenarios, it prioritizes clarity. This adaptive adjustment mechanism solves the technical deficiency of traditional fixed-threshold models in dealing with wide dynamic range self-noise, where they "lose some aspects while focusing on others."

[0248] The self-noise cancellation method for mobile devices provided in this invention introduces a dynamic threshold optimization module that is dynamically adjusted by the intensity of physical motion in real time, based on the traditional speech separation model. Compared with the black-box processing in the prior art where parameters are fixed and there is no awareness of changes in operating conditions, this invention exhibits extremely high scene-adaptive robustness. It ensures that the speech separation model of the mobile device can automatically switch to the optimal operating point, whether in quiet standby or in extreme scenarios of high-speed running. This achieves a perfect dynamic balance between high noise suppression ratio and high speech fidelity across the entire operating range, greatly improving the availability and reliability of the final output target speech signal.

[0249] Based on any of the above embodiments, in this method, the motion state data includes the joint rotation speed of the mobile device and the radar power of the mobile device. Accordingly, determining the motion intensity of the mobile device based on the motion state data includes:

[0250] A third coefficient is determined based on the ratio of the joint rotation speed to the maximum joint rotation speed of the moving device;

[0251] A fourth coefficient is determined based on the ratio of the radar power to the maximum radar power of the mobile device;

[0252] The motion intensity of the mobile device is obtained by weighted summation of the third and fourth coefficients.

[0253] Here, joint rotational speed refers to the angular velocity of each moving joint of the mobile device at the current moment, which has a strong positive correlation with the amplitude and dominant frequency of mechanical vibration noise. Maximum joint rotational speed refers to the limit angular velocity physical quantity that the joint motor can achieve, set in the factory hardware parameters of the mobile device; that is, the maximum joint rotational speed is the speed preset according to the joint parameters of the mobile device.

[0254] Here, radar power refers to the transmission power of the sensing radar (such as lidar or millimeter-wave radar) on the mobile device at the current moment. The higher the radar power, the stronger the electromagnetic wave energy radiated into the surrounding space, thus inducing stronger electromagnetic interference noise in the microphone circuit. The maximum radar power is the limit operating power set in the manufacturer's specifications for the radar component; that is, the maximum radar power is the power preset according to the radar parameters of the mobile device.

[0255] Here, the third coefficient is normalized (within the range of [0, 1]) to characterize the intensity of the current mechanical motion of the mobile device. Based on this, the potential intensity of the mechanical noise of the mobile device can be accurately and in real time quantified. As the most direct physical quantity that causes mechanical vibration, the introduction of the third coefficient allows for a first-level assessment of the self-noise level from the root of mechanical kinematics.

[0256] Here, the fourth coefficient is normalized to characterize the current electromagnetic radiation intensity of the mobile device. Based on this, the intensity of electromagnetic noise interference in the non-acoustic domain is quantified. Electromagnetic noise cannot be directly perceived by conventional acoustic sensors, but it has a significant impact on the quality of voice signal acquisition. The introduction of the fourth coefficient enables a second-level assessment of the self-noise level from its electromagnetic origin.

[0257] In one embodiment, the mechanical noise generated by high-speed joint movement has a more significant impact on the final speech signal-to-noise ratio than radar electromagnetic noise. Therefore, the weight of the third coefficient is greater than that of the fourth coefficient.

[0258] For example, the formula for calculating exercise intensity is as follows:

[0259] ;

[0260] In the formula, Indicates exercise intensity; Indicates joint rotation speed; Indicates the maximum rotational speed of the joint; This indicates the weight of the third coefficient. It is usually greater than 0.5, for example, 0.6. This indicates the weight of the fourth coefficient; Indicates radar power; This indicates the radar's maximum power.

[0261] It should be understood that by using weighted summation calculation, the two core physical sources of self-noise (mechanical motion and electromagnetic radiation) are scientifically integrated to output a quantitative index that can accurately and comprehensively reflect the current overall intensity of self-noise of the device, thus providing an extremely reliable decision-making basis for the dynamic threshold adaptive adjustment of the subsequent speech separation model.

[0262] The self-noise cancellation method for mobile devices provided in this invention provides a specific way to determine motion intensity through the above-described method. This method overcomes the technical defects of existing technologies that rely solely on a single physical quantity (such as speed or power) to assess device status, leading to biased and inaccurate evaluations. By weightedly fusing joint rotation speed (best representative of mechanical noise intensity) and radar power (best representative of electromagnetic noise intensity), this invention constructs a multimodal physical quantification evaluation model that closely matches the self-noise generation mechanism of mobile devices. This ensures that the final calculated motion intensity most realistically and comprehensively reflects the instantaneous comprehensive deterioration level of device self-noise. This guarantees that the top-level speech separation model can make the optimal noise suppression threshold decision based on the most accurate feedback from the underlying physical state, further improving the self-noise cancellation effect and ensuring the acquisition of clean speech signals.

[0263] Based on any of the above embodiments, the spatial filtering method is as follows:

[0264] The direction of arrival (DOA) of the audio signal is estimated to obtain the current speech direction angle.

[0265] If the difference between the current speech direction angle and the previous speech direction angle is greater than or equal to a preset angle threshold, or if the magnitude of the residual of the beam weight is greater than or equal to a preset error threshold, the current beam weight is updated.

[0266] Based on the latest beam weights, the audio signal is weighted to obtain a spatially filtered audio signal.

[0267] Here, direction of arrival estimation refers to calculating the physical direction of arrival of the sound source signal in space using the collected audio signal and a specific signal processing algorithm.

[0268] In one specific embodiment, DOA estimation can employ the Multiple Signal Classification (MUSIC) algorithm. This MUSIC algorithm utilizes the orthogonality principle between the signal and noise subspaces, performing eigenvalue decomposition on the array covariance matrix and spatial spectrum search to locate the current speech direction angle with extremely high accuracy and resolution.

[0269] It should be understood that DOA estimation provides precise geometric pointing constraints for adaptive beamforming. By calculating the direction of arrival of the speech signal in real time, the target direction that should be protected can be dynamically determined, which is a prerequisite for enhancing the speech signal and suppressing self-noise sources.

[0270] Here, the speech direction angle at the previous time step is the speech direction angle estimated during the previous time step. The preset angle threshold is an angle set according to actual needs, for example, it is 5°. The preset error threshold is a threshold set according to actual needs, for example, it is 0.5.

[0271] Wherein, the initial value of the beam weight is the minimum variance distortionless response (MVDR) beam weight determined based on the initial speech direction angle and the initial noise reference signal; the residual of the beam weight is determined based on the difference between the desired signal and the actual signal; the desired signal is determined based on the current speech direction angle and the audio signal; and the actual signal is determined based on the current beam weight and the audio signal.

[0272] Here, the initial speech direction angle is obtained from the first DOA estimation when the mobile device is started or stationary. The initial noise reference signal can be acquired by an auxiliary microphone array deployed near the self-noise source to construct a high-precision noise covariance matrix. Based on this, the initial values ​​of the beam weights are calculated using the Minimum Variance Distortionless Response (MVDR) algorithm. The goal of the MVDR algorithm is to minimize the total output power of noise in all other directions while ensuring that the speech direction gain is 1 (distortion-free). Based on this, an optimal initial beamform is established through a single high-precision MVDR calculation, which forms a main lobe pointing towards the speech direction in space and a very deep concave shape in the known self-noise direction.

[0273] The calculation method of MVDR beam weights is illustrated by example. The goal of the MVDR algorithm is to calculate the beam weights at the initial speech direction angle. The algorithm aims to maintain a distortion-free signal while minimizing noise power in other directions. The input to the algorithm is the initial speech direction angle. Given the initial noise reference signal, the output is a 4×1 beam weight W=[w1, w2, w3, w4], which is a vector containing 4 complex numbers, each of which corresponds to the gain (determining the volume) and phase (determining the time delay) of the i-th microphone.

[0274] First, establish the optimization objective and constraints. The optimization objective (minimize output noise power) is as follows:

[0275] ;

[0276] In the formula, This indicates finding the minimum value; express The audio signal at that moment, E[] represents the original mixed signal vector (containing speech and noise) collected by four microphones at time t; E[] represents the average power of the signal (first take the square of the modulus to get the instantaneous power, and then calculate the mathematical expectation); This represents the noise covariance matrix constructed based on the initial noise reference signal; It is the conjugate transpose of W.

[0277] The constraints (no distortion in the speech direction) are as follows:

[0278] ;

[0279] In the formula, It is the array manifold vector corresponding to the initial speech direction angle.

[0280] Secondly, the weights are solved using the Lagrange multiplier method, and the Lagrange function is constructed (incorporating the constraints into the objective function):

[0281] ;

[0282] In the formula, It is a Lagrange multiplier (complex number).

[0283] Then, the beam weight vector Taking the partial derivative and setting it to 0, we get:

[0284] Ultimately, this simplifies to the final formula for calculating the MVDR weights:

[0285] .

[0286] Specifically, instead of recalculating complex MVDR weights at every sampling point, the system monitors two triggering conditions in real time. First, it continuously calculates the difference between the current speech direction angle and the previous speech direction angle. When this difference is greater than or equal to a preset angle threshold, it determines that the mobile device has undergone a significant attitude change and the current beam weights need to be updated. Second, it calculates the residual of the beam weights, which is the difference between the desired signal (the ideal output generated based on the current speech direction angle) and the actual signal (the actual output filtered based on the current beam weights). When the magnitude of this residual is greater than or equal to a preset error threshold, it determines that an unknown strong interference source has appeared, and the beam also needs to be updated. Based on this, a low-power intelligent sleep and wake-up mechanism is constructed. When the mobile device is moving smoothly, no weight updates are required, greatly saving computational resources. Only when the mobile device undergoes a clear attitude change or encounters sudden strong interference is the update process initiated as needed, achieving an optimal balance between computational efficiency and real-time tracking performance.

[0287] Specifically, once the triggering condition is met, the current beam weights are updated.

[0288] In one specific embodiment, the update process employs the Recursive Least Squares (RLS) algorithm. RLS is a fast-converging adaptive filtering algorithm that introduces a forgetting factor to assign higher weights to new data and avoids complex inversion operations by utilizing matrix inversion, thus achieving millisecond-level iterative convergence of beam weights with extremely low computational cost. Based on this, extremely rapid dynamic tracking of weights is achieved. After the mobile device changes direction, there is no need to recalculate the highly complex MVDR weights; instead, rapid iterative approximation is performed directly based on the existing weights, ensuring that the beam pointing closely follows the dynamic changes in the speech direction. This solves the technical problems of slow convergence speed and inability to handle highly dynamic scenarios in traditional adaptive algorithms.

[0289] For example, the desired signal is determined based on the following formula:

[0290] ;

[0291] In the formula, express Expected signal at time, Indicates the current speech direction angle ( The array manifold vector corresponding to (time point); express The audio signal at a given moment.

[0292] For example, the actual signal is determined based on the following formula:

[0293] ;

[0294] In the formula, express The actual signal at that moment, express The audio signal at that moment, This indicates the current beam weights.

[0295] For example, the residuals of the beam weights are determined based on the following formula:

[0296] ;

[0297] In the formula, Represents the residual of the beam weights. express Expected signal at time, express The actual signal at that moment.

[0298] Here, weighted processing refers to performing complex multiplication operations on the audio signal with the corresponding weights in the latest beam weight vector (initial MVDR weights or weights updated by RLS), and then summing the results from multiple paths to obtain the final single-path output signal, i.e., the spatially filtered audio signal. Based on this, the separation of target and interference signals is achieved in the physical space dimension. Through weighted processing, signal components from the target speech direction are enhanced or preserved because they are located in the main lobe of the beam, while signal components from the self-noise direction of the fuselage are significantly suppressed because they are located in the beam concave area, thereby significantly improving the signal-to-noise ratio of the output audio signal.

[0299] For example, the weighted processing method is as follows:

[0300] ;

[0301] In the formula, This represents the spatially filtered audio signal at time t; This indicates the latest beam weights. express The audio signal at a given moment.

[0302] The self-noise cancellation method for mobile devices provided in this invention constructs an event-driven adaptive spatial beam tracking method specifically designed for highly dynamic mobile devices. It employs a unique dual-trigger mechanism combining angle drift and residual amplitude. Compared to existing technologies with fixed beamforming (unable to adapt to motion) or conventional adaptive filtering (slow convergence and high computational cost), this invention achieves an intelligent operating mode of "optimal when stationary (MVDR), extremely fast when moving, and no unnecessary calculations (event triggering)." This ensures that the mobile device can accurately lock onto the user's voice direction regardless of whether it is stationary, moving slowly, making sharp turns, or climbing. Simultaneously, it continuously suppresses dynamically changing self-noise sources within the auditory blind zone of spatial filtering, providing an extremely clean signal input for subsequent speech recognition from a spatial perspective. This further improves the self-noise cancellation effect and ensures the acquisition of clean speech signals.

[0303] Based on any of the above embodiments, in this method, the motion state data includes at least one of the joint motion state data of the mobile device, the posture state data of the mobile device, and the radar operating state data of the mobile device.

[0304] Here, joint motion state data is a collection of data characterizing the instantaneous kinematic state of each movable joint of the mobile device (such as the hip and knee joints of a robot dog). In one embodiment, this data can be collected by sensors such as encoders deployed at each joint. Based on this, the joint motion state data has a strong direct physical correlation with the mechanical self-noise of the mobile device. High-speed rotation and abrupt acceleration / deceleration of the joints directly cause high-frequency whistling and friction noise within the motor and the gear set of the reducer. Therefore, by directly introducing joint kinematic parameters, the noise characteristic prediction model can model and predict the generation of mechanical self-noise from the root of mechanical kinematics, thereby accurately predicting the amplitude, frequency, and phase of the acoustic noise directly caused by joint motion.

[0305] Here, attitude state data is a set of data characterizing the attitude and displacement changes of the mobile device's overall body in space. In one embodiment, it can be acquired by an inertial measurement unit (IMU) deployed on the mobile device's main control board. This attitude state data is strongly correlated with the mobile device's structural vibration noise and wind noise. Severe shaking, running, or jumping of the body generates low-frequency structural vibrations, which are transmitted to the microphone through the solid structure; simultaneously, rapid movement of the body causes intense friction with the air, generating broadband wind noise. Based on this, by introducing body attitude and acceleration data, the noise characteristic prediction model can effectively predict low-frequency structural vibrations and broadband wind noise from a macroscopic kinematic perspective, compensating for the inadequacy of relying solely on joint data to comprehensively characterize the overall motion state.

[0306] Here, radar operating status data is a set of data characterizing the current operating mode and power level of the sensing radar (such as lidar or millimeter-wave radar) mounted on the mobile device. In one embodiment, this data can be directly obtained from the radar's control unit. This radar operating status data has an absolute causal relationship with the electromagnetic interference self-noise of the mobile device. When the radar operates at high power, its transmitting unit radiates strong electromagnetic waves into the surrounding space. These electromagnetic waves are sensed by the microphone's signal acquisition circuit, forming non-acoustic electromagnetic noise that severely pollutes the audio signal. Based on this, by introducing radar operating status data, the noise feature prediction model can accurately predict non-acoustic electromagnetic noise from its electromagnetic origin, solving the technical problem that traditional acoustic noise reduction algorithms cannot perceive or process electromagnetic interference.

[0307] The self-noise cancellation method for mobile devices provided in this invention constructs a multimodal, cross-physical domain self-noise source representation model through the aforementioned approach, overcoming the one-sidedness of existing technologies that rely solely on single-dimensional information (such as sound or speed alone) for noise estimation. By fusing joint motion data representing mechanical noise, attitude state data representing structural and wind noise, and radar operating state data representing electromagnetic noise, this invention enables the noise feature prediction model to perform comprehensive and global reasoning based on all physical sources of self-noise generated by the mobile device, thereby further improving the self-noise cancellation effect and ensuring the acquisition of clean speech signals.

[0308] Based on any of the above embodiments, a combined solution of "structural isolation + acoustic filtering" is adopted to address wind noise and body vibration generated by the movement of the mobile device. Specifically, for microphone physical isolation: the microphone is encapsulated in a silicone shock-absorbing sleeve, which is connected to the body through three elastic support points, forming a two-stage shock-absorbing structure. This significantly reduces the vibration acceleration transmitted from the body to the microphone, cutting off the transmission path of structural vibration. For wind noise acoustic filtering: the microphone front end is equipped with an acoustic windproof cover, employing a double-layer honeycomb porous structure. The outer layer is a stainless steel mesh with a 0.3mm pore size (blocking large air particles), and the inner layer is a polytetrafluoroethylene membrane with a 0.1mm pore size (filtering fine air particles). The two layers are spaced 1mm apart, forming an air damping cavity. This reduces wind noise intensity by more than 30dB without affecting the transmission of voice signals above 200Hz. Furthermore, the microphone signal leads use shielded twisted-pair cables to reduce the impact of electromagnetic interference on signal transmission and ensure the purity of the acquired signal.

[0309] The noise cancellation device for a mobile device provided by the present invention is described below. The noise cancellation device for a mobile device described below can be referred to in correspondence with the noise cancellation method for a mobile device described above.

[0310] Figure 6This is a schematic diagram of the self-noise cancellation device for the mobile device provided by the present invention, as shown below. Figure 6 As shown, the self-noise cancellation device of the mobile device includes: a data acquisition module 610, a feature prediction module 620, and a signal generation module 630.

[0311] The data acquisition module 610 is used to acquire motion state data of the mobile device in real time; the motion state data is correlated with the self-noise of the mobile device.

[0312] The feature prediction module 620 is used to input the motion state data into the noise feature prediction model to obtain the noise feature parameters output by the noise feature prediction model; the noise feature parameters include the target amplitude, target frequency and target phase of the self-noise generated by the mobile device; the noise feature prediction model is trained based on the sample motion state data.

[0313] The signal generation module 630 is used to generate a reverse cancellation signal based on the noise characteristic parameters, and to control the reverse cancellation component of the mobile device to output the reverse cancellation signal.

[0314] Figure 7 This is the fourth structural schematic diagram of the mobile device provided by the present invention, as shown below. Figure 7 As shown, the mobile device may include a processor 710, a communications interface 720, a memory 730, and a communication bus 740, wherein the processor 710, communications interface 720, and memory 730 communicate with each other via the communication bus 740. The processor 710 can call logical instructions in the memory 730 to execute a self-noise cancellation method for the mobile device. This method includes: real-time acquisition of motion state data of the mobile device; the motion state data being correlated with the self-noise of the mobile device; inputting the motion state data into a noise feature prediction model to obtain noise feature parameters output by the noise feature prediction model; the noise feature parameters including the target amplitude, target frequency, and target phase of the self-noise generated by the mobile device; the noise feature prediction model being trained based on sample motion state data; and generating a reverse cancellation signal based on the noise feature parameters, and controlling the reverse cancellation component of the mobile device to output the reverse cancellation signal.

[0315] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0316] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the self-noise cancellation method for a mobile device provided by the above methods. The method includes: real-time acquisition of motion state data of the mobile device; the motion state data being correlated with the self-noise of the mobile device; inputting the motion state data into a noise feature prediction model to obtain noise feature parameters output by the noise feature prediction model; the noise feature parameters including the target amplitude, target frequency, and target phase of the self-noise generated by the mobile device; the noise feature prediction model being trained based on sample motion state data; generating a reverse cancellation signal based on the noise feature parameters, and controlling the reverse cancellation component of the mobile device to output the reverse cancellation signal.

[0317] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a self-noise cancellation method for a mobile device provided by the methods described above. The method includes: real-time acquisition of motion state data of the mobile device; the motion state data being correlated with the self-noise of the mobile device; inputting the motion state data into a noise feature prediction model to obtain noise feature parameters output by the noise feature prediction model; the noise feature parameters including the target amplitude, target frequency, and target phase of the self-noise generated by the mobile device; the noise feature prediction model being trained based on sample motion state data; and generating a reverse cancellation signal based on the noise feature parameters, and controlling the reverse cancellation component of the mobile device to output the reverse cancellation signal.

[0318] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0319] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0320] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for self-noise cancellation in a mobile device, characterized in that, include: The motion state data of the mobile device is collected in real time; the motion state data is correlated with the self-noise of the mobile device. The motion state data is input into the noise feature prediction model to obtain the noise feature parameters output by the noise feature prediction model; The noise characteristic parameters include the target amplitude, target frequency, and target phase of the self-noise generated by the mobile device; the noise characteristic prediction model is trained based on sample motion state data. Based on the noise characteristic parameters, a reverse cancellation signal is generated, and the reverse cancellation component of the mobile device is controlled to output the reverse cancellation signal; The step of inputting the motion state data into the noise feature prediction model to obtain the noise feature parameters output by the noise feature prediction model includes: The motion state data is denoised using a Kalman filter algorithm to obtain denoised motion state data. The process noise covariance and / or observation noise covariance of the Kalman filter algorithm are dynamically adjusted based on the motion intensity coefficient of the mobile device, which is used to characterize the stability of the current motion state of the mobile device. The noise-reduced motion state data is input into the noise feature prediction model to obtain the noise feature parameters output by the noise feature prediction model. The motion state data includes the acceleration of each joint of the mobile device and the three-dimensional linear acceleration of the fuselage; The exercise intensity coefficient is determined based on the following method: The first coefficient is determined based on the ratio of the maximum joint acceleration among the aforementioned joint accelerations to the maximum rated joint acceleration of the moving device. A second coefficient is determined based on the ratio of the linear acceleration of the mobile device's fuselage to the maximum rated linear acceleration of the mobile device's fuselage; the linear acceleration of the fuselage is determined based on the three-dimensional linear acceleration of the fuselage. The exercise intensity coefficient is obtained by weighted summation of the first coefficient and the second coefficient.

2. The self-noise cancellation method for a mobile device according to claim 1, characterized in that, The reverse cancellation signal includes a reverse electromagnetic signal, and the reverse cancellation component includes a cancellation coil of the radar disposed on the mobile device. The reverse cancellation component controlling the moving device outputs the reverse cancellation signal, including: Controlling the cancelling coil to generate a reverse electromagnetic field to output the reverse electromagnetic signal; and / or, The reverse cancellation signal includes a reverse acoustic wave signal, and the reverse cancellation component includes an audio output component disposed on the mobile device; The reverse cancellation component controlling the moving device outputs the reverse cancellation signal, including: Control the audio output component to generate a reverse sound wave to output the reverse sound wave signal; and / or, After inputting the motion state data into the noise feature prediction model and obtaining the noise feature parameters output by the noise feature prediction model, the method further includes: Based on the noise characteristic parameters, the target displacement of the reverse mechanical vibration is determined; The calculation deviation is determined based on the target displacement and the actual displacement of the piezoelectric ceramic component located on the moving device. The control voltage is determined based on the calculated deviation, and the piezoelectric ceramic component is driven to generate reverse mechanical vibration based on the control voltage.

3. The self-noise cancellation method for a mobile device according to any one of claims 1 to 2, characterized in that, After the reverse cancellation component controlling the moving device outputs the reverse cancellation signal, it further includes: Acquire the audio signal collected by the voice signal acquisition component of the mobile device; The audio signal is subjected to spatial filtering and / or speech separation processing to eliminate self-noise signals in the audio signal and obtain the target speech signal.

4. The self-noise cancellation method for a mobile device according to claim 3, characterized in that, The speech separation processing method is as follows: Audio features are extracted from the frequency domain signal of the audio signal to obtain an audio feature vector; The audio feature vector is input into the speech separation model to obtain a new amplitude spectrum output by the speech separation model; the speech separation model is used to eliminate self-noise signals in the audio signal. Based on the phase spectrum of the frequency domain signal and the new amplitude spectrum, the audio signal after speech separation processing is determined.

5. The self-noise cancellation method for a mobile device according to claim 4, characterized in that, Before inputting the audio feature vector into the speech separation model to obtain the new amplitude spectrum output by the speech separation model, the method further includes: Based on the motion state data, the motion intensity of the mobile device is determined; Based on the motion intensity, the noise suppression threshold of the speech separation model is adaptively adjusted; the noise suppression threshold is used to determine the amount of noise suppression and the amount of speech preservation.

6. The self-noise cancellation method for a mobile device according to claim 5, characterized in that, The motion state data includes the joint rotation speed of the mobile device and the radar power of the mobile device; Determining the motion intensity of the mobile device based on the motion state data includes: A third coefficient is determined based on the ratio of the joint rotation speed to the maximum joint rotation speed of the moving device; A fourth coefficient is determined based on the ratio of the radar power to the maximum radar power of the mobile device; The motion intensity of the mobile device is obtained by weighted summation of the third coefficient and the fourth coefficient.

7. The self-noise cancellation method for a mobile device according to claim 3, characterized in that, The spatial filtering method is as follows: The direction of arrival (DOA) of the audio signal is estimated to obtain the current speech direction angle. If the difference between the current speech direction angle and the previous speech direction angle is greater than or equal to a preset angle threshold, or if the magnitude of the residual of the beam weight is greater than or equal to a preset error threshold, the current beam weight is updated. The initial value of the beam weight is the minimum variance distortionless response (MVDR) beam weight determined based on the initial speech direction angle and the initial noise reference signal. The residual of the beam weight is determined based on the difference between the desired signal and the actual signal. The desired signal is determined based on the current speech direction angle and the audio signal. The actual signal is determined based on the current beam weight and the audio signal. Based on the latest beam weights, the audio signal is weighted to obtain a spatially filtered audio signal.

8. The self-noise cancellation method for a mobile device according to any one of claims 1 to 2, characterized in that, The motion state data includes at least one of the joint motion state data of the mobile device, the posture state data of the mobile device, and the radar operating state data of the mobile device.

9. A self-noise cancellation device for a mobile device, characterized in that, include: The data acquisition module is used to collect motion state data of the mobile device in real time; the motion state data is correlated with the self-noise of the mobile device. The feature prediction module is used to input the motion state data into the noise feature prediction model to obtain the noise feature parameters output by the noise feature prediction model. The noise characteristic parameters include the target amplitude, target frequency, and target phase of the self-noise generated by the mobile device; the noise characteristic prediction model is trained based on sample motion state data. A signal generation module is used to generate an inverse cancellation signal based on the noise characteristic parameters, and to control the inverse cancellation component of the mobile device to output the inverse cancellation signal; The step of inputting the motion state data into the noise feature prediction model to obtain the noise feature parameters output by the noise feature prediction model includes: The motion state data is denoised using a Kalman filter algorithm to obtain denoised motion state data. The process noise covariance and / or observation noise covariance of the Kalman filter algorithm are dynamically adjusted based on the motion intensity coefficient of the mobile device, which is used to characterize the stability of the current motion state of the mobile device. The noise-reduced motion state data is input into the noise feature prediction model to obtain the noise feature parameters output by the noise feature prediction model. The motion state data includes the acceleration of each joint of the mobile device and the three-dimensional linear acceleration of the fuselage; The exercise intensity coefficient is determined based on the following method: The first coefficient is determined based on the ratio of the maximum joint acceleration among the aforementioned joint accelerations to the maximum rated joint acceleration of the moving device. A second coefficient is determined based on the ratio of the linear acceleration of the mobile device's fuselage to the maximum rated linear acceleration of the mobile device's fuselage; the linear acceleration of the fuselage is determined based on the three-dimensional linear acceleration of the fuselage. The exercise intensity coefficient is obtained by weighted summation of the first coefficient and the second coefficient.

10. A mobile device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the self-noise cancellation method of the mobile device as described in any one of claims 1 to 8.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the self-noise cancellation method of the mobile device as described in any one of claims 1 to 8.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the self-noise cancellation method of the mobile device as described in any one of claims 1 to 8.

Images