Interference mitigation using sensor fusion within the device

Abstract

Interference mitigation using sensor fusion within a device is described herein. In an embodiment, a method of performing interference mitigation in a device including a millimeter wave radar includes transmitting a radar signal with the millimeter wave radar, receiving a reflected radar signal with the millimeter wave radar, the reflected radar signal corresponding to the transmitted radar signal, generating a first spectrogram based on the reflected radar signal, generating a second spectrogram indicative of movement of a non-target object, generating a compensated radar spectrogram based on the first spectrogram and the second spectrogram to compensate for an effect of the movement of the non-target object in the first spectrogram, and detecting a target or a characteristic of the target based on the compensated radar spectrogram.

Description

Technical Field

[0001] The present disclosure relates generally to electronic systems and methods and, in particular embodiments, to interference mitigation using sensor fusion within a device. Background Art

[0002] Applications in the millimeter wave (mm-wave) frequency region have gained significant attention over the past few years due to the rapid development of low-cost semiconductor technologies such as silicon germanium (SiGe) and fine-geometry complementary metal oxide semiconductor (CMOS) processes. The availability of high-speed bipolar and metal oxide semiconductor (MOS) transistors has led to a growing demand for integrated circuits for mm-wave applications at 24 GHz, 60 GHz, 77 GHz, and 80 GHz, as well as beyond 100 GHz. Such applications include, for example, automotive radar systems and multi-gigabit communication systems.

[0003] In some radar systems, the distance between the radar and the target is determined by transmitting a frequency modulated signal, receiving a reflection of the frequency modulated signal (also called an echo), and determining the distance based on the time delay and / or frequency difference between the transmission and reception of the frequency modulated signal. Therefore, some radar systems include a transmit antenna that transmits a radio frequency (RF) signal and a receive antenna that receives the reflected RF signal, as well as associated RF circuits for generating the transmit signal and receiving the RF signal. In some cases, multiple antennas can be used to implement directional beams using phased array technology. Multiple-input multiple-output (MIMO) configurations with multiple chipsets can also be used to perform coherent and non-coherent signal processing. Summary of the Invention

[0004] According to an embodiment, a method for interference mitigation in a device including a millimeter-wave radar includes: transmitting a radar signal using the millimeter-wave radar; receiving a reflected radar signal using the millimeter-wave radar, the reflected radar signal corresponding to the transmitted radar signal; generating a first spectrogram based on the reflected radar signal; generating a second spectrogram indicating movement of a non-target object; generating a compensated radar spectrogram based on the first spectrogram and the second spectrogram to compensate for the effect of the movement of the non-target object in the first spectrogram; and detecting a target or a characteristic of the target based on the compensated radar spectrogram.

[0005] According to an embodiment, an apparatus includes: a millimeter-wave radar configured to transmit a radar signal and receive a reflected radar signal, wherein the reflected radar signal corresponds to the transmitted radar signal; and a controller configured to: generate a first spectrogram based on the reflected radar signal; generate a second spectrogram indicating the movement of a non-target object; generate a compensated radar spectrogram based on the first spectrogram and the second spectrogram to compensate for the effect of the movement of the non-target object in the first spectrogram; and detect a target or an attribute of the target based on the compensated radar spectrogram.

[0006] According to an embodiment, a mobile device includes: a millimeter-wave radar configured to transmit a radar signal and receive a reflected radar signal, wherein the reflected radar signal corresponds to the transmitted radar signal; a speaker; a microphone configured to measure the output of the speaker to generate a measurement output; an enclosure at least partially surrounding the millimeter-wave radar, the speaker, and the microphone; and a controller configured to: generate a first spectrum based on the reflected radar signal; generate a second spectrum based on the measurement output; generate a compensated radar spectrum based on the first spectrum and the second spectrum to compensate for the effect of movement of the speaker or the enclosure on the first spectrum; and detect a target or an attribute of the target based on the compensated radar spectrum. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] For a more complete understanding of the present invention and its advantages, reference is now made to the following description taken in conjunction with the accompanying drawings, in which:

[0008] Figure 1 According to an embodiment of the present invention, a radar system is shown;

[0009] Figure 2 According to an embodiment of the present invention, a circuit diagram of a circuit in a housing of an electronic device is shown. Figure 1 millimeter-wave radar;

[0010] Figure 3 A flow chart illustrating an embodiment method for compensating radar measurements to mitigate interference, according to an embodiment of the present invention;

[0011] Figure 4 According to an embodiment of the present invention, a method implemented as an ARMA filter is shown. Figure 3 Moving average filter;

[0012] Figure 5 According to an embodiment of the present invention, an electronic device is shown;

[0013] Figure 6 A flow chart illustrating an embodiment method for compensating radar measurements to mitigate interference, according to an embodiment of the present invention;

[0014] Figure 7 According to an embodiment of the present invention, an electronic device is shown;

[0015] Figure 8 A flow chart illustrating an embodiment method for compensating radar measurements to mitigate interference, according to an embodiment of the present invention;

[0016] Figure 9 According to an embodiment of the present invention, a method implemented as an ARMA filter is shown. Figure 8 Moving average filter; and

[0017] Figure 10According to an embodiment of the present invention, a flow chart illustrating an embodiment method for mitigating interference using a trained prediction model is shown;

[0018] Corresponding numerals and symbols in the different figures generally indicate corresponding parts unless otherwise indicated.The figures are drawn to clearly illustrate the relevant aspects of the preferred embodiments and are not necessarily drawn to scale. DETAILED DESCRIPTION

[0019] The making and using of the disclosed embodiments are discussed in detail below. However, it should be understood that the present invention provides many applicable inventive concepts that can be embodied in a wide variety of specific circumstances. The specific embodiments discussed are merely illustrative of specific ways to make and use the invention and do not limit the scope of the invention.

[0020] The following description shows various specific details to provide a deep understanding of several example embodiments according to the present description. An embodiment can be obtained without one or more specific details, or by utilizing other methods, components, materials, etc. In other cases, known structures, materials, or operations are not shown or described in detail to avoid confusing different aspects of the embodiment. References to "embodiments" in this specification indicate that a particular configuration, structure, or feature described with respect to that embodiment is included in at least one embodiment. Therefore, phrases such as "in one embodiment" that may appear at different points in this specification do not necessarily refer to the same embodiment precisely. In addition, in one or more embodiments, specific forms, structures, or features may be combined in any appropriate manner.

[0021] Embodiments of the present invention will be described in the specific context of a mobile device having a millimeter wave (mm-wave) radar that uses one or more sensors to mitigate interference caused by actuators of the mobile device. Embodiments of the present invention can be used in other types of devices, such as computers, sound systems, televisions, voice assistants, etc.

[0022] In an embodiment of the present invention, a millimeter wave radar mitigates interference caused by the movement of an object (e.g., a speaker or enclosure). The millimeter wave radar transmits a radar signal and receives a reflected radar signal. The sensor is used to sense a signal indicating the movement of an object. During the processing of the reflected radar signal, the sensing signal is transformed and subtracted from the reflected radar signal or the processed reflected radar signal to reduce or eliminate interference. In some embodiments, the millimeter wave radar generates a first spectrogram based on the reflected radar signal, wherein the first spectrogram includes an interference component caused by the moving object. The millimeter wave radar also generates a second spectrogram based on the sensing signal, and then uses the first spectrogram and the second spectrogram (e.g., subtracting the second spectrogram from the first spectrogram) to obtain a compensated spectrogram, wherein the compensated spectrogram has a reduced interference component compared to the first spectrogram. The compensated spectrogram is then used to perform further radar processing, thereby reducing or eliminating false positives.

[0023] In some embodiments, the object causing interference is a non-target object in the field of view (FoV) of the millimeter-wave radar (ie, an object that the millimeter-wave radar should ignore).

[0024] Millimeter wave radar can be used, for example, to detect moving objects or stationary objects in the field of view. For example, according to an embodiment of the present invention, Figure 1 A millimeter wave radar system 100 is shown. The millimeter wave radar system 100 includes a millimeter wave radar 102 and a processor 104.

[0025] During normal operation, millimeter wave radar 102 transmits radar signal 106, such as a plurality of radiation pulses, such as chirps (e.g., linear chirps), to scene 108. Transmitted radar signal 106 is reflected by objects in scene 108. Reflected radar signal 306 ( Figure 1 ), also known as an echo signal, is detected by the millimeter wave radar 102 and processed by the processor 104 to, for example, identify objects, detect the position and / or Doppler velocity and / or trajectory of objects, and / or other, and / or determine characteristics of objects in the scene 108.

[0026] The objects in scene 108 may include static objects or moving objects. For example, in some embodiments, scene 108 may include one or more of a static person, a moving person, a person's face, a person's finger or hand (e.g., gesturing), a fingerprint, an object that moves periodically such as a fan, a static object such as furniture, etc. Other objects may also be included in scene 108.

[0027] Millimeter wave radar 102 operates as a frequency modulated continuous wave (FMCW) radar or pulse Doppler radar and includes millimeter wave radar sensor circuitry, a transmit antenna, and a receive antenna. Millimeter wave radar 102 transmits and receives signals in the range of 20 GHz to 122 GHz. Alternatively, frequencies outside this range, such as between 1 GHz and 20 GHz, or between 122 GHz and 300 GHz, may be used.

[0028] In some embodiments, the return signal received by the receiving antenna of the millimeter-wave radar 102 is filtered and amplified by, for example, the millimeter-wave radar 102 in a manner known in the art using a bandpass filter (BPF), a low-pass filter (LPF), a mixer, a low-noise amplifier (LNA), and an intermediate frequency (IF) amplifier. The return signal is then digitized using one or more analog-to-digital converters (ADCs) for further processing, for example, by the processor 104. Other implementations are also possible.

[0029] Processor 104 can be implemented as a general-purpose processor, a controller, or a digital signal processor (DSP), which includes, for example, a combination circuit coupled to a memory. In some embodiments, the DSP can be implemented, for example, using an ARM architecture. In some embodiments, processor 104 can be implemented as a customized application-specific integrated circuit (ASIC). In some embodiments, processor 104 includes a plurality of processors, each having one or more processing cores. In other embodiments, processor 104 includes a single processor having one or more processing cores. Other implementations are also possible. For example, some embodiments can implement a decoder using software running in a general-purpose microcontroller or processor, which has, for example, a CPU coupled to a memory and is implemented using an ARM or x86 architecture. Some embodiments can be implemented as a combination of a hardware accelerator and software running on a DSP or a general-purpose microcontroller.

[0030] In some embodiments, the processor 104 is integrated into the millimeter wave radar 102. In other embodiments, the processor 104 is external to the millimeter wave radar 102.

[0031] In some embodiments, the millimeter wave radar 102 may be inside a housing, such as inside a housing of a smart phone. Figure 2 The millimeter wave radar 102 is shown within an enclosure 208 of an electronic device 200. The electronic device 200 includes a speaker 204 and a microphone 206.

[0032] During normal operation, the speaker 204 can reproduce audio, such as songs, human speech, ringtones, etc. To reproduce the audio, the speaker 204 receives an electrical signal (not shown) corresponding to the audio to be reproduced. When the speaker 204 reproduces the audio, a portion of the speaker 204, such as a diaphragm, moves to push air, thereby converting the electrical signal received by the speaker 204 into sound waves 210.

[0033] The speaker 204 can reproduce audio while the millimeter wave radar 102 transmits and receives radar signals to perform, for example, object detection, recognition, and other radar processing. It is possible that when the speaker 204 is reproducing audio, movement of the portion of the speaker 204 (e.g., movement of the diaphragm) is detected as an object by the millimeter wave radar 102. For example, in some embodiments, the speaker 204 is arranged within the field of view of the millimeter wave radar 102. The millimeter wave radar 102 transmits a radar signal 106 that is reflected by the moving diaphragm of the speaker 204 (the moving diaphragm may be reflected multiple times between the diaphragm of the speaker 204 and the enclosure 208), and then reaches the millimeter wave radar 102. This received reflected radar signal 306 ( Figure 2 ) may cause the millimeter wave radar 102 to detect an object, resulting in errors such as false positive errors.

[0034] In one embodiment, microphone 206 is configured to receive sound waves 212 (corresponding to sound waves 210) and convert them into received electrical audio signals. The received electrical audio signals can be used to determine movement of the portion (e.g., the diaphragm) of speaker 204. The determined movement of the portion of speaker 204 can be used to compensate measurements performed by millimeter-wave radar 102 to avoid false positive errors caused by movement of the portion of speaker 204.

[0035] The electronic device 200 may be, for example, a smartphone or tablet. In some embodiments, the electronic device 200 may be a smartwatch, laptop, or other mobile device. In some embodiments, the electronic device 200 may be a desktop computer, a television, a voice assistant, a sound system, or any other device including a microphone, a speaker, and a millimeter-wave radar.

[0036] Speaker 204 can be, for example, a cone-shaped speaker (i.e., the diaphragm is cone-shaped). In some embodiments, speaker 204 can be a piezoelectric speaker. Other types of speakers can also be used. In some embodiments, speaker 204 is inside enclosure 208. In other embodiments, speaker 204 can be outside enclosure 208. For example, in some embodiments, enclosure 208 can be close to (e.g., adjacent to) another device that includes speaker 204. For example, a smartphone that includes enclosure 208 is in a room, and an external device that includes speaker 204 (e.g., another smartphone, a subwoofer, a loudspeaker, etc.) is also in the room and close to (e.g., adjacent to) the smartphone that includes enclosure 208.

[0037] Microphone 206 may be, for example, a digital microphone. In some embodiments, an analog microphone is used.

[0038] In an embodiment of the present invention, the millimeter wave radar mitigates interference caused by the movement of an object (e.g., a speaker or enclosure) by using a microphone to sense sound waves indicating the movement of the object. A first spectrogram is generated based on the reflected signal received by the millimeter wave radar. The audio signal generated by the microphone is used to generate a second spectrogram, which is then combined with the first spectrogram (e.g., subtracted from the first spectrogram) to generate a compensated spectrogram. Advantageously, interference caused by moving objects is minimized or eliminated from the compensated spectrogram, thereby advantageously reducing false positives.

[0039] According to an embodiment of the present invention, Figure 3 A flow chart illustrating an embodiment method 300 for compensating radar measurements to mitigate interference is shown. Method 300 may be performed, for example, by millimeter wave radar system 100 within electronic device 200, for example.

[0040] During step 302, mmWave radar 102 transmits radar signal 106, which reflects off objects in the field of view of mmWave radar 102 to generate reflected radar signal 306. Objects in the field of view of mmWave radar 102 may include enclosure 208, speaker 204, and objects outside enclosure 208, such as hands, fingers, faces, and the like.

[0041] During step 308, millimeter wave radar 102 receives reflected radar signal 306. Reflected radar signal 306 may include a radar signal reflected from a diaphragm of speaker 204, for example.

[0042] During step 312, millimeter wave radar 102 generates a first spectrogram (e.g., using processor 104) based on reflected radar signal 306 received during step 308. Millimeter wave radar 102 may generate the first spectrogram by performing an FFT and then performing an absolute function on reflected radar signal 306. The first spectrogram may include data associated with the movement of the diaphragm of speaker 204.

[0043] During step 304, speaker 204 reproduces audio to produce sound wave 210. During step 310, microphone 206 receives sound wave 212, where sound wave 212 corresponds to sound wave 210. For example, in some embodiments, sound wave 210 reflects off enclosure 208 to generate sound wave 212. In some embodiments, microphone 206 receives sound wave 212 directly from speaker 204 and without reflection off enclosure 208.

[0044] During step 314, the millimeter wave radar 102 generates a second spectrogram (eg, using the processor 104) based on the sound waves 212 received during step 310. The millimeter wave radar 102 may generate the second spectrogram by performing an FFT on the sound waves 212 and then performing an absolute function.

[0045] During step 316, an error spectrum is generated based on the first and second spectrum using a moving average (MA) filter, such as an autoregressive (AR) MA (ARMA) filter (according to an embodiment of the present invention, in Figure 4 ). An example implementation of step 316 is shown in FIG. The error spectrogram is subtracted from the first spectrogram to generate a compensated spectrogram during step 318. The compensated spectrogram minimizes or eliminates data associated with, for example, movement of a diaphragm of speaker 204, thereby mitigating any disturbances caused by, for example, speaker 204 reproducing audio.

[0046] During step 320, the mmWave radar 102 performs further radar processing based on the compensated spectrogram. The further radar processing may include detecting a target (e.g., by comparing a peak of the compensated spectrogram to a threshold value) and determining characteristics of the target, such as the target's Doppler velocity, material, shape, etc.

[0047] Using an ARMA filter to predict interference caused by actuators (such as speakers as described above in an exemplary manner) or other objects (such as an enclosure) based on signals measured by sensors such as microphones advantageously allows millimeter-wave radar to mitigate interference regardless of any nonlinearities in the electronic device and regardless of any manufacturing variations that the electronic device may exhibit.

[0048] In some embodiments, such as when the electronic device is a smartphone or tablet, the electronic device may already include a microphone for, for example, noise cancellation. The same microphone can be used for radar interference mitigation, thereby allowing for better radar performance without adding additional sensors.

[0049] According to an embodiment of the present invention, Figure 4 The moving average filter 316 is shown implemented as an ARMA filter. The moving average filter 316 includes filters 402 and 404 and a summing module 406. The purpose of the moving average filter 316 is to reduce the power of the error generated at the output of the summing module 406 by adjusting the coefficients of the filter 404.

[0050] like Figure 4 As shown in FIG, the first spectrogram includes radar data A associated with the movement of speaker 204 and radar data R associated with other objects in the field of view of millimeter-wave radar 102 (e.g., other than speaker 204). The second spectrogram includes data A' associated with the movement of speaker 204 (e.g., captured by microphone 206). Data A' is similar to data A but not necessarily identical. The difference between data A' and data A may be due to the path taken by sound wave 210 to reach microphone 206 (e.g., due to the linearity of the microphone's transfer function), the path taken by radiation signal 106 to reach the rear millimeter-wave radar 102, etc.

[0051] During normal operation, filter 402 receives a first spectrogram and generates a filtered version of the first spectrogram. Filter 404 receives a second spectrogram and generates a filtered version of the second spectrogram. Figure 4 As shown in FIG, to minimize the power of the error at the output of summing module 406, the coefficients of filter 404 are modified so that filter 404 generates from the second spectrogram substantially similar to (and ideally equal to) the filtered radar data (A 滤波 ) is a filtered version (A' 滤波 ). The output of filter 404 is the error spectrum that is used during step 318 to generate the compensation spectrum.

[0052] In some embodiments, filter 316 uses an algorithm such as steepest descent to minimize the power of the error at the output of summation module 406. Other algorithms may also be used.

[0053] Filters 402 and 404 may be implemented as, for example, finite impulse response (FIR) filters. Summing module 406 may be implemented as a digital block that performs the subtraction of the output of filter 404 from filter 402. Other implementations are also possible.

[0054] In some embodiments, sensors other than microphones may be used to mitigate interference caused by one or more actuators of the mobile device. Figure 5 As an example, electronic device 500 is shown. Electronic device 500 may operate in a similar manner to electronic device 200. However, electronic device 500 includes a haptic actuator 502 and an accelerometer 506.

[0055] During normal operation, the sound waves 210 can cause the enclosure 508 to vibrate. For example, a low frequency of the sound waves 210 (e.g., from 100 Hz to 1 kHz) can cause the enclosure 508 to vibrate. Similarly, the haptic actuator 502, which can vibrate at a frequency from 80 Hz to 500 Hz, can cause the enclosure 508 to vibrate.

[0056] The enclosure 508 can vibrate while the millimeter wave radar 102 transmits and receives radar signals to perform, for example, object detection, identification, and other radar processing. It is possible that the vibration of the enclosure 508 is detected by the millimeter wave radar 102 as an object. For example, in some embodiments, the millimeter wave radar 102 transmits a radar signal 106, which is reflected by the enclosure 508 and then reaches the millimeter wave radar 102. This received reflected radar signal 306 ( Figure 5 ) may cause the millimeter wave radar 102 to detect an object, resulting in a false positive error.

[0057] In one embodiment, accelerometer 506 is used to measure vibrations of enclosure 508. The measured vibrations can be used to determine movement of enclosure 508. The determined movement of enclosure 508 can be used to compensate measurements performed by mmWave radar 102 to avoid false positive errors caused by movement of enclosure 508.

[0058] The haptic actuator 502 can be implemented in any manner known in the art. For example, in some embodiments, the haptic actuator is an eccentric rotating mass (ERM) type. In other embodiments, the haptic actuator 502 is a linear resonant actuator (LRA) type. Other haptic actuator implementations, such as piezoelectric type haptic actuators and brushless DC motor (BLDC) type haptic actuators, are also possible.

[0059] Accelerometer 506 can be implemented in any manner known in the art. For example, in some embodiments, the accelerometer can be attached to enclosure 508. In other embodiments, accelerometer 506 can be attached to a surface that is mechanically coupled to enclosure 508 and vibrates along enclosure 508. Other implementations are also possible.

[0060] In some embodiments, the accelerometer 506 measures vibrations only along the y-axis and passes the measurements to the processor 104 for further processing. In other embodiments, vibrations in other directions, such as along the x-axis and / or the z-axis, are also passed to the processor 104 for further processing.

[0061] According to an embodiment of the present invention, Figure 6 Flowchart showing an embodiment method 600 for compensating radar measurements to mitigate interference. Method 600 may be performed, for example, by millimeter wave radar system 100 within electronic device 500. Figure 3 Steps 302, 304, 316, 318, and 320 are described above.

[0062] During step 608, millimeter wave radar 102 receives reflected radar signal 306. Reflected radar signal 306 may include a radar signal reflected from enclosure 508, for example.

[0063] During step 612, millimeter wave radar 102 generates a first spectrogram (e.g., using processor 104) based on reflected radar signal 306 received during step 608. Millimeter wave radar 102 may generate the first spectrogram by performing an FFT and then performing an absolute function on reflected radar signal 306. The first spectrogram may include data associated with the movement of enclosure 508.

[0064] During step 304 , speaker 204 can reproduce audio to generate sound waves 210 , which can generate vibrations of enclosure 508 . During step 602 , haptic actuator 502 can reproduce haptic effects 606 that generate vibrations of enclosure 508 .

[0065] During step 610, accelerometer 506 measures vibrations of enclosure 508. During step 614, mmWave radar 102 uses these vibration measurements to generate a second spectrogram (e.g., using processor 104). mmWave radar 102 may generate the second spectrogram by performing an FFT on the measured vibrations and then performing an absolute function.

[0066] For example, as regards Figure 3 As described, steps 316, 318, and 320 are performed to generate a compensated spectrogram and perform further radar processing.

[0067] In some embodiments, interference from more than one source may be mitigated. For example, according to embodiments of the present invention, Figure 7 An electronic device 700 is shown. The electronic device 700 includes a speaker 204, a microphone 206, a haptic actuator 502, and an accelerometer 506.

[0068] According to an embodiment of the present invention, Figure 8 Flowchart showing an embodiment method 800 for compensating radar measurements to mitigate interference. Method 800 may be performed, for example, by millimeter wave radar system 100 within electronic device 700. Figure 3 and Figure 6 Steps 302, 304, 310, 320, 602, and 614 are described above.

[0069] During step 808, millimeter wave radar 102 receives reflected radar signal 306. Reflected radar signal 306 may include radar signals reflected from enclosure 508 and / or from a diaphragm of speaker 204, for example.

[0070] During step 812, mmWave radar 102 generates a first spectrogram (e.g., using processor 104) based on reflected radar signal 306 received during step 808. mmWave radar 102 may generate the first spectrogram by performing an FFT and then performing an absolute function on reflected radar signal 306. The first spectrogram may include data associated with movement of enclosure 508 and / or the diaphragm of speaker 204.

[0071] During step 304 , speaker 204 can reproduce audio to generate sound waves 210 , which can generate vibrations of enclosure 508 . During step 602 , haptic actuator 502 can reproduce haptic effects 606 that generate vibrations of enclosure 508 .

[0072] During step 810, the accelerometer 506 and / or the microphone 206 measure vibrations of the enclosure 508 (in some embodiments, the microphone 206 may capture vibrations because such vibrations may produce sounds in the audible range). The mmWave radar 102 uses such acoustic and vibration measurements during steps 314 and 814 to generate second and third spectrograms, respectively (e.g., using the processor 104).

[0073] During step 816, an error spectrum is generated based on the second and third spectrograms using an MA filter, such as an ARMA filter. The error spectrum is subtracted from the first spectrogram to generate a compensation spectrum during step 818. The compensation spectrum minimizes or eliminates data associated with movement of the diaphragm of the speaker 204 and / or movement of the enclosure 508, thereby mitigating any disturbances caused by the reproduction of audio by the speaker 204 and / or by vibrations of the enclosure 508.

[0074] For example, as regards Figure 3 As described, step 320 is executed to perform further radar processing.

[0075] In some embodiments, such as when the electronic device is a smartphone or tablet, the electronic device may already include a microphone and accelerometer for, for example, noise cancellation. The same microphone and accelerometer can be used for radar interference mitigation, thereby allowing for better radar performance without adding additional sensors.

[0076] According to an embodiment of the present invention, Figure 9 The moving average filter 816 is shown implemented as an ARMA filter. The moving average filter 816 includes filters 402, 404, and 902, and summing modules 406 and 904. The purpose of the moving average filter 816 is to reduce the power generation of the error at the output of the summing module 906 by adjusting the coefficients of the filters 404 and 902.

[0077] like Figure 9 As shown in FIG, the first spectrogram includes radar data A associated with the movement of the speaker 204, radar data V associated with the movement of the enclosure 508, and radar data R associated with other objects in the field of view of the millimeter-wave radar 102 (e.g., other than the speaker 204 and the enclosure 508). The second spectrogram includes data A' associated with the movement of the speaker 204 (e.g., captured by the microphone 206). Data A' is similar to data A but not necessarily equal. In some embodiments, the second spectrogram may also include data V' associated with the movement of the enclosure 508 (because some vibrations may occur in the audible range). The difference between data A' and data A and the difference between data V' and V can be attributed to, for example, the linearity of the transfer function of the microphone, the path taken by the sound wave 210 to reach the microphone 206, the path taken by the radiation signal 106 to reach the rear millimeter-wave radar 102, etc.

[0078] The third spectrogram includes data V' associated with the movement of the enclosure 508 (e.g., captured by the accelerometer 506). The data V' is similar to, but not necessarily identical to, the data V. The difference between the data V" and the data V can be attributed to, for example, the linearity of the accelerometer's transfer function, the path taken by the radiated signal 106 to reach the rear millimeter-wave radar 102, etc.

[0079] During normal operation, filters 402, 404, and 902 receive the first, second, and third spectrograms, respectively, and generate filtered versions of the first, second, and third spectrograms, respectively. Figure 4 As shown in FIG, to minimize the power of the error at the output of the summing module 406, the coefficients of the filters 404 and 902 are modified so that R 滤波 +A 滤波 +V 滤波 –A' 滤波 –V’ 滤波 –V” 滤波 Equal to R滤波 Summing module 904 adds the outputs of filters 404 and 902 to generate an error spectrum that is used during step 818 to generate a compensation spectrum.

[0080] In some embodiments, filter 816 uses an algorithm such as steepest descent to minimize the power of the error at the output of summation module 906. Other algorithms may also be used.

[0081] Filters 402, 404, and 902 may be implemented as FIR filters, for example. Summing modules 904 and 906 may be implemented as digital blocks that perform addition and / or subtraction. Other implementations are also possible.

[0082] In some embodiments, the estimation of interference may be performed by using a predetermined model, such as a pre-trained model. For example, according to an embodiment of the present invention, Figure 10 A flow chart illustrating an embodiment method 1000 for mitigating interference using a trained prediction model 1002 is shown. The method 1000 may be performed, for example, by a millimeter wave radar system 100 within an electronic device 200, 700, or 900, for example.

[0083] like Figure 10 As shown in , enter X t-N To X t is fed to the trained prediction model 1002, where the input X t-N To X t The prediction model 1002 then calculates the movement of non-target objects such as speakers or enclosures based on the input X. t-N To X t Generate prediction output. The summation module 1010 takes input X as t Subtract the predicted output X t To generate the compensation output X t , where the compensation output X t Does not include information corresponding to the movement of non-target objects (or when compared to the input X t Then during step 1012, the compensation output X t Used for further radar processing.

[0084] In some embodiments, input X t-N to X t is the Mel-Frequency Cepstral Coefficient (MFCC). In such an embodiment, the output X t is the time series prediction of MFCC (at time t). In other embodiments, the input X t-N to X t is the raw time domain data from the millimeter wave radar 102. In such an embodiment, the output Xt Is the time domain prediction of the next time domain original data (the predicted value of the original time domain data at time t). In other embodiments, the input X t-N To X t is the feature extracted from the previous time-discrete convolutional neural network (CNN). In such an embodiment, the output X t is the feature extracted by CNN at time t. In other embodiments, the input X t-N To X t For example, the first spectrum (eg, where input X t-N To X t ) together forming a first spectrogram, such as the first spectrogram generated in steps 312, 612, or 812. In such an embodiment, output X t is a predicted spectrogram indicative of movement of non-target objects such as enclosure 208 (or 508) and / or speaker 204, where speaker 204 may be inside enclosure 208 (or 508) or outside enclosure 208 (or 508).

[0085] In some embodiments, the trained prediction model 1002 includes a long short-term memory network (LSTM) autoencoder, which includes an encoder LSTM model 1004, a decoder LSTM model 1006, and a dense and sigmoid activation module 1008. The LSTM autoencoder can be understood as a neural network model whose purpose is to learn a compressed representation of the input in order to recreate the input.

[0086] During normal operation, the encoder LSTM model 1004 reads the input X step by step t-N to X t . When reading input X t-N to X t After the entire sequence is obtained, the encoder LSTM model 1004 generates a fixed-length vector. The decoder LSTM model 1006 receives the fixed-length vector and generates a predicted sequence. The dense and sigmoid activation module 1008 can be used to receive the predicted sequence and recreate the input by applying weights to the predicted sequence. Other LSTM autoencoder implementations known in the art can also be used.

[0087] In some embodiments, the trained prediction model 1002 can be trained after the device is assembled. For example, once the electronic device 200 is assembled and the speaker 204, the millimeter wave radar 102, and the microphone 206 are inside the enclosure 208, the prediction model 1002 is trained, for example, based on known audio received by the microphone 206. The known audio can be reproduced, for example, by the speaker 204, which can be inside or outside the enclosure (e.g., 208 or 508) of the device.

[0088] In some embodiments, each manufactured electronic device (e.g., 200 or 700) undergoes training of the prediction model 1002. Thus, each manufactured electronic device can have a different trained prediction model 1002 because the trained prediction model 1002 is based on specific characteristics of, for example, the microphone 206, the speaker 204, and the enclosure 208, which can vary from electronic device to electronic device. Using the trained prediction model 1002 advantageously allows for better convergence and accuracy than implementations using, for example, an ARMA filter.

[0089] Example embodiments of the invention are summarized herein. Other embodiments can be understood from the overall description and claims submitted herein.

[0090] Example 1. A method for interference mitigation in a device including a millimeter-wave radar, the method comprising: transmitting a radar signal using the millimeter-wave radar; receiving a reflected radar signal using the millimeter-wave radar, the reflected radar signal corresponding to the transmitted radar signal; generating a first spectrum based on the reflected radar signal; generating a second spectrum indicating movement of a non-target object; generating a compensated radar spectrum based on the first spectrum and the second spectrum to compensate for the effect of the movement of the non-target object in the first spectrum; and detecting a target or a characteristic of a target based on the compensated radar spectrum.

[0091] Example 2. The method of Example 1, wherein the non-target object comprises an actuator, the method further comprising measuring an output of the actuator using a sensor to generate a measurement output, wherein the second spectrogram is based on the measurement output.

[0092] Example 3. The method of one of Examples 1 or 2, further comprising activating an actuator, wherein the device comprises an actuator.

[0093] Example 4. The method of one of Examples 1 or 2, wherein the actuator is external to the device.

[0094] Example 5. The method of one of Examples 1 to 4, wherein the actuator comprises a speaker, wherein the sensor is a microphone, and wherein the output of the actuator comprises a sound wave.

[0095] Example 6. The method of one of Examples 1 to 5, wherein the non-target object comprises a housing of the device, the method further comprising: measuring vibrations of the housing of the device using an accelerometer; and generating a third spectrum based on output of the accelerometer, wherein generating the compensated radar spectrum further comprises subtracting the third spectrum from the first spectrum.

[0096] Example 7. The method of one of Examples 1 to 6, wherein the actuator comprises a tactile actuator, and wherein the sensor is an accelerometer.

[0097] Example 8. The method of one of Examples 1 to 7, wherein generating the compensated radar spectra comprises subtracting the second spectra from the first spectra.

[0098] Example 9. The method of one of examples 1 to 8, wherein generating the second spectrogram includes using a moving average filter.

[0099] Example 10. The method of one of Examples 1 to 9, wherein the generative moving average filter is an autoregressive moving average (ARMA) filter.

[0100] Example 11. The method of one of Examples 1 to 8, wherein generating the second spectrogram comprises predicting the second spectrogram based on a model that receives the reflection-based radar signal as input data.

[0101] Example 12. The method of one of Examples 1 to 8 or 11, wherein the model is based on a neural network.

[0102] Example 13. A method according to one of Examples 1 to 8 or 11 to 12, wherein the model is based on a recurrent neural network.

[0103] Example 14. The method of one of Examples 1 to 8 or 11 to 13, wherein the model is a long short-term memory (LSTM) recurrent neural network.

[0104] Example 15. The method of one of Examples 1 to 8 or 11 to 14, further comprising generating the model by using a training set.

[0105] Example 16. A device includes: a millimeter wave radar configured to transmit a radar signal and receive a reflected radar signal, wherein the reflected radar signal corresponds to the transmitted radar signal; and a controller configured to: generate a first spectrum based on the reflected radar signal; generate a second spectrum indicating the movement of a non-target object; generate a compensated radar spectrum based on the first spectrum and the second spectrum to compensate for the effect of the movement of the non-target object in the first spectrum; and detect a target or an attribute of a target based on the compensated radar spectrum.

[0106] Example 17. The apparatus of Example 16, wherein the non-target object comprises an actuator, the apparatus further comprising a sensor configured to measure an output of the actuator to produce a measurement output, wherein the controller is configured to generate the second spectrogram based on the measurement output.

[0107] Example 18. The device of one of Examples 16 or 17, further comprising an actuator.

[0108] Example 19. The apparatus of one of Examples 16 to 18, further comprising an enclosure at least partially surrounding the millimeter wave radar and the actuator.

[0109] Example 20. The device of one of Examples 16 to 19, wherein the enclosure further at least partially surrounds the sensor.

[0110] Example 21. The apparatus of one of Examples 16 to 20, wherein generating the compensated radar spectra comprises subtracting the second spectra from the first spectra.

[0111] Example 22. The apparatus of one of Examples 16 to 21, wherein generating the second spectrogram comprises predicting the second spectrogram based on a model that receives data based on the reflected radar signal as input data.

[0112] Example 23. A mobile device includes: a millimeter-wave radar configured to transmit a radar signal and receive a reflected radar signal, wherein the reflected radar signal corresponds to the transmitted radar signal; a speaker; a microphone configured to measure the output of the speaker to generate a measurement output; an enclosure at least partially surrounding the millimeter-wave radar, the speaker, and the microphone; and a controller configured to: generate a first spectrum based on the reflected radar signal; generate a second spectrum based on the measurement output; generate a compensated radar spectrum based on the first spectrum and the second spectrum to compensate for the effect of movement of the speaker or the enclosure on the first spectrum; and detect a target or an attribute of a target based on the compensated radar spectrum.

[0113] Example 24. The mobile device of example 23, wherein generating the second spectrogram comprises using an autoregressive moving average (ARMA) filter.

[0114] Example 25. The mobile device of one of Examples 23 or 24, wherein generating the compensated radar spectrum comprises subtracting the second spectrum from the first spectrum.

[0115] Although the present invention has been described with reference to illustrative embodiments, this description is not intended to be construed in a limiting sense. Various modifications and combinations of the illustrative embodiments, as well as other embodiments of the present invention, will be apparent to those skilled in the art in view of this specification. Accordingly, the appended claims are intended to cover any such modifications or embodiments.

Claims

1. A method for mitigating interference in a device including a millimeter-wave radar, the method comprising: emitting a radar signal using the millimeter-wave radar; receiving, using the millimeter-wave radar, a reflected radar signal corresponding to the transmitted radar signal; generating a first spectrogram based on the reflected radar signal; generating a second spectrogram indicative of movement of a first one of the non-target objects; generating a compensated radar spectrum based on the first spectrum and the second spectrum to compensate for the effect of the movement of the first non-target object in the first spectrum; as well as detecting a target or a characteristic of the target based on the compensated radar spectrum, Wherein a second non-target object among the non-target objects is a casing of the device, the method further comprises: measuring vibration of the housing of the device using an accelerometer; and A third spectrogram is generated based on the output of the accelerometer, wherein generating the compensated radar spectrogram further comprises subtracting the third spectrogram from the first spectrogram. 2 . The method of claim 1 , wherein a first one of the non-target objects is an actuator, the method further comprising measuring an output of the actuator using a sensor to generate a measured output, wherein the second spectrogram is based on the measured output.

3. The method of claim 2, further comprising activating the actuator, wherein the device comprises the actuator. The method of claim 2 , wherein the actuator is external to the device.

5. The method of claim 2, wherein the actuator comprises a speaker, wherein the sensor is a microphone, and wherein the output of the actuator comprises sound waves. The method of claim 2 , wherein the actuator comprises a haptic actuator, and wherein the sensor is an accelerometer. 7 . The method of claim 1 , wherein generating the compensated radar spectra comprises subtracting the second spectra from the first spectra. The method of claim 1 , wherein generating the second spectrogram comprises using a moving average filter.

9. The method of claim 8, wherein generating the moving average filter is an autoregressive moving average (ARMA) filter.

10. The method of claim 1, wherein generating the second spectrogram comprises predicting the second spectrogram based on a model that receives as input data the radar signal based on the reflections. The method of claim 10 , wherein the model is based on a neural network.

12. The method of claim 10, wherein the model is based on a recurrent neural network.

13. The method of claim 10, wherein the model is a long short-term memory (LSTM) recurrent neural network. The method of claim 10 , further comprising generating the model by using a training set.

15. A radar device comprising: a millimeter-wave radar configured to transmit a radar signal and receive a reflected radar signal, wherein the reflected radar signal corresponds to the transmitted radar signal; as well as The controller is configured to: generating a first spectrogram based on the reflected radar signal; generating a second spectrogram indicative of movement of a first one of the non-target objects; generating a compensated radar spectrum based on the first spectrum and the second spectrum to compensate for the effect of the movement of the first non-target object in the first spectrum; detecting a target or a characteristic of the target based on the compensated radar spectrum, measuring vibration of a second non-target object among the non-target objects using an accelerometer, wherein the second non-target object is a casing of the radar apparatus; and A third spectrogram is generated based on the output of the accelerometer, wherein generating the compensated radar spectrogram further comprises subtracting the third spectrogram from the first spectrogram.

16. The radar apparatus according to claim 15, wherein a first non-target object among the non-target objects is an actuator, the radar apparatus further comprising a sensor configured to measure an output of the actuator to generate a measurement output, wherein the controller is configured to generate the second spectrogram based on the measurement output. The radar apparatus according to claim 16 , further comprising the actuator. 18 . The radar apparatus according to claim 17 , wherein the enclosure at least partially surrounds the millimeter-wave radar and the actuator. The radar apparatus according to claim 18 , wherein the enclosure further at least partially surrounds the sensor.

20. The radar apparatus of claim 15, wherein generating the compensated radar spectrum comprises subtracting the second spectrum from the first spectrum.

21. The radar apparatus of claim 15, wherein generating the second spectrogram comprises predicting the second spectrogram based on a model that receives as input data the radar signal based on the reflections.

22. A mobile device comprising: a millimeter-wave radar configured to transmit a radar signal and receive a reflected radar signal, wherein the reflected radar signal corresponds to the transmitted radar signal; speaker; a microphone configured to measure an output of the speaker to generate a measurement output; a casing at least partially surrounding the millimeter-wave radar, the speaker, and the microphone; as well as The controller is configured to: generating a first spectrogram based on the reflected radar signal; generating a second spectrum based on the measurement output; generating a compensated radar spectrogram based on the first spectrogram and the second spectrogram to compensate for the influence of the speaker in the first spectrogram; or measuring the vibration of the enclosure using an accelerometer, generating a third spectrogram based on the output of the accelerometer, Based on the The second spectrum and the third spectrum generate an error spectrum, and a compensation radar spectrum is generated based on the first spectrum and the error spectrum. picture , to compensate for the influence of the movement of the shell in the first spectrum; An object or a characteristic of the object is detected based on the compensated radar spectrum.

23. The mobile device of claim 22, wherein generating the second spectrogram comprises using an autoregressive moving average (ARMA) filter.

24. The mobile device of claim 22, wherein generating the compensated radar spectra comprises subtracting the second spectra from the first spectra.

Images