Radar-based noise filtering

Abstract

In an example implementation, a device is provided. The device includes a microphone, a radar, a memory, and a processor in communication with the microphone, the radar, and the memory. The microphone is configured to receive an audio signal. The radar is configured to collect data regarding a user in a location. The memory is configured to store known body positions associated with engaging in a private conversation. The processor is configured to determine that the user is engaging in a private conversation based on the data collected by the radar compared to the known body positions associated with engaging in a private conversation, and to filter noise associated with the private conversation received by the microphone from a direction associated with the user.

Description

Background Art

[0001] Businesses use various types of communication and communication devices to improve productivity. One type of communication is teleconferencing. For example, a speaker and microphone device can be used in a conference room with many people and connected to another device in a different geographical location. These devices allow individuals to speak to each other from different locations during a teleconference. A teleconference can be conducted using a speaker and microphone device or using a computer that can also provide video images and audio. BRIEF DESCRIPTION OF THE DRAWINGS

[0002] Figure 1 is a block diagram of an example apparatus with radar and noise filtering of the present disclosure;

[0003] Figure 2 is a block diagram of an apparatus operating in a conference room of the present disclosure;

[0004] Figure 3 is a block diagram of an example of a radar image of the present disclosure;

[0005] Figure 4 is a block diagram of noise filtering of radar information based on the present disclosure;

[0006] Figure 5 is a flow chart of an example method for generating an audio signal with noise filtering based on data collected from a radar of the present disclosure; and

[0007] Figure 6 is a block diagram of an exemplary non-transitory computer-readable storage medium storing instructions executable by a processor to generate an audio signal with noise filtering based on data collected from a radar. DETAILED DESCRIPTION

[0008] The examples described herein provide an apparatus and method for generating an audio signal with noise filtering based on data collected from radar. As noted above, some businesses can use teleconferencing to improve productivity. A teleconference may have many individuals in a conference room speaking to an individual in another location.

[0009] In some instances, individuals in a conference room may begin to have side conversations with each other. Side conversations may generate additional noise and make it difficult for individuals on the other end of the conference call to hear one or more speakers. Side conversations may be distracting.

[0010] The examples herein provide a device with radar to detect when a private conversation is occurring and perform noise filtering to remove the private conversation from the sound or audio signal. For example, the radar can detect body orientation. Certain body orientations can determine whether a person is listening to a speaker or engaging in a private conversation with another user.

[0011] When radar information detects that one or more individuals in a conference room are engaged in a private conversation, noise or audio from these individuals can be filtered out of the generated audio signal. For example, radar can gather information about where individuals are seated in the conference room. The microphone on the device collecting audio signals from the determined direction can be muted, or the audio collected by the microphone can be filtered out.

[0012] The device can generate an audio signal from which private conversations are filtered out. As a result, the transmitted audio signal may be clearer, and individuals on the other end of the conference call can hear the speaker more clearly without being distracted by the private conversation.

[0013] Figure 1 An example apparatus 100 of the present disclosure is illustrated. In one example, apparatus 100 may be a computing device such as a desktop computer, laptop computer, tablet computer, or the like. In one example, apparatus 100 may be a teleconferencing device. For example, a teleconferencing device may be a device that can be placed in a conference room and connected to another remotely located teleconferencing device to transmit and receive audio signals.

[0014] In one example, apparatus 100 may include a processor 102, a memory 104, a radar 108, and a microphone 110. In one example, processor 102 may be communicatively coupled to memory 104, radar 108, and microphone 110. Processor 102 may control the operation of radar 108 and microphone 110. Processor 102 may also execute instructions stored in memory 104 to perform the functions described herein.

[0015] In one example, the memory 104 may be a non-transitory computer-readable medium. For example, the memory 104 may be a hard drive, a solid-state drive, a random access memory (RAM), a read-only memory (ROM), etc. The memory 104 may include multiple memory devices (e.g., a hard drive and RAM).

[0016] In one example, memory 104 may store known body positions associated with conducting a private conversation 106 (also referred to herein as "known body positions" 106). Known body positions 106 may be predefined or pre-learned. Known body positions 106 may be used to determine whether the user's position data indicates that the user is conducting a private conversation. Processor 102 may then filter noise associated with the private conversation from the generated audio signal.

[0017] In one example, as radar 108 collects more position data of the user at a particular location over time, known body position 106 can be dynamically updated. For example, machine learning or deep learning can be used to dynamically update known body position 106 at a particular location for a particular group of users over time. Thus, known body position 106 can be customized for a particular group of users at a particular location over time.

[0018] In one example, radar 108 can collect positional data for one or more users in a location or room. Radar 108 can be a millimeter wave detection device that can transmit a radio frequency (RF) signal and measure the response of the RF signal after it bounces off an object. Although a millimeter wave detection device is used, other types of radar devices can also be implemented, such as light wave detection devices or lidar.

[0019] Radar 108 may transmit multiple RF signals that collect multiple positional data points of one or more users in the room. The data collected by radar 108 may include the distance from radar 108 or device 100, the angle at which the user is sitting relative to device 100, the user's motion vector, the user's orientation relative to device 100, etc.

[0020] In one example, radar 108 can detect positional data over 180 degrees. Thus, in a room where each person sits in front of device 100, a single radar 108 can be used. In a room where users sit around device 100, two radars 108 can be placed back-to-back to obtain positional data for the user at all angles around the room, 360 degrees.

[0021] Figure 3 An example of a radar image 302 is illustrated in FIG. As noted above, the radar 108 may transmit a plurality of RF signals that collect a plurality of position data points 304 1 to 304 . l (Also referred to individually or collectively as position data points 304 hereinafter.) In one example, the position of the user may be determined based on an average of position data points 304. For example, the distance of the user from radar 108 or device 100 may be the average of the distances from each of position data points 304.

[0022] In one example, the user's orientation or posture can be determined based on the shape of orientation data points 304. For example, orientation data points 304 above line 306 can represent the user's head. Orientation data points 304 below line 306 can represent the user's torso. Thus, the user's orientation or posture can be determined based on the estimated orientation of the user's head and torso.

[0023] For example, a certain arrangement of orientation data points 304 may indicate that the user's head is turned in a particular direction. A certain arrangement of orientation data points 304 may indicate that the user's torso is leaning away from device 100 or turning in a particular direction.

[0024] The angle of the user relative to device 100 can also be estimated based on the arrangement of position data points 304. For example, when the user is facing directly toward device 100 (e.g., associated with 0 degrees relative to device 100), the arrangement of position data points 304 can have a maximum width. When the user turns their torso left or right, the torso may appear narrower. When the user turns sideways relative to device 100 (e.g., associated with 90 degrees), the arrangement of position data points 304 can have a minimum width. When the user is at 45 degrees, the arrangement of position data points 304 can have a width between the maximum and minimum widths.

[0025] As noted above, known body positions 106 may store known body positions associated with conducting private conversations. The arrangement of position data points 304 may be compared to positions stored in known body positions 106 (eg, a predefined arrangement of position data points 304).

[0026] For example, a known body orientation 106 may associate a user positioned at an angle of 45 degrees or greater relative to the device 100 with engaging in a private conversation (e.g., the user is turned toward another user). In another example, if the user's torso is leaning away from the device and the user's head is turned left or right, the user may be engaging in a private conversation (e.g., the user is leaning back and speaking to someone behind the user). In another example, if the torso is leaning forward and the head is lowered, the user may be engaging in a private conversation (e.g., the user is ducking downward to avoid being seen while the user is engaging in a private conversation). The examples described herein are just a few examples of possible known body orientations associated with engaging in a private conversation. Other examples may be within the scope of this disclosure.

[0027] In some examples, known body positions 106 may store known body positions for multiple users. For example, a private conversation may occur between two users who are at a certain angle relative to each other or within a distance threshold between the users. Position data points 304 for two adjacent users may be compared to a predefined arrangement of position data points 304 for adjacent users in known body positions 106 to determine whether the two users are engaging in a private conversation.

[0028] Furthermore, if there are multiple users in the room, position data points 304 may be collected for each user. The radar 108 may also collect directional data. For example, motion vector data may indicate whether the user is moving and in which direction the user is positioned relative to the radar 108.

[0029] If the arrangement of position data points 304 matches a predefined arrangement in known body positions 106, processor 102 may determine that the user is having a private conversation. Processor 102 may then filter out noise from the private conversation from the generated audio signal.

[0030] In one example, the position data points 304 of the users in the room can be tracked continuously. In one example, the position data points 304 of the users in the room can be tracked periodically (e.g., every 10 seconds, every 30 seconds, every minute, etc.).

[0031] In one example, known body positions 106 may also store body positions that may be associated with other types of movement that may generate distracting noises. For example, known body positions 106 may store body positions associated with a person leaving a room, which may generate distracting noises such as a creaking chair, rustling clothes, or a door closing. Known body positions 106 may also store body positions associated with a person eating, which may generate distracting noises such as chewing, crunching, or rustling wrapping paper.

[0032] Return Reference Figure 1 , the apparatus 100 may further include a microphone 110. The microphone 110 may receive an audio signal from the room in which the apparatus 100 is located. The audio signal may include the voice of a speaker, or any other person speaking in the room.

[0033] When a private conversation is taking place in a room, the noise and / or speech associated with the private conversation may be distracting to a listener attempting to hear the speaker on another remotely connected device 100. Therefore, when the processor 102 determines that a user is engaging in a private conversation, the processor 102 may filter the noise associated with the private conversation from the audio signal captured from the user in the room. In other words, the audio signal may be modified to remove speech or noise associated with the detected private conversation.

[0034] Figure 4 An example of microphone 110 is shown. In one example, microphone 110 may include multiple microphones or microphone inputs 4021 to 4022. o (Also referred to individually or collectively as microphones 402 hereinafter.) Each microphone 402 may be responsible for receiving audio signals in a specific angular direction relative to the center of microphone 110. For example, there may be 12 microphones 402 arranged around microphone 110. Each microphone 402 may be associated with a 30-degree coverage area. Each microphone 402 may receive audio signals from different directions 4041 to 4040 (hereinafter also referred to as directions 404 or collectively as directions 404 ) within a corresponding 30-degree range.

[0035] In one example, with the user sitting in front of the device 100, half of the microphones 402 can be used to obtain 180-degree coverage (e.g., similar to the radar 108). When the user sits around the device 100, all microphones 402 can be used to obtain 360-degree coverage.

[0036] As noted above, processor 102 can determine the direction of the user relative to device 100. Processor 102 can then determine that microphone 402 covers direction 404 of the user having a private conversation. For example, the user may be having a private conversation from direction 4042 covered by microphone 4022. In other words, the direction of the user may be within the angular range covered by microphone 4022. In other words, the angular range of microphone 4022 may include directions from which noise associated with the private conversation is received.

[0037] In one example, the processor 102 may filter the noise by muting the microphone 4022. Thus, the audio signal from the direction covered by the microphone 4022 may be eliminated from the audio signal generated by the processor 102. In other words, the processor 102 may selectively control the operation of each microphone 402 based on the microphone 402 covering the direction from which the user engaging in the private conversation is detected.

[0038] In one example, the processor 102 may digitally filter the noise. For example, the processor 102 may identify that a user in the direction covered by the microphone 4022 is having a private conversation. The processor 102 may then digitally remove the audio signal received by the microphone 4022. In other words, the microphones 4021 to 4022 may be used to filter the noise. o can remain on, and the processor 102 can digitally remove the audio signal from one of the microphones 4021 to 402 o In one example, when the audio signals from the specific microphones 4021 to 402 o When the audio signal is filtered out, a modified audio signal can be generated that excludes noise or speech associated with the private conversation.

[0039] In one example, processor 102 can continue to collect position data points 304 from radar 108 to determine whether a private conversation is occurring. For example, after microphone 4022 is muted or the audio signal is digitally removed, processor 102 can continue to analyze position data points 304 for the user in the direction covered by microphone 4022. At a later time, the user may stop having the private conversation. As a result, processor 102 can unmute microphone 4022 or add the audio signal received by microphone 4022 back to the audio signal or signals received during the conference call.

[0040] In one example, in addition to the known body position 106, additional characteristics can be used to determine whether a private conversation is occurring. For example, the volume level of noise or speech received in conjunction with the known body position 106 can be used to determine whether a private conversation is occurring. For example, a private conversation may have a lower volume than other speech from a user who is speaking at a given location. Therefore, if the speech or noise is below a volume threshold and the user's position data point 304 matches the known body position 106, the processor 102 can determine that a private conversation is occurring.

[0041] It should be noted that although Figure 4 12 microphones 402 are illustrated by way of example in FIG, but any number of microphones may be deployed. For example, more than 12 microphones may be deployed for greater granularity, or fewer than 12 microphones may be deployed for lower cost but less accuracy.

[0042] Return Reference Figure 1It should be noted that the apparatus 100 has been simplified for ease of explanation. For example, the apparatus 100 may include additional components not shown, such as a display, a speaker, a user interface, input buttons or controls, and the like.

[0043] In one example, the known body positions 106 may also be stored in a remotely located server. The known body positions 106 may be updated over time based on real-time data at the location where the device 100 is deployed. The remotely located server may then upload the updated known body positions 106 to other devices 100 located at different locations. Thus, the known body positions 106 stored locally at the device 100 may remain up to date even if they are not frequently used at a particular location.

[0044] Figure 2 An example block diagram of the apparatus 100 operating in a conference room is illustrated. Figure 2 The diagram shows a case where the device 100 faces users 2101 to 210 n (hereinafter also referred to as user 210 individually or collectively as user 210) arrangement 202, and the users 2121 to 212 therein m An arrangement 204 (also referred to hereinafter individually as users 212 or collectively as users 212 ) is seated around the device 100 .

[0045] In arrangement 202, device 100 may be placed toward the front of a conference room on table 206. User 210 may be sitting in front of device 100 and within 180 degrees of radar 108.

[0046] In one example, radar 108 may collect positional data of user 210 as described above. Processor 102 may begin analyzing the positional data of user 210 collected by radar 108 after user 210 sits down. For example, user 210 may enter a room and have a motion vector measured by radar 108 indicating movement greater than a threshold. When user 210 sits down, the motion vector of the positional data collected by radar 108 may be zero or less than the threshold. When the motion vector is less than the threshold, processor 102 may determine that user 210 is sitting down.

[0047] In other words, when user 210 is moving, processor 102 can determine that user 210 is entering the room and that the meeting has not yet started. Therefore, user 210 is free to conduct a different private conversation. However, after user 210 sits down, processor 102 can determine that the meeting has started and begin filtering noise from the private conversation to allow the audio signal generated by the speaker to be as clear as possible.

[0048] After the processor 102 determines that the user 210 is sitting down and the meeting begins, the radar 108 can collect the information of each user 2101-210 n Position data of each user 2101-210 n The position data of users 2102 and 2103 may be compared with predefined positions in known body positions 106. At a later time, processor 102 may determine that the position data of users 2102 and 2103 indicates that users 2102 and 2103 are having a private conversation while user 2101 is speaking.

[0049] Processor 102 may identify microphone 402 that covers audio signals from directions associated with users 2102 and 2103. In some examples, users 2102 and 2103 may be within range associated with multiple microphones 402 (e.g., microphone 4023 may cover the direction associated with user 2102, and microphone 4024 may cover the direction associated with user 2103). Processor 102 may then mute the identified microphone 402 or digitally remove the audio signal received by the identified microphone 402 from the audio signal generated by processor 102.

[0050] Arrangement 204 illustrates apparatus 100 placed on table 208 in a room. User 212 may be seated around apparatus 100. Apparatus 100 may include two radars 1081 and 1082 arranged back-to-back to provide 360-degree coverage of positional data of user 212 around apparatus 100. For example, each radar 1081 and 1082 may provide 180-degree coverage of user positional data around apparatus 100.

[0051] The device 100 may operate in arrangement 204 similar to how the device 100 is described as operating in arrangement 202. Thus, the device 100 may filter noise associated with private conversations from users 212 sitting around the device or from user 210 sitting in front of the device 100.

[0052] Figure 5 A flow chart of an example method 500 for generating an audio signal with noise filtering based on data collected from a radar of the present disclosure is illustrated. In one example, the method 500 may be performed by Figure 6 The method may be performed by the apparatus 100 or the apparatus 600 illustrated in FIG and discussed below.

[0053] At block 502, method 500 begins. At block 504, method 500 collects radar data from users to determine the location of each user. For example, the radar data can be collected by a radar coupled to an apparatus or computing device. The radar can be a millimeter wave detection device that transmits RF signals toward an object and collects location data based on the returned RF signals. The location data can include the angle at which the user is sitting relative to the device, the distance from the device, the user's motion vector, the user's direction relative to the device, and the like.

[0054] In one example, radar data may include multiple position data points for each user. Some of the position data may be an average of the position data points. For example, the user's distance may be based on the average distance of each position data point for the user. Some of the position data may be derived based on the arrangement of the position data points. For example, the width of a position data point may be related to the angle relative to the device, or the shape of a position data point may indicate posture, how the user is tilted, etc.

[0055] At block 506, method 500 compares the position of each user to known body positions associated with users engaging in private conversations. For example, a memory may store body positions known to be associated with users engaging in private conversations. The body positions may be predefined arrangements of position data points. The body positions may be for multiple users. For example, the body positions of two users leaning toward each other, the body positions of two users within a predefined distance of each other, and so on.

[0056] In one example, the comparison can begin when a delta in radar data falls below a threshold. For example, the delta in certain radar data can be tracked to determine when comparison block 506 should begin. In one example, the radar data can be a user's motion vector. For example, if the delta in the user's motion vector at two different points in time falls below a threshold, it can be assumed that the user is sitting down and the meeting has already begun. However, if the delta in the motion vector is above the threshold, it can be assumed that the user is still entering the room and the meeting has not yet begun.

[0057] At block 508, method 500 determines that the user is engaging in a private conversation. For example, when the arrangement of the position data points of the user or users matches a predefined arrangement of position data points known to be associated with a private conversation, the user may be engaging in a private conversation. In response, a microphone or multiple adjacent microphones covering the direction associated with the user engaging in the private conversation may be identified.

[0058] At block 510, method 500 filters noise associated with the private conversation from the audio signal. In one example, the noise can be filtered by muting one or more microphones identified as covering a direction associated with the user engaging in the private conversation. In one example, the noise can be digitally filtered by removing the audio signal received by the identified microphone from the audio signal generated by the processor for transmission (e.g., the audio signal from the speakers in the room). The modified audio signal, from which the noise from the private conversation has been removed, can then be transmitted as part of the conference call.

[0059] In one example, blocks 504-510 may be repeated continuously. As a result, the identified microphone may be turned back on, or the audio signal received by the identified microphone may be added to the audio signal generated when the private conversation ends. For example, the radar data may no longer indicate that the user is engaging in a private conversation. In other words, the arrangement of the position data points may no longer match the predefined arrangement of position data points associated with the body position indicating that the private conversation is occurring. At block 512, method 500 ends.

[0060] Figure 6 An example of an apparatus 600 is illustrated. In one example, the apparatus 600 may be the apparatus 100. In one example, the apparatus 600 may include a processor 602 and a non-transitory computer-readable storage medium 604. The non-transitory computer-readable storage medium 604 may include instructions 606, 608, 610, and 612 that, when executed by the processor 602, cause the processor 602 to perform various functions.

[0061] In one example, instructions 606 may include instructions for determining when the movement of a user in the room is below a threshold. Instructions 608 may include instructions for collecting radar data of the user in response to the user's movement being below the threshold to determine the position of each of the users in the room. Instructions 610 may include instructions for determining that the user is engaging in a private conversation based on the user's position matching a known body position associated with engaging in a private conversation. Instructions 612 may include instructions for filtering noise associated with a private conversation from the audio signal.

[0062] It will be appreciated that variations of the above-disclosed and other features and functions, or alternatives thereto, may be combined into many other different systems or applications. Various presently unforeseen or unanticipated alternatives, modifications, variations, or improvements therein may subsequently be made by those skilled in the art, which alternatives, modifications, variations, or improvements are also intended to be covered by the appended claims.

Claims

1. An apparatus for filtering noise associated with a private conversation in a conference room, comprising: a microphone for receiving audio signals; radar, which collects data about users in a location; a memory for storing known body positions associated with conducting a private conversation; as well as A processor communicates with the microphone, radar, and memory, wherein the processor is used to: determining that the user is engaging in a private conversation based on data collected by the radar compared to the known body position associated with engaging in a private conversation; as well as Noise associated with the private conversation received by a microphone from a direction associated with the user is filtered. 2 . The apparatus of claim 1 , wherein the radar comprises a millimeter wave detection device that collects data about the user within a span of 180 degrees. 3 . The apparatus according to claim 2 , wherein the radar comprises two millimeter wave detection devices arranged back to back to collect data about the user within a span of 360 degrees.

4. The device of claim 1, wherein the data includes an angle at which a user is sitting relative to the device, a distance from the device, and a motion vector of the user. The apparatus of claim 1 , wherein the microphone comprises a plurality of microphones, wherein each of the plurality of microphones is associated with an angular range. 6 . The apparatus of claim 5 , wherein the processor is configured to filter out the noise by muting one of the plurality of microphones, the one microphone being associated with an angular range that includes a direction associated with the user.

7. A method for filtering noise associated with a private conversation in a conference room, comprising: collecting radar data of the users to determine a position of each of the users; comparing the position of each of the users to known body positions associated with engaging in private conversations; Determine if the user is engaging in a private conversation; as well as Noise associated with the private conversation is filtered from the audio signal.

8. The method of claim 7, wherein the filtering further comprises: determining a direction of the user from the radar data; as well as A microphone is determined for receiving an audio signal in the direction of the user.

9. The method of claim 8, wherein the filtering comprises removing the noise from the audio signal.

10. The method of claim 8, wherein the filtering comprises muting the microphone.

11. The method of claim 7, wherein the radar data comprises a plurality of signals associated with the user, wherein the plurality of signals provide an image of an approximate body orientation of the user.

12. The method of claim 7, wherein said collecting comprises: Periodically tracking increments of the radar data of the user; as well as The comparison is initiated by determining that the increment of the radar data of the user is below a threshold.

13. A non-transitory computer-readable storage medium encoded with instructions executable by a processor for filtering noise associated with a private conversation in a conference room, the non-transitory computer-readable storage medium comprising: instructions for determining when movement of users in the room is below a threshold; instructions for collecting radar data of the users to determine a location of each of the users in the room in response to movement of the users being below a threshold; instructions for determining that the user is engaging in a private conversation based on a matching of the user's orientation to known body orientations associated with engaging in private conversations; as well as Instructions for filtering noise associated with the private conversation from the audio signal.

14. The non-transitory computer-readable storage medium of claim 13, further comprising: Instructions for updating the known body position associated with conducting the private conversation over time based on the collected radar data.

15. The non-transitory computer-readable storage medium of claim 14, wherein the known body location associated with conducting a private conversation is stored in a remotely located server.

Images