Apparatus, method and computer program for analyzing an audio environment

Abstract

The present disclosure example relates to a device that includes components for: querying one or more acoustic reporters in an audio environment using a radio frequency beam having a wavelength below about 10 millimeters; analyzing one or more sound signals reported by the one or more acoustic reporters to determine the location of one or more sound sources providing the one or more sound signals; and using the location of the one or more sound sources to determine one or more sound propagation paths within the audio environment.

Description

Technical Field

[0001] The present disclosure examples relate to apparatuses, methods, and computer programs for analyzing an audio environment. Some relate to apparatuses, methods, and computer programs for analyzing an audio environment to determine one or more sound propagation paths within the audio environment. Background Art

[0002] In many cases, it is useful to have information about the audio environment. For example, if a user wants to eliminate noise, or if the detection of a particular sound in the audio environment is important. Summary of the Invention

[0003] According to various but not necessarily all examples of the present disclosure, there is provided an apparatus including components for: querying one or more acoustic reporters in an audio environment using a radio frequency beam having a wavelength below about 10 millimeters; analyzing one or more sound signals reported by the one or more acoustic reporters to determine the location of one or more sound sources providing the one or more sound signals; and using the location of the one or more sound sources to determine one or more sound propagation paths within the audio environment.

[0004] The components can be used to enable scanning of the radio frequency beam to determine the location of the one or more acoustic reporters.

[0005] The location of the one or more acoustic reporters can be used together with the location of the one or more sound sources to determine one or more sound propagation paths within the audio environment.

[0006] One or more sound propagation paths can include at least one sound propagation path from a sound source to a user within the audio environment.

[0007] At least one sound propagation path can include an indirect sound propagation path.

[0008] The components can be used to estimate one or more anti-sound signals to be provided to a user within the audio environment using the one or more sound propagation paths and information about the one or more reported sound signals.

[0009] The components can be used to enhance one or more sound signals to be provided to the user using the one or more sound propagation paths.

[0010] One or more sound propagation paths can be determined by identifying how the same sound signal is reported by different acoustic reporters within the audio environment.

[0011] The components can be used to configure the apparatus to monitor sounds at the determined locations using the determined locations of the one or more sound sources.

[0012] The component can be used to detect one or more other devices in an audio environment and share information with one or more other devices, where the shared information relates to one or more of the following: the location of one or more sound sources, the location of one or more acoustic reporters, and one or more sound propagation paths in the audio environment.

[0013] The component can be used to determine which one or more of the devices in the audio environment should query one or more of the acoustic reporters in the audio environment.

[0014] The device that should query one or more acoustic reporters in the audio environment can be determined based on one or more of the following: the location of the device relative to one or more acoustic reporters, the activity level of the device, and reducing the number of devices used to query one or more acoustic reporters.

[0015] The component can be used to prioritize queries to one or more acoustic reporters based on the frequency of the sound signals reported by one or more acoustic reporters, such that high-frequency signals have a higher priority than low-frequency signals.

[0016] The radio frequency beam used to query the acoustic reporter can include one or more data packets.

[0017] The component can be used to provide information to a user regarding the location of one or more acoustic reporters.

[0018] The radio frequency beam used to query the acoustic reporter can include a 5G signal.

[0019] According to various but not necessarily all examples of the present disclosure, a method can be provided, including: querying one or more acoustic reporters in an audio environment using a radio frequency beam having a wavelength below about 10 millimeters; analyzing one or more sound signals reported by one or more acoustic reporters to determine the location of one or more sound sources providing the one or more sound signals; and using the location of the one or more sound sources to determine one or more sound propagation paths in the audio environment.

[0020] According to various but not necessarily all examples of the present disclosure, a computer program including computer program instructions is provided, which when executed by a processing circuitry causes: querying one or more acoustic reporters in an audio environment using a radio frequency beam having a wavelength below about 10 millimeters; analyzing one or more sound signals reported by one or more acoustic reporters to determine the location of one or more sound sources providing the one or more sound signals; and using the location of the one or more sound sources to determine one or more sound propagation paths in the audio environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Some examples will now be described with reference to the accompanying drawings, in which:

[0022] Figure 1 An example device is shown;

[0023] Figure 2 An example electronic device is shown;

[0024] Figure 3 An example audio environment is shown;

[0025] Figure 4 An example method is shown;

[0026] Figure 5 Another example method is shown; and

[0027] Figure 6 Another example audio environment is shown. Detailed Description

[0028] Examples of the present disclosure relate to a device 101, a method, and a computer program 109 that may be used to analyze an audio environment 203. In some examples, the analysis of the audio environment 203 may be used to provide noise cancellation for a user within the audio environment 203. In some examples, the analysis of the audio environment 203 may be used to monitor events occurring within the audio environment 203 or for any other suitable purpose.

[0029] Figure 1 Schematically illustrates a device 101 according to an example of the present disclosure. Figure 1 The device 101 illustrated in may be a chip or a chipset. In some examples, the device 101 may be provided within an electronic device, such as a mobile phone, a smart speaker, or any other suitable device configured to implement wireless communication.

[0030] In Figure 1 the example of, the device 101 includes a controller 103. In Figure 1 the example of, the controller 103 may be implemented as controller circuitry. In some examples, the controller 103 may be implemented solely in hardware, have certain aspects in software including only firmware, or may be a combination of hardware and software (including firmware).

[0031] As Figure 1 illustrated in, the controller 103 may be implemented using instructions that implement hardware functionality, for example, by using executable instructions of a computer program 109 in a general-purpose or a special-purpose processor 105, and the executable instructions may be stored on a computer-readable storage medium (disk, memory, etc.) for execution by such a processor 105.

[0032] The processor 105 is configured to read from and write to the memory 107. The processor 105 may also include an output interface and an input interface, and data and / or commands are output from the processor 105 via the output interface, and data and / or commands are input into the processor 105 via the input interface.

[0033] The memory 107 is configured to store a computer program 109 including computer program instructions (computer program code 111), and when loaded into the processor 105, the computer program instructions control the operation of the device 101. The computer program instructions of the computer program 109 provide the logic and routines that enable the device 101 to execute Figure 4 and Figure 5 the methods illustrated in. The processor 105 can load and execute the computer program 109 by reading the memory 107.

[0034] Accordingly, the device 101 includes: at least one processor 105; and at least one memory 107 including computer program code 111, the at least one memory 107 and the computer program code 111 being configured to, together with the at least one processor 105, cause the device 101 to at least perform: querying 401 an audio environment 203 for one or more acoustic reporters 303 using a radio frequency beam having a wavelength below approximately 10 mm; analyzing 403 one or more sound signals reported by the one or more acoustic reporters 303 to determine the location of one or more sound sources 305 providing the one or more sound signals; and using 405 the location of the one or more sound sources to determine one or more sound propagation paths within the audio environment 203.

[0035] As Figure 1 illustrated in, the computer program 109 may reach the device 101 via any suitable delivery mechanism 113. For example, the delivery mechanism 113 may be a machine-readable medium, a computer-readable medium, a non-transitory computer-readable storage medium, a computer program product, a storage device, a recording medium such as a compact disc read-only memory (CD-ROM) or a digital versatile disc (DVD) or a solid-state memory, an article of manufacture including or tangibly embodying the computer program 109. The delivery mechanism may be a signal configured to reliably convey the computer program 109. The device 101 may propagate or transmit the computer program 109 as a computer data signal. In some examples, using a wireless protocol such as Bluetooth, Bluetooth Low Energy, Bluetooth Smart, 6LoWPan (IPv6 over Low-Power Personal Area Network), ZigBee, ANT+, Near Field Communication (NFC), Radio Frequency Identification, Wireless Local Area Network (Wireless LAN) or any other suitable protocol, the computer program 109 may be sent to the device 101.

[0036] Computer program 109 includes computer program instructions for causing device 101 to perform at least the following operations: using radio frequency beam 401 having a wavelength below approximately 10 mm to query one or more acoustic reporters 303 in audio environment 203; analyzing 403 one or more sound signals reported by one or more acoustic reporters 303 to determine the location of one or more sound sources 305 providing the one or more sound signals; and using 405 the location of the one or more sound sources to determine one or more sound propagation paths within the audio environment.

[0037] The computer program instructions may be included in computer program 109, a non-transitory computer-readable medium, a computer program product, a machine-readable medium. In some but not necessarily all examples, the computer program instructions may be distributed over more than one computer program 109.

[0038] Although memory 107 is shown as a single component / circuitry, it may be implemented as one or more separate components / circuitry, some or all of which may be integrated / removable and / or may provide permanent / semi-permanent / dynamic / cache storage.

[0039] Although processor 105 is shown as a single component / circuitry, it may be implemented as one or more separate components / circuitry, some or all of which may be integrated / removable. Processor 105 may be a single-core or multi-core processor.

[0040] References to “computer-readable storage medium,” “computer program product,” “tangibly embodied computer program,” etc. or “controller,” “computer,” “processor,” etc. should be understood to include not only computers having different architectures, such as single / multi-processor architectures and serial (von Neumann) / parallel architectures, but also dedicated circuits, such as field programmable gate arrays (FPGA), application specific circuitry (ASIC), signal processing devices, and other processing circuitry. References to computer programs, instructions, code, etc. should be understood to include software for programmable processors or firmware, such as, for example, programmable content for hardware devices, whether instructions for a processor or configuration settings for fixed function devices, gate arrays, or programmable logic devices, etc.

[0041] In this application, the term “circuitry” may refer to one or more or all of the following:

[0042] (a) pure hardware circuitry implementation (such as implemented only in analog and / or digital circuitry) and

[0043] (b) a combination of hardware circuitry and software, such as (where applicable):

[0044] (i) A combination of (one or more) analog and / or digital hardware circuits and software / firmware, and

[0045] (ii) Any portion of (one or more) hardware processors (including digital signal processors) with software, the software, and (one or more) memories, which work together to enable a device such as a mobile phone or a server to perform various functions, and

[0046] (c) (One or more) hardware circuits and / or (one or more) processors, such as (one or more) microprocessors or a portion of (one or more) microprocessors, which require software (e.g., firmware) to operate, but the software may be absent when not needed to operate.

[0047] This definition of circuitry applies to the term in this application, including all uses in any claim. As another example, as used in this application, the term circuitry also encompasses implementations of only hardware circuits or processors and their accompanying software and / or firmware. For example and if applicable to a particular claim element, the term circuitry also encompasses a baseband integrated circuit system for a mobile device, or a similar integrated circuit system in a server, cellular network device, or other computing or network device.

[0048] Figure 4 and Figure 5 The boxes illustrated in and may represent steps in a method and / or segments of code in a computer program 109. The illustration of a particular order of the boxes does not necessarily imply a required or preferred order for the boxes, but the order and arrangement of the boxes may vary. Additionally, it is possible to omit some boxes.

[0049] Figure 2 An example electronic device 201 that can be used to implement examples of the present disclosure is shown. The electronic device 201 can be a mobile phone, a smart speaker, or any other suitable device that can be configured to implement wireless communication.

[0050] The electronic device 201 includes a device 101, a user interface 209, an audio output 207, and a transceiver 205. Figure 2 Only the components of the electronic device 201 that are mentioned in the following description are shown. It should be understood that in an implementation of the present disclosure, the electronic device 201 may include Figure 2 additional components not shown in. For example, the electronic device 201 may include a power supply and other suitable components.

[0051] The device 101 can be a controller 103 that includes a processor 105 and a memory 107, as Figure 1As shown. Device 101 may be configured to implement control of electronic device 201. For example, device 101 may be configured to control the radio frequency beam transmitted by transceiver 205.

[0052] In some examples, device 101 may be configured to implement signal processing. In such an example, the signal received by transceiver 205 may be provided to device 101 to enable processing of the received signal. Device 101 may be configured to process the signal to analyze the audio environment 203 in which electronic device 201 is located. In some examples, device 101 may be configured to enable electronic device 201 to implement methods such as the methods shown in Figure 4 and Figure 5 as shown.

[0053] User interface 209 may include any component that enables a user to control electronic device 201. User interface 209 may include a user output device and a user input device.

[0054] The user output device may include any component that may be configured to provide output to the user of electronic device 201. In some examples, the user output device may include a display or other output component that may enable visual output to be provided to the user.

[0055] The user input device may include any component that may be configured to enable the user of electronic device 201 to provide input to electronic device 201. In some examples, electronic device 201 may include a touch-sensitive display or any other suitable user input component. The user input device may be configured to enable the user to select menu options or perform any other suitable type of user input. This may enable the user to control one or more functions of electronic device 201.

[0056] User interface 209 is configured to receive input from device 101 and provide output to device 101. This may enable controller 103 of device 101 to control the output provided by the user output device. This may also enable user input from the user input device to be used to control the functions performed by controller 103.

[0057] The audio output device 207 may include any component that may be configured to provide an audio output. The audio output device 207 is configured to convert an electrical input signal into an output sound signal. The audio output device 207 may include one or more speakers, headphones, headsets, or any other suitable type of device.

[0058] In Figure 2In the example, the audio output device 207 is part of the electronic device 201. In other examples, one or more audio output devices 207 may be provided as a peripheral device for the electronic device 201. For example, a speaker can be connected to an electronic device 201 such as a mobile phone.

[0059] The transceiver 205 may be configured to implement wireless communication. The transceiver 205 may include any component that can be configured to enable the electronic device 201 to transmit and receive radio frequency signals.

[0060] The transceiver 205 may be configured to implement wireless communication using millimeter waves. The transceiver 205 may be configured to implement wireless communication using a wavelength less than approximately 10 mm. A wavelength less than approximately 10 mm may be considered a short wavelength. The transceiver 205 may be configured to enable wireless communication using high frequencies. High frequencies may be above 24 GHz. In some examples, the frequency may be between 24 and 39 GHz.

[0061] The transceiver 205 may be configured to implement 5G communication. The transceiver 205 may be configured to implement communication within a new radio network. New radio is the 3GPP (Third Generation Partnership Project) name for 5G technology.

[0062] The transceiver 205 may be configured to implement beamforming so that one or more radio frequency beams can be provided by the transceiver 205. The transceiver 205 may include multiple antenna elements to enable the provision of radio frequency beams.

[0063] The transceiver 205 may be configured to be able to scan radio frequency beams. That is, the transceiver 205 may be configured to enable radio frequency beams to be directed at multiple different angles. In some examples, radio frequency beams can be directed at multiple different angles simultaneously. For example, digital beamforming can enable multiple different radio frequency beams to be provided in different directions simultaneously. In other examples, the radio frequency beams may point in different directions at different times.

[0064] In Figure 2 In the example shown, multiple electronic devices 201 are provided within the audio environment 203. The multiple electronic devices 201 may be configured to communicate wirelessly with each other. The multiple electronic devices 201 may be configured to communicate with each other using a 5G network or any other suitable means.

[0065] In Figure 2 In the example shown, the similarity of each electronic device 201 is that they each include the device 101, the user interface 209, the audio output 207, and the transceiver 205. In other examples, different types of electronic devices 201 may be provided that include different components.

[0066] Multiple electronic devices 201 are all provided within the same audio environment 203. The audio environment can include any environment that includes one or more sound signals. The audio environment 203 can include a room, a part of a room, or any other suitable environment.

[0067] The sound signals within the audio environment 203 can be provided by one or more sound sources. In some examples, sound sources can be provided within the audio environment 203. In other examples, sound sources can be provided outside the audio environment 203, but the sound signals can be enabled to propagate into the audio environment 203. For example, if the audio environment includes a room, then sound signals caused by traffic or pedestrians on the street can propagate into the room and thus be audible within the audio environment 203.

[0068] The electronic device 201 can be configured such that the apparatus 101 within the electronic device 201 can analyze the audio environment 203 using the signals detected by the transceiver 205.

[0069] Figure 3 An example audio environment 203 is schematically shown. The audio environment 203 includes the electronic device 201 used by the user 301, multiple acoustic reporters 303, and multiple sound sources 305. Figure 3 The entities in are not drawn to scale.

[0070] The electronic device 201 can be as above Figure 2 as shown. In Figure 3 the example shown, the audio environment 203 includes only one electronic device 201. It can be understood that the audio environment 203 can include multiple electronic devices 201. For example, everyone within the audio environment 203 has their own mobile phone and additional electronic devices 201, such as smart speakers, smart watches, or laptops.

[0071] The electronic device 201 is configured to provide a radio frequency beam 307. The radio frequency beam includes a directional transmission from the antenna elements of the transceiver 205. The electronic device 201 can be configured to control the direction of the radio frequency beam 307.

[0072] In Figure 3 the example shown, the audio environment 203 includes two sound sources 305. It should be understood that any number of sound sources 203 can be provided in other examples of the present disclosure. Also in Figure 3 the example of, the sound sources 305 are located within the audio environment 203. In other examples, the sound sources 305 can be outside the audio environment and the sound signals created by the sound sources 305 can propagate into the audio environment 203. For example, outdoor noise such as traffic can propagate into a building through a window.

[0073] The sound source 305 may include any entity that generates a sound signal. In Figure 3 the example, the sound source 305 may include a person who is speaking or making other noises. The sound of a person speaking may have a higher frequency compared to background noise such as traffic. In other examples of the present disclosure, there may be other types of sound sources 305.

[0074] The acoustic reporter 303 may include any entity that provides a detectable response when a sound signal impinges on them. The acoustic reporter 303 may include a surface that reflects or scatters an incident sound signal and is also susceptible to the reflection of the radio frequency beam provided by the electronic device 201. The acoustic reporter 303 may include furniture items, metal fixtures on walls, curtain weights, the electronic device 201, or anything else that can be configured to report a sound signal.

[0075] When a sound signal impinges on them, the acoustic reporters 303 may vibrate or oscillate. This response can be detected by the radio frequency beam 307 provided by the transceiver 205 of the electronic device 201. This enables the sound signal to be reported by the acoustic reporter 303.

[0076] In an example of the present disclosure, the radio frequency beam 307 provided by the transceiver 205 may include a high-frequency beam having a wavelength of about 10 mm or shorter. For example, the radio frequency beam 307 provided by the transceiver 205 may be a 5G beam. The short wavelength of the radio frequency beam 307 provided by the transceiver 205 enables the detection of the high-frequency oscillations of the acoustic reporter 303, and thus information about a high-frequency sound source 305 such as a person speaking can be provided. This information may include the energy level of the corresponding frequency components of the sound source 305. In some examples, this may enable high-frequency sound sources to be classified as a particular type of sound source 305. For example, a person speaking has a known and recognizable spectrum within the audible frequency range of the human ear of about 8 Hz to 20 kHz. Voice components falling within a predetermined frequency range that can assist in the speech intelligibility for the user 301 can also be identified, such as approximately: 500 Hz, 1 kHz, 2 kHz, and 4 kHz. Any other suitable frequency or frequency range may be set to identify other types of sound sources 305.

[0077] In Figure 3 the example shown, the acoustic reporter 303 is provided within the line of sight of the electronic device 201. This enables the radio frequency beam 307 to be directly transmitted towards the acoustic reporter 303. In other examples, the acoustic reporter 303 does not need to be located within the line of sight of the electronic device 201. This can provide a more complex propagation path for the radio frequency beam 307 between the acoustic reporter 303 and the electronic device 201.

[0078] In Figure 3In the example shown, an object within the audio environment 203 is shown as a sound source 305 or an acoustic reporter 303. It should be understood that in some examples, the object can be both a sound source 305 and an acoustic reporter 303. Additionally, in addition to being able to monitor the audio environment 203, the electronic device 201 can also be a sound source 305.

[0079] Figure 4 An example method that can be implemented using the apparatus 101 and the electronic device 201 as shown above is shown. The electronic device 201 including the apparatus 101 can be located in an audio environment as Figure 3 shown and described above.

[0080] The method includes, at block 401, querying one or more acoustic reporters 303 in the audio environment 203 using a radio frequency beam 307 having a wavelength of less than approximately 10 mm.

[0081] The radio frequency beam 307 can be provided by an antenna element of the transceiver 205 of the electronic device 201. The electronic device 201 can be configured to scan the radio frequency beam 307 around the audio environment 203. The electronic device 201 can be configured to direct the radio frequency beam 307 at different angles so as to be able to query different acoustic reporters 303 at different locations.

[0082] The acoustic reporter 303 causes reflection or scattering of the sound signal provided by the sound source 305. The radio frequency beam 307 directed at the acoustic reporter 303 is reflected by the acoustic reporter 303. Due to the short wavelength of the radio frequency beam 307, any sound detected by the acoustic reporter 303 will affect the reflected radio frequency beam and enable the sound signal 305 to be reported to the electronic device 201 by the radio frequency beam 307 reflected back to the electronic device 201. This can enable the radio frequency beam 307 to detect high-frequency audio signals reported by the acoustic reporter 303. The high-frequency audio signal can be a person speaking or any other suitable high-frequency signal.

[0083] The method includes, at block 403, analyzing one or more sound signals reported by one or more acoustic reporters 303 to determine the location of one or more sound sources 305 providing the one or more sound signals.

[0084] The transceiver 205 of the electronic device 201 may be configured to receive reflected radio frequency beams and provide these reflected radio frequency beams to the device 101 to enable signal processing. In some examples, the device 101 within the electronic device 201 that receives the reflected radio frequency beams may be used to perform signal processing. In other examples, the electronic device 201 may receive the reflected radio frequency beams and then provide information related to the received reflected radio frequency beams to another electronic device 201, which is configured to perform signal processing. For example, multiple electronic devices 201 may be configured to provide radio frequency beams 307 for querying acoustic reporters, and a central electronic device 201 may be configured to collect and analyze the responses. Such a system may have radio frequency beams 307 that are reflected to the central electronic device, or radio frequency beams that are reflected back to the original electronic device and analyzed before being transmitted to the central electronic device 201 for analysis, or a combination of such features.

[0085] The signals received by the electronic device 201 can be processed to enable determination of the location of one or more sound sources 305. The location may be the location and orientation of the sound source 305 within the audio environment. For example, multiple reflected radio frequency beams from multiple different acoustic reporters 303 may be detected. If the locations of the acoustic reporters 303 are known, the reflected radio frequency beams can be processed to identify the sound signals detected by the acoustic reporters 303. The sound signals can be identified by comparing the energy levels of different frequency bands of the sound signals. This gives an indication of the sound scene at the location of the acoustic reporter 303. The sound scenes available at different locations of different acoustic reporters 303 can be analyzed to determine the location of one or more sound sources 305 relative to the acoustic reporters 303.

[0086] Any suitable process can be used to determine the location of the acoustic reporter 303. For example, the direction of the radio frequency beam 307 used to query the acoustic reporter 303 gives an indication of the angular position of the acoustic reporter 303 relative to the electronic device 201, and the time taken to detect the reflected radio frequency beam 307 gives an indication of the distance between the electronic device 201 and the acoustic reporter 303. In some examples, scanning of the radio frequency beam 307 can be used to determine the location of the acoustic reporter 303 before it is queried. In some examples, scanning of the radio frequency beam 307 can be performed at different time points to enable identification of changes in the location of one or more acoustic reporters 303.

[0087] In other examples, other means can be used to determine the location of the acoustic reporter 303. For example, an acoustic reporter 303 such as a wall mount may be in a fixed position. Information related to this position may be accessible to the electronic device 201.

[0088] At block 405, the method includes using the locations of one or more sound sources 305 to determine one or more sound propagation paths within the audio environment 203. In some examples, the locations of one or more acoustic reporters 303 are used in conjunction with the determined locations of the sound sources 305 to determine one or more sound propagation paths within the audio environment 203.

[0089] A sound propagation path can be determined by identifying how different acoustic reporters 303 within the audio environment 203 report the same sound signal. The same sound signal can be identified by correlating the sound signals reported by different acoustic reporters 303. Differences in the energy levels and direction aspects of the sound signals reported by different acoustic reporters 303 can be used to identify the same sound signal. The differences in the ways different acoustic reporters 303 report the same sound signal give information about the location of the sound source relative to the acoustic reporters 303. This information and the locations of the acoustic reporters can be used to estimate the sound propagation path.

[0090] A sound propagation path defines the trajectory taken by a sound signal between a sound source 305 and an electronic device 201. In some examples, the sound propagation path includes at least one sound propagation path from the sound source 305 to a user within the audio environment 201. The user can be a user of the electronic device 201 or another user within the audio environment. The sound propagation path can be a direct path or an indirect path. An indirect sound propagation path can include reflections and scatterings from entities such as acoustic reporters 303 within the audio environment 203. Since the radio frequency beam 307 used to query the acoustic reporters 303 has a short wavelength, this allows for the monitoring of complex sound propagation paths.

[0091] In some examples, an indirect sound propagation path can include sound passing through an object such as a wall. For example, a sound source 305 can be provided in a different room from the user 301 or the electronic device 201, but the sound signal can be large enough such that the sound can pass through the wall and still be heard by the user 301.

[0092] Sound propagation paths can be used for any suitable purpose. In some examples, a sound propagation path and information about the reported sound signal can be used to estimate one or more anti-sound signals. The anti-sound signals can be used to provide noise cancellation within the audio environment 203. The anti-sound signals can be provided to a user of the electronic device 201. For example, they can be provided to a user of the electronic device 201 to cancel background noise for the user 301.

[0093] The anti-sound signals can be directional signals that take into account the directionality of the sound signal. This can enable different speakers within a surround sound system to provide different anti-sound signals, or different earpieces within a binaural sound system to provide different anti-sound signals.

[0094] Since the radio frequency beam 307 used to query the acoustic reporter 303 has a wavelength of less than about 10 mm, this enables the detection of high-frequency sounds. Therefore, when creating an anti-sound signal, high-frequency sounds can be taken into account. This can enable the provision of an anti-sound signal that cancels high-frequency sounds such as human speech. Thus, examples of the present disclosure can provide a noise cancellation system that can eliminate high-frequency noise such as human speech.

[0095] In some examples, an anti-sound signal can be provided to the user 301 of the electronic device 201. It can be assumed that the user 301 is located in a position close to the electronic device 201. In other examples, an anti-sound signal can be provided to a different user. For example, an anti-sound signal can be provided so that another person such as a sleeping baby or toddler is not disturbed by the sound signal. In such an example, the position of the other user can be determined by the electronic device 201 or by any other suitable means.

[0096] In some examples, the sound propagation path can be used to enhance one or more sound signals to be provided to the user. For example, the user may wish to receive an alert for an event such as a doorbell ringing, a pot starting to boil, or other suitable events that create sound. In this case, the acoustic reporter 303 will report a sound signal when the event occurs so that the query of the acoustic reporter 303 can be used to detect when the event occurs. In such an example, an anti-sound signal can then be provided to reduce background noise but increase the relative volume of the detected sound signal so that the user 301 is aware of the event that caused the sound signal.

[0097] In some examples, the electronic device 201 may be configured to enable information about the acoustic environment 203 to be provided to a user of the electronic device 201. For example, a map or an image may be displayed on a display indicating the location of one or more sound sources 305 and / or one or more acoustic reporters 303. The user 301 may use this information to adjust the audio environment 203 or for any other suitable purpose. For example, if the user 301 is warned that one or more acoustic reporters 303 are causing reflections of sound signals and creating unwanted noise for the user 301, the user 301 may reduce this effect by moving the acoustic reporters 303 or by covering them. This may help the user create a safer audio environment 203. For example, it may help the user 301 move or modify the acoustic reporters 303 to ensure that their conversation cannot be heard by others in or near the audio environment 203. In other examples, it may enable the user to create an environment 203 in which they can hear the conversation of a person located in a position where it may be difficult to hear the conversation. For example, if a group of friends are sitting at several tables in a café, a bar, or other noisy environment, it may be difficult to hear a person's speech if they are not sitting together. Examples of the present disclosure may be configured to enhance the sound signal corresponding to the conversation and enable the users to hear each other.

[0098] In some examples, the electronic device 201 may be configured to detect one or more other electronic devices 201 within the audio environment 203. The electronic device 201 may include means 101 capable of implementing these disclosed methods. The electronic device 201 may be configured to communicate with one or more other detected electronic devices 201. This may enable information to be shared between different means 101 in different electronic devices 201. The shared information may include information related to: the location of one or more sound sources 305, the location of one or more acoustic reporters 303, one or more sound propagation paths within the audio environment 203, or any other suitable information.

[0099] In some examples, the shared information may include a map of the audio environment 203 created by one means 101 and shared with another means 101. The map may include an indication of the location of the electronic devices 201 and acoustic reporters within the audio environment 203 303. In other examples, the shared information may include information that may be used by one or more devices 101 to create a map of the audio environment 203. For example, a first means 101 may determine the location of a subset of the location sound sources 305 and may provide this information to another means to enable the other means to create a map of the audio environment 203.

[0100] In an example where multiple electronic devices 201 are provided within an audio environment, the apparatus 101 of a first electronic device 201 may be configured to determine which available electronic device 201 should be used to query an acoustic reporter 303 within the audio environment 203. Any suitable criteria may be used to determine which electronic device 201 to use to query each acoustic reporter 303.

[0101] In some examples, the electronic device 201 closest to the acoustic reporter 303 may be used to query the acoustic reporter 303.

[0102] In some examples, the electronic device 201 that should be used to query the acoustic reporter 303 may be determined based on the activity level of the apparatus 101 within the electronic device 201. For example, apparatuses 101 with the lowest activity levels may be used to query because they have more resources available. The available resources may be power, processing capacity, signal bandwidth, or any other suitable resource bandwidth.

[0103] In some examples, the electronic device 201 that should be used to query the acoustic reporter 303 may be determined so as to reduce the number of apparatuses 101 used to query the acoustic reporter 303. This may help reduce the bandwidth of the electronic device(s) 201 used to query the acoustic reporter.

[0104] In some examples, queries to one or more acoustic reporters 303 may be prioritized to reduce the bandwidth used for the queries or to optimize any other suitable parameter. For example, in some examples, sound signals with higher frequencies may be prioritized over sound signals with lower frequencies.

[0105] In some examples, the radio frequency beam 307 used to query the acoustic reporter may include one or more data packets. This may enable the information that needs to be transmitted to be included within one or more data packets. When signals including the data packets are transmitted within the audio environment, they are modulated by the acoustic reporter 303 upon which they are incident. This modulation provides information about the location of the acoustic reporter 303 within the audio environment 203. This enables the signals to be used for querying the acoustic reporter 303 and also for transmitting information, and thus may reduce the bandwidth and energy requirements of the electronic device 201.

[0106] In some examples, the method can be used to provide an alert 301 to a user. For example, the radio frequency beam 307 and the acoustic reporter 303 can be configured to detect noises such as a window being broken or a door being opened. When such noises are detected, an alert can be provided to the user of the electronic device 201 instead of providing an anti-sound signal. Similarly, the electronic device 201 can be configured to monitor the movement of a vulnerable person such as an elderly person. In such an example, the electronic device can be configured to detect the sound of a person falling. In some examples, the electronic device 201 can be configured to provide an alert if no sound is detected within a predetermined time, for example, if a vulnerable person moving around the audio environment 203 does not make a sound. In such an example, the electronic device 201 can be configured to provide an alert to the user of a different electronic device 201, for example, an alert can be provided to a caregiver or family member who does not need to be within the audio environment 203.

[0107] Figure 5 Another example method that can be implemented using examples of the present disclosure is shown. The method can be implemented using the apparatus 101 and the electronic device 201 as shown in Figure 1 and Figure 2 In this example, the method is used to provide an anti-sound signal to a user for noise reduction purposes. It should be understood that the method can be applicable to other purposes.

[0108] In this example, the method is used to provide an anti-sound signal to a user for noise reduction purposes. It should be understood that the method can be applicable to other purposes.

[0109] The method includes, at block 501, performing a scan to determine the location of one or more acoustic reporters 303 within the audio environment 203. The scan can be performed by directing the radio frequency beam 307 of the electronic device 201 in different directions. Reflections from different directions can provide an indication of the position of the acoustic reporter 303 relative to the (one or more) electronic devices 201.

[0110] The scan of the radio frequency beam 307 can be performed by multiple electronic devices 201. The multiple electronic devices 201 can perform the scan simultaneously. The multiple electronic devices 201 can be configured such that different electronic devices 201 scan different regions of the audio environment 203.

[0111] At block 503, record the location of any new acoustic reporter 303 detected in the scan. The location of the new acoustic reporter 303 can be recorded in a map or database, or by any other suitable means. The recorded location of the acoustic reporter 303 provides a virtual microphone array, where the acoustic reporter 303 acts as a virtual microphone by responding to an incident sound signal.

[0112] At block 503, any changes in the position of the other acoustic reporter 503 can also be recorded. For example, if the acoustic reporter 303 has moved, the new position can be recorded. In some examples, other changes in the state of the acoustic reporter 303 can also be recorded. For example, the user can open or close a curtain that may cover or uncover the acoustic reporter 303 without changing the position of the acoustic reporter 303 itself.

[0113] Blocks 501 and 503 can be repeated as needed. In some examples, blocks 501 and 503 can be repeated periodically so that the scanning of the acoustic reporter 303 is performed at regular intervals. This can enable the detection of any changes in the acoustic reporter 303. For example, it can detect whether the acoustic reporter 303 has moved, or whether an object such as a person or a door has moved relative to the acoustic reporter 303, or whether a new acoustic reporter 303 has been added to the audio environment, or any other suitable change.

[0114] In some examples, the scanning of the acoustic reporter 303 can be performed in response to one or more trigger events. The trigger event can be a user input via the user interface 209, a detected change in the audio environment 203, or any other suitable trigger event.

[0115] At block 505, the acoustic reporter 303 is queried so that one or more acoustic reporters 303 can report sound signals. The acoustic reporter 303 is queried by directing a radio frequency beam 307 towards one or more acoustic reporters 303. The radio frequency beam can be a high-frequency beam with a wavelength below about 10 mm. The reflected high-frequency beam includes information about the sound signals detected by the acoustic reporter 303.

[0116] Queries of the acoustic reporter 303 can be performed at multiple times. The frequency at which the queries occur can be determined by several factors, such as the noise of the audio environment 203, the number of electronic devices 201 available to perform the queries, or any other suitable factors. In some examples, a user 301 of the electronic device can provide an input via the user interface 209 to indicate the settings under which they desire the electronic device 201 to operate. This setting allows the user 301 to indicate whether the audio environment 203 is a high-noise environment or a low-noise environment. In a high-noise environment, the queries can occur more frequently. In some examples, queries of the acoustic reporter can be used to determine the noise of the audio environment 203 and the proportion of high-frequency and low-frequency noise within the audio environment 203. This can be used to determine the frequency at which the queries should be performed. In some examples, based on the queries of the acoustic reporter 303, one or more regions of the audio environment 203 can be determined to be noisier and / or have a greater proportion of high-frequency noise than one or more other regions of the audio environment 203. For example, it can be determined that a first region is noisier than a second different region based on a threshold level, such as setting a noise level, or a percentage threshold, such as a noise level 20% higher than the second region. Then, queries can be performed more frequently for one or more regions of the audio environment 203 that are determined to be noisier than one or more other regions of the audio environment 203.

[0117] At block 507, analyze the sound signals detected by the acoustic reporter 303 and reported back to the device 201 in a manner reflected by the radio frequency beam 307. This analysis can use the sound signals reported by multiple different acoustic reporters 303 to determine the location of one or more sound sources 305 inside or outside the audio environment 203. By comparing how different acoustic reporters 303 report the same sound signal, the location of one or more sound sources 305 can be determined.

[0118] In some examples, the location of one or more sound sources 305 can be used to create a map of the audio environment 203. The map can include information indicating the relative positions of the sound sources 305, the acoustic reporters 303, and the electronic device 201. In some examples, the map can also include additional information, such as the location of one or more users who may be at a different location than the location of the electronic device 201.

[0119] At block 509, the acoustic reporter 303 reports the sound signal to one or more electronic devices 201. The map created at block 507 can be used to determine the location of the sound source 305 that created the sound signal. In some examples where the sound source 305 is outside the audio environment, the location where the sound signal enters the audio environment 203 can be determined. For example, traffic noise or other outdoor noise may enter the audio environment mainly through a window or other opening.

[0120] At block 511, the sound signal reported to the electronic device 201 and the locations of one or more acoustic reporters 303 are used to determine the sound propagation path to the user 301. In this example, the user 301 is the user of the electronic device 201. In other examples, the user 301 can be a different user located within the audio environment 203.

[0121] At block 513, the sound reported by the acoustic reporter 303 and the sound propagation path are used to calculate an anti-sound, which can be used to provide noise cancellation for the user within the audio environment 203. The anti-sound signal can be provided by the audio output 207 of the electronic device 201. In other examples, another audio output device can be used to provide the anti-sound signal. In such examples, the anti-sound signal needs to be adjusted to account for the position of the audio output device relative to the user 301.

[0122] Figure 6 Another example audio environment 203 is shown. In this example, the audio environment 203 includes a café where multiple people are sitting at tables throughout the audio environment 203.

[0123] The user 301 is sitting at a table with their electronic device 201. The electronic device 201 can be a mobile phone or a laptop computer or any other suitable type of electronic device 201. The user 301 wishes to focus on their reading material and thus controls their electronic device 201 to provide an anti-sound signal for active noise cancellation.

[0124] The audio environment 203 includes a speaker that provides a first sound source 305A and an acoustic reporter 303A. The speaker can be configured to provide a sound signal 601A, for example, the speaker can play music. The speaker can also include one or more surfaces that vibrate if a sound signal is incident on those surfaces and thus also act as an acoustic reporter 303.

[0125] The audio environment 203 includes a person speaking loudly, who provides a second sound source 305B and an acoustic reporter 303B. The person speaking loudly provides another sound signal 601B. The act of speaking causes vibrations in the person's throat. Such vibrations can be detected by a radio frequency beam, and thus this person also provides an acoustic reporter 303B.

[0126] The electronic device 201 uses a radio frequency beam 307 with a wavelength below approximately 10 mm to determine the locations of the acoustic reporters 303A, 303B and any sound sources 305 within the audio environment. The acoustic reporters 303A, 303B can then be queried to determine the sound signals provided by the sound sources 305A, 305B and an anti-sound signal can be estimated.

[0127] Sound signals from sound sources 305C that are not located within the audio environment 203 can also propagate within the audio environment 203. For example, the sound source 305C can be a dropped plate, which creates a very loud and sudden sound signal that can propagate through walls or doors or other obstacles. In the example, queries to the acoustic reporter 303 can be used to identify the possible location of the noisy sound source 305. For example, in this example, it can identify the location of the kitchen where the plate might have been dropped and the pans might have made clanging sounds. Once the electronic device 201 knows the possible location of the noise sound source 305, the electronic device can configure the antenna element to direct the radio frequency beam 307 towards that location. This enables the monitoring of the noisy sound source so that they can be taken into account in the anti-sound signal.

[0128] In Figure 6 the example shown, multiple other electronic devices 201 can be provided in other audio environments. For example, different electronic devices 201 can be provided in different rooms within the same building. The electronic device 201 can be configured to provide information related to the audio environment 203 and the location of the sound source within the audio environment. A 5G communication network or any other suitable type of communication network can be used to share this information. The electronic device 201 can use this information to determine a more complex propagation path of the sound signal within the audio environment 203 in which the user is located. For example, it can provide an indication of the sound source 305 that can create a sound signal that propagates through the wall to the audio environment of the user 301.

[0129] Accordingly, the example of the present disclosure provides an apparatus 101, a method, and a computer program 109 capable of analyzing an audio environment. This analysis can be used to provide noise cancellation or for any other suitable purpose.

[0130] Querying the acoustic reporter 303 using the radio frequency beam 307 can enable the system (apparatus 101, method, and computer program 109) to be used to modify high-frequency sound signals such as human conversations. The radio frequency beam 307 propagates much faster in the audio environment 203 than the sound signal, so this can provide an anti-sound signal or other enhancement to the sound signal more quickly. Compared with existing noise cancellation systems that need to analyze the received sound signal, this can provide better adaptation to high-frequency sound signals. Such existing systems cannot adapt to high-frequency sound signals quickly enough and thus cannot provide sufficient anti-sound signals for the high-frequency components.

[0131] Systems, devices, methods, and computer programs can use machine learning, which can include statistical learning. Machine learning is a field of computer science that gives computers the ability to learn without being explicitly programmed. A computer learns from experience E with respect to a class of tasks T and a performance measure P if its performance (measured by P) on tasks in T improves with experience E. A computer can typically learn from previous training data to predict future data. Machine learning includes fully or partially supervised learning and fully or partially unsupervised learning. It can achieve discrete outputs (e.g., classification, clustering) and continuous outputs (e.g., regression). For example, machine learning can be implemented using different methods such as cost function minimization, artificial neural networks, support vector machines, and Bayesian networks. For example, cost function minimization can be used for linear and polynomial regression and K-means clustering. For example, an artificial neural network with one or more hidden layers models the complex relationship between an input vector and an output vector. Support vector machines can be used for supervised learning. A Bayesian network is a directed acyclic graph that represents the conditional independence of multiple random variables.

[0132] The term "comprising" as used in this document has an inclusive rather than an exclusive meaning. That is, any reference to X comprising Y means that X can include only one Y or can include multiple Ys. If an exclusive meaning of "comprising" is intended, then it will be mentioned in the context by referring to "including only one..." or by using "consisting of".

[0133] In this description, various examples are referred to. The description of a feature or function related to an example indicates that the feature or function exists in that example. Whether explicitly stated or not, the use of the terms "example" or "for example" or "may" or "can" in the text means that such a feature or function exists at least in the described example, whether described as an example or not, and they can but do not necessarily appear in some or all other examples. Thus, "example", "for example", "may", or "can" refer to a particular instance within a class of examples. The attributes of an instance can be attributes of only that instance or attributes of a class or attributes of a subclass of the class, where the subclass includes some but not all of the instances in the class. Thus, it is implicitly disclosed that features described with reference to one example rather than another can, where possible, be used as part of a working combination in those other examples, but do not necessarily have to be used in those other examples.

[0134] Although several examples have been described in the previous paragraphs with reference to various examples, it should be understood that the given examples can be modified without departing from the scope of the claims.

[0135] Several features described in the previous description can be used in combinations different from those explicitly described above.

[0136] Although functions have been described with reference to certain features, these functions may be executable by other features whether or not they are described.

[0137] Although features have been described with reference to certain examples, these features may also be present in other examples whether or not they are described.

[0138] The terms "a" or "the" used in this document are inclusive rather than exclusive. That is, any reference to X including a / the Y indicates that X may include only one Y or may include more than one Y, unless the context clearly indicates the contrary. If it is intended to use "a" or "the" with an exclusive meaning, it will be specified explicitly in the context. In some cases, the use of "at least one" or "one or more" may be used to emphasize the inclusive meaning, but the absence of these terms should not be taken as inferring any exclusive meaning.

[0139] The presence of a feature (or combination of features) in a claim is a reference to the feature or (combination of features) itself and also a reference to features (equivalent features) that achieve substantially the same technical effect. For example, equivalent features include features that are variants and that achieve substantially the same result in substantially the same way. For example, equivalent features include features that perform substantially the same function in substantially the same way to achieve substantially the same result.

[0140] In this description, various examples have been referred to that use adjectives or adjective phrases to describe example characteristics. Such a description of a characteristic related to an example indicates that the characteristic is present in some examples exactly as described and is present in other examples substantially as described.

[0141] Although efforts have been made in the foregoing specification to draw attention to those features considered important, it should be understood that the applicant may seek protection via the claims for any patentable feature or combination of features mentioned above and / or shown in the drawings, whether or not such point has been emphasized.

Claims

1. An apparatus for analyzing an audio environment, comprising: at least one processor; and at least one memory including computer program code stored thereon, wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus to at least: query one or more acoustic reporters in the audio environment using a radio frequency beam having a wavelength below 10 millimeters; analyze one or more sound signals reported by the one or more acoustic reporters to determine the location of one or more sound sources providing the one or more sound signals; use the location of the one or more sound sources to determine one or more sound propagation paths within the audio environment; and prioritize querying the one or more acoustic reporters based on the frequency of the sound signals reported by the one or more acoustic reporters, such that higher frequency signals have a higher priority than lower frequency signals.

2. The apparatus according to claim 1, wherein the at least one memory and the computer program code are configured to, with the at least one processor, further cause the apparatus to: perform a scan of the radio frequency beam to determine the location of one or more acoustic reporters.

3. The apparatus according to claim 1, wherein the location of the one or more acoustic reporters is used together with the location of the one or more sound sources to determine the one or more sound propagation paths within the audio environment.

4. The apparatus according to claim 1, wherein the one or more sound propagation paths include at least one sound propagation path from a sound source to a user within the audio environment, and the at least one sound propagation path includes an indirect sound propagation path.

5. The apparatus according to claim 1, wherein the at least one memory and the computer program code are configured to, with the at least one processor, further cause the apparatus to: use the one or more sound propagation paths and information about one or more of the reported sound signals to estimate one or more anti-sound signals to be provided to a user within the audio environment.

6. The apparatus according to claim 1, wherein the at least one memory and the computer program code are configured to, with the at least one processor, further cause the apparatus to: use the one or more sound propagation paths to enhance one or more sound signals to be provided to the user.

7. The apparatus according to claim 1, wherein the one or more sound propagation paths are determined by identifying how the same sound signal is reported by different acoustic reporters within the audio environment.

8. The apparatus according to claim 1, wherein the at least one memory and the computer program code are configured to, with the at least one processor, further cause the apparatus to: use the determined location of the one or more sound sources to configure the apparatus to monitor sounds at the determined locations.

9. The apparatus according to claim 1, wherein the at least one memory and the computer program code are configured to, together with the at least one processor, further cause the apparatus to: Detect one or more other devices within the audio environment and share information with the one or more other devices, where the shared information relates to one or more of the following: the location of one or more sound sources, the location of one or more acoustic reporters, one or more sound propagation paths within the audio environment.

10. The apparatus according to claim 9, wherein the at least one memory and the computer program code are configured to, together with the at least one processor, further cause the apparatus to: Determine which one of the one or more other devices within the audio environment should query one or more of the acoustic reporters within the audio environment.

11. The apparatus according to claim 10, wherein the device that should query one or more acoustic reporters within the audio environment is determined based on one or more of the following: The position of the device relative to one or more acoustic reporters, the activity level of the device, reducing the number of devices for querying one or more acoustic reporters.

12. The apparatus according to claim 1, wherein the radio frequency beam for querying the acoustic reporter comprises a 5G signal.

13. A method for analyzing an audio environment, comprising: Querying one or more acoustic reporters in an audio environment using a radio frequency beam having a wavelength below 10 millimeters; Analyzing one or more sound signals reported by the one or more acoustic reporters to determine the location of one or more sound sources providing the one or more sound signals; Using the location of the one or more sound sources to determine one or more sound propagation paths within the audio environment; And Prioritizing the querying of the one or more acoustic reporters based on the frequency of the sound signals reported by the one or more acoustic reporters, such that high-frequency signals have a higher priority than low-frequency signals.

14. The method according to claim 13, further comprising: Performing a scan of the radio frequency beam to determine the location of one or more acoustic reporters.

15. The method according to claim 13, wherein the location of the one or more acoustic reporters is used together with the location of the one or more sound sources to determine the one or more sound propagation paths within the audio environment.

16. The method according to claim 13, further comprising: Using the one or more sound propagation paths and information about one or more of the reported sound signals to estimate one or more anti-sound signals to be provided to a user within the audio environment.

17. A non-transitory computer-readable medium, comprising program instructions stored thereon for at least performing the following: Querying one or more acoustic reporters in an audio environment using a radio frequency beam having a wavelength below 10 millimeters; Analyze one or more sound signals reported by the one or more acoustic reporters to determine the location of one or more sound sources providing the one or more sound signals; Use the location of the one or more sound sources to determine one or more sound propagation paths within the audio environment; And Prioritize the query of the one or more acoustic reporters based on the frequency of the sound signals reported by the one or more acoustic reporters, such that high-frequency signals have a higher priority than low-frequency signals.

18. The non-transitory computer-readable medium according to claim 17, wherein the program instructions are further configured to cause: Perform a scan of the radio frequency beam to determine the location of one or more acoustic reporters.

19. The non-transitory computer-readable medium according to claim 17, wherein the location of the one or more acoustic reporters is used together with the location of the one or more sound sources to determine the one or more sound propagation paths within the audio environment.

Images