Device with loudspeaker and image sensor

Abstract

In one implementation, a method of playing audio data is performed at a device including a frame configured for insertion into an external ear, a speaker coupled to the frame, an image sensor coupled to the frame, one or more processors, and a non-transitory memory. The method includes capturing one or more images of a physical environment using the image sensor. The method includes generating audio data based on the one or more images of the physical environment. The method includes playing the audio data via the speaker.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Patent No. 63 / 354,018 filed on June 21, 2022, which is hereby incorporated by reference in its entirety. Technical Field

[0003] The present disclosure generally relates to devices that include one or more speakers and one or more image sensors. Background Art

[0004] Various ear-worn devices, such as headphones or earbuds, include speakers that output sound to the user. Various head-worn devices, such as binaural headphones or extended reality (XR) headsets, may similarly include speakers. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] So that the present disclosure can be understood by those of ordinary skill in the art, a more detailed description may be obtained with reference to aspects of some exemplary implementations, some of which are illustrated in the accompanying drawings.

[0006] Figure 1 is a perspective view of a head mounted device according to some specific implementations.

[0007] Figure 2 is a block diagram of an exemplary operating environment according to some implementations.

[0008] Figure 3 Various fields of view are shown according to some specific implementations.

[0009] Figure 4 is a flowchart representation of a method of playing audio data according to some specific implementations.

[0010] As is common practice, the various features shown in the drawings may not be drawn to scale. Therefore, for clarity, the sizes of the various features may be arbitrarily expanded or reduced. In addition, some drawings may not depict all components of a given system, method, or device. Finally, throughout the specification and drawings, similar reference numerals may be used to represent similar features. Summary of the invention

[0011] Various embodiments disclosed herein include devices, systems, and methods for playing audio data. In various embodiments, the method is performed by a device including a frame configured to be inserted into an outer ear, a speaker coupled to the frame, an image sensor coupled to the frame, one or more processors, and a non-transitory memory. The method includes capturing one or more images of a physical environment using an image sensor. The method includes generating audio data based on one or more images of a physical environment. The method includes playing the audio data via a speaker.

[0012] According to some specific implementations, a device includes a frame configured to be inserted into an outer ear. The device includes one or more processors coupled to the frame. The device includes a speaker coupled to the frame and configured to output sound based on audio data received from the one or more processors. The device includes an image sensor, which is coupled to the frame and configured to provide one or more images of a physical environment to the one or more processors. The one or more processors are configured to generate audio data based on the one or more images of the physical environment.

[0013] According to some specific implementations, a device includes one or more processors, non-volatile memory, and one or more programs; one or more programs are stored in the non-volatile memory and are configured to be executed by one or more processors. One or more programs include instructions for executing or causing the execution of any method described herein. According to some specific implementations, a non-volatile computer-readable storage medium stores instructions that, when executed by one or more processors of the device, cause the device to execute or cause the execution of any of the methods described herein. According to some specific implementations, a device includes: one or more processors, non-volatile memory, and a component for executing or causing the execution of any of the methods described herein. DETAILED DESCRIPTION

[0014] Many details are described in order to provide a thorough understanding of the example implementations shown in the accompanying drawings. However, the accompanying drawings only illustrate some example aspects of the present disclosure and should not be considered limiting. One of ordinary skill in the art will appreciate that other effective aspects and / or variations do not include all of the specific details described herein. In addition, well-known systems, methods, components, devices, and circuits are not described in detail so as not to obscure more relevant aspects of the example implementations described herein.

[0015] Various ear-worn devices such as headphones or earbuds include speakers that output sound to the user. Various head-worn devices such as binaural headphones or extended reality (XR) headsets may similarly include speakers. By including an image sensor on such a device to capture images of the physical environment and output audio based on the captured images, a variety of user experiences may be enabled.

[0016] Figure 1A perspective view of a head mounted device 150 according to some specific implementations is shown. The head mounted device 150 includes a frame 151 including two earphones 152, each earphone being configured to abut a respective outer ear of a user. The frame 151 also includes a front component 154 configured to reside in front of the user's field of view. Each earphone 152 includes an inwardly facing speaker 160 (e.g., facing inwardly, facing outwardly, facing downwardly, etc.) and an outwardly facing imaging system 170. In addition, the front component 154 includes a display 181 for displaying images to the user, an eye tracker 182 for determining the gaze direction or gaze point of the user (which may include one or more rear-facing image sensors configured to capture images of at least one eye of the user), and a scene tracker 190 (which may include one or more forward-facing image sensors configured to capture images of the physical environment) that may supplement the imaging system 170 of the earphones 152.

[0017] In various implementations, the head mounted device 150 does not include the front member 154. Thus, in various implementations, the head mounted device is embodied as a headphone device that includes a frame 151 having two earphones 152 each configured to surround a corresponding outer ear of a user, and a headband coupled to the earphones 152 and configured to rest on top of the user's head. In various implementations, each earphone 152 includes an inward facing speaker 160 and an outward facing imaging system 170.

[0018] In various embodiments, the head-mounted device is a headband. Therefore, in various embodiments, the head-mounted device 150 (or its earphones 150) is embodied as one or more earplugs or earphones. For example, the earplug includes a frame configured to be inserted into the outer ear. Specifically, in various embodiments, the frame is configured to be inserted into the outer ear of a human, a person and / or a user of the earplug. The earplug includes a speaker 160 and an imaging system 170 coupled to the frame, the speaker being configured to output sound, and the imaging system being configured to capture one or more images of the physical environment in which the earplug exists. In various embodiments, the imaging system 170 includes one or more cameras (or image sensors). The earplug also includes one or more processors coupled to the frame. The speaker 160 is configured to output sound based on audio data received from the one or more processors, and the imaging system 170 is configured to provide image data to the one or more processors. In various embodiments, the audio data provided to the speaker 160 is based on image data obtained from the imaging system 170.

[0019] As described above, in various embodiments, the earplug includes a frame configured for insertion into the outer ear. Specifically, in various embodiments, the frame is sized and / or shaped to be inserted into the outer ear. The frame includes a surface that rests in the intertragic notch, thereby preventing the earplug from falling vertically downward. In addition, the frame includes surfaces that abut the tragus and anti-tragus, thereby holding the earplug horizontally in place. When inserted, the speaker 160 of the earplug is directed toward the ear canal, and the imaging system 170 of the earplug is directed outward and exposed to the physical environment.

[0020] Although the head mounted device 150 is an exemplary device that can perform one or more of the methods described herein, it should be understood that other wearable devices having one or more speakers and one or more cameras can also be used to perform the methods. The wearable audio device can be embodied in other wired or wireless form factors, such as a head mounted device, an in-ear device, a circum-ear device, an on-ear device, an open device, a closed device, a bone conduction device, or other audio devices.

[0021] Figure 2 is a block diagram of an operating environment 20 according to some implementations. The operating environment 20 includes a headset 200. In various implementations, the headset 200 corresponds to Figure 1 The headset 200 includes a frame 201. In various implementations, the frame 201 is configured for insertion into an outer ear. The headset 200 includes one or more processors 210 coupled to the frame 201 and in various implementations within the frame 201. The headset 200 includes a memory 220 (e.g., non-volatile memory) coupled to the frame 201 and in various implementations coupled to the one or more processors 210 within the frame 201.

[0022] The headset 200 includes a speaker 230 coupled to the frame 201 and configured to output sound based on audio data received from the one or more processors 210. The headset 200 includes an imaging system 240, which is coupled to the frame 201 and configured to capture an image of the physical environment in which the headset 200 exists and provide image data representing the image to the one or more processors 210. In various specific implementations, the imaging system 240 includes one or more cameras 241A, 241B. In various specific implementations, different cameras 241A, 241B have different fields of view. For example, in various specific implementations, the imaging system 240 includes a forward-facing camera and a rear-facing camera. In various specific implementations, at least one of the cameras 241A includes a fisheye lens 242, for example to increase the size of the field of view of the camera 241A. In various specific implementations, the imaging system 240 includes a depth sensor 243. Thus, in various embodiments, for each pixel of a plurality of pixels representing a location in a physical environment, the image data includes a color (or grayscale) value for the location representing the amount and / or wavelength of light detected at the location and a depth value representing the distance from the headset 200 to the location.

[0023] In various implementations, the headset 200 includes a microphone 250 coupled to the frame 201 and configured to generate ambient sound data indicative of sounds in the physical environment. In various implementations, the headset 200 includes an inertial measurement unit (IMU) 260 coupled to the frame 201 and configured to determine movement and / or orientation of the headset 200. In various implementations, the IMU 260 includes one or more accelerometers and / or one or more gyroscopes. In various implementations, the headset 200 includes a communication interface 270 coupled to the frame and configured to send data and receive data from other devices. In various implementations, the communication interface 270 is a wireless communication interface.

[0024] The headset 200 includes one or more communication buses 204 within the frame 201 for interconnecting the various components described above and / or additional components that may be included with the headset 200.

[0025] In various implementations, operating environment 20 includes a second earphone 280, which may include any or all of the components of earphone 200. In various implementations, frame 201 of earphone 200 is configured for insertion into one outer ear of a user, and the frame of second earphone 200 is configured for insertion into the other outer ear of the user (e.g., by being a mirrored version of frame 201).

[0026] In various implementations, the operating environment 20 includes a controller device 290. In various implementations, the controller device 290 is a smart phone, a tablet computer, a laptop computer, a desktop computer, a set-top box, a smart TV, a digital media player, or a smart watch. The controller device 290 includes one or more processors 291 coupled to a memory 292, a display 293, and a communication interface 294 via one or more communication buses 214. In various implementations, the controller device 290 includes additional components, such as any or all of the components described above with respect to the headset 200.

[0027] In various implementations, the display 293 is configured to display images based on display data provided by the one or more processors 291. In contrast, in various implementations, the earphone 200 (and similarly, the second earphone 280) does not include a display, or at least does not include a display within the user's field of view when inserted into the user's outer ear.

[0028] In various implementations, the one or more processors 210 of the headset 200 generate audio data that is provided to the speaker 230 based on the image data received from the imaging system 240. In various implementations, the one or more processors 210 of the headset 240 send the image data to the controller device 290 via the communication interface 270, the one or more processors of the controller device 290 generate the audio data based on the image data, and the headset 200 receives the audio data via the communication interface 270. In any set of implementations, the audio data is based on the image data.

[0029] Figure 3 Various fields of view are shown according to some specific implementations. The user field of view 301 of the user 30 generally extends about 300 degrees, with varying degrees of visual perception within this range. For example, excluding far peripheral vision, the user field of view 301 is only about 120 degrees, and the user field of view 301 including only foveal vision (or central vision) is only about 5 degrees.

[0030] In contrast, the system ( Figure 1The head mounted device 150 of the embodiment of the present invention may have a device field of view that includes a field of view outside the user field of view 301 of the user 30. For example, the system may include a forward-facing and outward-facing camera that includes a fisheye lens with a 180-degree field of view close to each ear of the user 30, and may have a device with a forward field of view 302 of approximately 300 degrees. In addition, the system may also include a rearward-facing and outward-facing camera that includes a fisheye lens with a 180-degree field of view close to each ear of the user 30, and may also have a device with a rearward field of view 303 of approximately 300 degrees. In various specific implementations, a system including multiple cameras close to each ear of the user may have a full 360-degree device field of view (e.g., including a forward device field of view 302 and a rearward device field of view 303). It should be understood that in various specific implementations, a camera (or combination of cameras) may have a smaller or larger field of view than the above examples.

[0031] The above-described system can perform a variety of functions. For example, in various implementations, when audio (e.g., music or an audiobook) is playing via a speaker, in response to detecting a particular gesture in an image captured by an imaging system (even a gesture performed outside of the user's field of view), the system can alter the playback of the audio (e.g., by pausing or changing the volume of the audio). For example, in various implementations, in response to detecting a gesture performed by a user near the user's ear: closing an open hand into a clenched fist, the system pauses the playback of the audio via the speaker.

[0032] As another example, in various implementations, when audio is played via a speaker, in response to detecting in an image captured by an imaging system that a person is attempting to engage the user in a conversation or otherwise talk to the user (even if the person is outside the user's field of view), the system may alter the playback of the audio. For example, in various implementations, in response to detecting that a person behind the user is attempting to talk to the user, the system reduces the volume of the audio played via the speaker and stops executing the active noise cancellation algorithm.

[0033] As another example, in various implementations, in response to detecting an object or event of interest in the physical environment in an image captured by the imaging system, the system generates an audio notification. For example, in various implementations, in response to detecting a person in the user's vicinity or outside the user's field of view trying to get the user's attention (e.g., by waving the person's arm), the device plays a warning notification (e.g., a sound similar to a person saying "Hi!") via a speaker. In various implementations, the system plays the warning notification spatially via two or more speakers so that the user perceives the warning notification as coming from the direction of the detected object.

[0034] As another example, in various implementations, in response to detecting an object or event of interest in a physical environment in an image captured by an imaging system, the system stores in memory an indication that a particular object was detected (which may be determined using the image from the imaging system) in association with a location where the object was detected (which may also be determined using the image from the imaging system) and the time at which the object was detected. In response to a user query (e.g., a voice query detected via a microphone), the system provides an audio response. For example, in response to detecting a water bottle in a user's office, the system stores an indication that the water bottle was detected in the office, and in response to a user query at a later time, "Where's my water bottle?", the device may generate audio that approximates a person saying "in your office".

[0035] As another example, in various implementations, in response to detecting an object in the physical environment approaching the user in an image captured by the imaging system, the system generates an audio notification. For example, in various implementations, in response to detecting that a car is approaching the user at a speed exceeding a threshold, the system plays a warning notification (e.g., a sound similar to the beeping of a car horn) via a speaker. In various implementations, the system plays the warning notification spatially via two or more speakers so that the user perceives the warning notification as coming from the direction of the detected object.

[0036] Figure 4 4 is a flowchart representation of a method 400 for playing an audio notification according to some implementations. In various implementations, the method 400 is performed by a device (e.g., Figure 1 Head mounted device 150 or Figure 2 In various implementations, method 400 is performed by a device including a frame configured to be inserted into an outer ear, a speaker coupled to the frame, and an image sensor coupled to the frame. In various implementations, method 400 is performed by a device without a display or by a device including a frame that is not physically coupled to a display. In various implementations, method 400 is performed by a device with a display. In various implementations, using an audio device (e.g., Figure 1 Head mounted device 150 or Figure 2 The headset 200) is combined with a peripheral device (e.g., Figure 2 The method 400 is performed by a controller device 290 of the present invention. In various specific implementations, the method 400 is performed by a processing logic component (including hardware, firmware, software, or a combination thereof). In various specific implementations, the method 400 is performed by a processor executing instructions (e.g., code) stored in a non-transitory computer-readable medium (e.g., memory).

[0037] The method 400 begins in block 410, where a device captures one or more images of a physical environment using an image sensor. In various implementations, the image sensor has a device field of view that is different from a user's field of view, at least at the corresponding one or more times of capturing the one or more images and inserting the frame into the outer ear. In various implementations, the image sensor includes a fisheye lens. Thus, in various implementations, the device field of view is between about 120 degrees and 180 degrees, and in particular, between about 170 degrees and 180 degrees.

[0038] Method 400 continues in block 420 where the device generates audio data based on the one or more images of the physical environment. In various implementations, generating audio data based on the one or more images of the physical environment includes sending the one or more images of the environment to a peripheral device and receiving audio data from the peripheral device.

[0039] Method 400 continues at block 430 where the device plays the audio data via a speaker.

[0040] (In box 420) Generating audio data based on one or more images of a physical environment covers a wide range of processing to enable various user experiences. For example, in various specific implementations, generating audio data based on one or more images of a physical environment includes detecting an object or event of interest in the physical environment and generating audio data based on the detection. In various specific implementations, generating audio data based on the detection includes creating an audio signal indicating the detection. Therefore, in various specific implementations, playing audio data includes playing new sounds that would not have been played if no object or event of interest was detected. In various specific implementations, playing audio data includes playing a sound when no sound would be played if no object or event of interest was detected. For example, in response to detecting a snake as an object with an object type of "SNAKE" using, for example, computer vision techniques such as a model trained to detect and classify various objects, the device generates an audio notification that simulates the sound of a person saying the object type of the object or simulates the sound of the object (e.g., the sound made by a rattlesnake).

[0041] In various specific implementations, generating audio data based on detection includes changing an audio stream. For example, in response to detecting a particular gesture, the device pauses playback of the audio stream or changes the volume of the audio stream. For another example, in response to detecting a person attempting to communicate with the user, the device stops performing active noise cancellation on the audio stream.

[0042] In various specific implementations, the device also includes a microphone configured to generate ambient sound data, and generating audio data is further based on the ambient sound data. In various specific implementations, the ambient sound data includes voice input. For example, in response to detecting in one or more images of the physical environment that a user performs a gesture indicating an object of a specific object type in the physical environment (e.g., pointing to a light) and detecting in the ambient sound data that the user issues a voice command to translate the object type of the object (e.g., "What is this called in Spanish?"), the device generates audio data that simulates the sound of a person speaking the translation of the object type of the object (e.g., "la lámpara"). For another example, in response to detecting in one or more images of the physical environment that a user brushes his teeth and detecting in the ambient sound data that the user issues a voice inquiry about the detection at a later time (e.g., "Did I brush my teeth this morning?"), the device generates audio data that simulates the sound of a person indicating the detection (e.g., "Yes, you brushed your teeth at 6:33 today").

[0043] As another example, in response to detecting a person attempting to communicate with the user based at least in part on the ambient sound data, the device pauses playback of the audio stream or reduces the volume of the audio stream.

[0044] In various implementations, the audio data is generated independently of the ambient sound data. For example, in response to detecting a moving object in one or more images of the physical environment independently of the ambient sound data, the device generates an audio notification of the detection. In various implementations, the audio notification simulates the sound of a person indicating the detection of motion, e.g., "motion." In various implementations, the audio notification simulates the sound of an object moving in the physical environment, such as the rustling of leaves or the breaking of branches, which can be based on the object type of the moving object.

[0045] In various embodiments, the device also includes an inertial measurement unit (IMU) configured to generate posture data, and generating audio data is further based on the posture data. For example, in response to detecting that a user has fallen based on one or more images of the environment and the posture data, the device generates an audio query ("Are you okay?"). In various embodiments, the audio data is played spatially from a location based on one or more images of the environment, for example, stereo panning or binaural rendering. For example, in response to detecting an object in one or more images of the environment, the device spatially plays an audio notification to be perceived as being generated from the location of the detected object. In various embodiments, the posture data is used to spatialize the audio data.

[0046] In various implementations, to spatially play audio, a second device (e.g., Figure 2 The headset 280) executes the method 400.

[0047] As mentioned above, in various embodiments, the image sensor has a device field of view that is different from the user's field of view. In various embodiments, the audio data is based on portions of one or more images of the physical environment outside the user's field of view. For example, in response to detecting a moving object (e.g., a vehicle) moving toward the device in a portion of the image of the physical environment outside the user's field of view (otherwise referred to as an entering object), the device generates an audio notification of the detection. In various embodiments, the audio notification simulates the sound of a person indicating that an entering object has been detected, such as "enter" or "caution." In various embodiments, the audio notification simulates the sound of an object moving in the physical environment, such as a car horn or a bicycle bell, which may be based on the object type of the entering object.

[0048] Although various aspects of specific implementations within the scope of the appended claims are described above, it should be apparent that the various features of the above-mentioned specific implementations can be embodied in a variety of forms, and any specific structures and / or functions described above are merely illustrative. Based on this disclosure, it should be understood by those skilled in the art that the aspects described herein can be implemented independently of any other aspects, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement a device and / or a method can be practiced. In addition, in addition to or different from one or more aspects set forth herein, other structures and / or functions can be used to implement such a device and / or such a method can be practiced.

[0049] It will also be understood that, although the terms "first", "second", etc. may be used to describe various elements in this article, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, a first node may be referred to as a second node, and similarly, a second node may be referred to as a first node, which changes the meaning of the description, as long as all occurrences of the "first node" are consistently renamed and all occurrences of the "second node" are consistently renamed. Both the first node and the second node are nodes, but they are not the same node.

[0050] The terms used herein are only for describing specific implementations and are not intended to limit the claims. As used in the description of this specific implementation and the appended claims, the singular forms of "one", "a kind of" and "the" are intended to also cover the plural forms, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" used herein refers to and covers any and all possible combinations of one or more items in the associated listed items. It will also be understood that the term "comprising" when used in this specification specifies the presence of stated features, integers, steps, operations, elements and / or parts, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts, and / or their groupings.

[0051] As used herein, the term “if” may be interpreted to mean “when the antecedent is true” or “when the antecedent is true” or “in response to determining” or “upon determining” or “in response to detecting” that the antecedent is true, depending on the context. Similarly, the phrase “if it is determined that [the antecedent is true]” or “if [the antecedent is true]” or “when [the antecedent is true]” is interpreted to mean “upon determining that the antecedent is true” or “in response to determining” or “upon determining” that the antecedent is true or “when detecting that the antecedent is true” or “in response to detecting” that the antecedent is true, depending on the context.

Claims

1. A method comprising: At a device comprising a frame configured for insertion into an outer ear, a speaker coupled to the frame, an image sensor coupled to the frame, one or more processors, and a non-transitory memory: capturing one or more images of a physical environment using the image sensor; generating audio data based on the one or more images of the physical environment; and The audio data is played via the speaker. 2 . The method of claim 1 , wherein generating the audio data based on the one or more images of the physical environment comprises sending the one or more images of the physical environment to a peripheral device and receiving the audio data from the peripheral device.

3. The method of claim 1 or 2, wherein generating the audio data comprises creating an audio signal. The method according to claim 1 , wherein generating the audio data comprises altering an audio stream.

5. The method according to any one of claims 1 to 4, wherein the device further comprises a microphone configured to generate ambient sound data, and wherein generating the audio data is further based on the ambient sound data. The method of claim 5 , wherein the ambient sound data comprises speech input.

7. The method of any one of claims 1 to 4, wherein the device further comprises a microphone configured to generate ambient sound data, and wherein generating the audio data is independent of the ambient sound data.

8. The method according to any one of claims 1 to 7, wherein the device further comprises an inertial measurement unit (IMU) configured to generate posture data, and wherein generating the audio data is further based on the posture data.

9. The method of any one of claims 1 to 8, wherein the audio data is played spatially from a position based on the one or more images of the environment.

10. A method according to any one of claims 1 to 9, wherein the image sensor has a device field of view that is different from a user field of view, and the audio data is based on portions of the one or more images of the physical environment that are outside the user field of view.

11. A device comprising: a frame configured for insertion into an outer ear; one or more processors coupled to the framework; a speaker coupled to the frame and configured to output sound based on audio data received from the one or more processors; and an image sensor coupled to the framework and configured to provide one or more images of the physical environment to the one or more processors, Wherein the one or more processors are configured to generate the audio data based on the one or more images of the physical environment.

12. The device of claim 1, wherein the one or more processors are configured to generate the audio data based on the one or more images of the environment by sending the one or more images of the physical environment to a peripheral device and receiving the audio data from the peripheral device.

13. The apparatus of claim 11 or 12, wherein the one or more processors are configured to generate the audio data by creating an audio signal.

14. The apparatus according to claim 11 or 12, wherein the one or more processors are configured to generate the audio data by modifying an audio stream.

15. The device of any one of claims 11 to 14, further comprising a microphone configured to generate ambient sound data, wherein the one or more processors are configured to generate the audio data further based on the ambient sound data.

16. The apparatus of claim 15, wherein the ambient sound data comprises voice input.

17. The device of any one of claims 11 to 14, further comprising a microphone configured to generate ambient sound data, wherein the one or more processors are configured to generate the audio data independently of the ambient sound data.

18. The device of any one of claims 11 to 17, further comprising an inertial measurement unit (IMU) configured to generate posture data, wherein the one or more processors are configured to generate the audio data further based on the posture data.

19. An apparatus according to any one of claims 11 to 18, wherein the audio data is played from a location spatially based on the one or more images of the environment.

20. A device according to any one of claims 11 to 19, wherein the image sensor has a device field of view that is different from a user field of view, and the audio data is based on portions of the one or more images of the physical environment that are outside the user field of view.

21. The apparatus of any one of claims 11 to 20, wherein the image sensor comprises a fisheye lens.

22. The device of any one of claims 11 to 21, wherein the frame is not physically coupled to a display.

23. A device comprising: a frame configured for insertion into an outer ear; a speaker coupled to the frame; an image sensor coupled to the frame; one or more processors; non-transitory memory; and One or more programs stored in the non-transitory memory, which, when executed by the one or more processors, cause the device to perform any of the methods according to claims 1 to 10.

24. A non-volatile memory storing one or more programs that, when executed by one or more processors of a device comprising a frame configured to be inserted into an outer ear, a speaker coupled to the frame, and an image sensor coupled to the frame, causes the device to perform any of the methods described in claims 1 to 10.

25. An apparatus comprising: a frame configured for insertion into an outer ear; a speaker coupled to the frame; an image sensor coupled to the frame; one or more processors; non-transitory memory; and Means for causing the device to perform any of the methods according to claims 1 to 10.

Images