Wearable device

Abstract

Disclosed is a wearable device, which relates to the field of multimedia. In the wearable device, microphones and vibration sensors are used, and the positions of the vibration sensors and the positions of sound pickup holes, that is, sound pickup positions of the microphones, are set to reduce the interference of a picked-up sound signal, thereby improving the sound signal pick-up function of the wearable device, expanding application scenarios of the picked-up sound signal, and improving the quality of the sound signal and the design tolerance and upper capability limits of algorithms. For example, the signal-to-noise ratio of the sound signal is improved, more accurate information regarding external sound directivity is provided, and a higher degree of freedom is provided for the design and development of algorithms such as sound source localization, spatial audio, sound leakage prevention, active noise reduction and voice enhancement.

Description

wearable devices

[0001] This application claims priority to Chinese patent application filed on January 2, 2025, with application number 202510006023.1 and entitled "Wearable Devices", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of multimedia, and more particularly to a wearable device. Background Technology

[0003] Currently, wearable devices, in addition to playback, also have sound pickup capabilities. They use sensors to capture ambient sound signals, voice signals, and sound signals played by the device, applying these signals to scenarios such as calls, noise reduction, and scene recognition. Typically, wearable devices use only one type of sensor to pick up sound signals. However, because the environments in which wearable devices operate are complex and variable, sound signals picked up using a single type of sensor are easily affected by external interference such as background noise and wind noise, resulting in poor sound pickup capabilities for wearable devices. Summary of the Invention

[0004] This application provides a wearable device that improves the sound pickup capability of the wearable device by using multiple types of sensors and setting the sound pickup positions of the multiple types of sensors.

[0005] In a first aspect, a wearable device is provided, including a housing, a first microphone, a first pickup hole, a second microphone, a second pickup hole, and a first vibration sensor. The housing of the wearable device includes a first suspension portion and a second suspension portion connected to the first suspension portion. The first suspension portion is used to suspend behind the auricle, and the second suspension portion is used to suspend in front of the auricle. The wearable device includes a first microphone and a first pickup hole. The first microphone is located within the first suspension portion, and the first pickup hole is formed on the wall of the first suspension portion, located on the side away from the user. The first microphone is used to pick up sound signals at the first pickup hole. The wearable device also includes a second microphone and a second pickup hole. The second microphone is located within the second suspension portion, and the second pickup hole is formed on the wall of the second suspension portion, located on the side away from the user. The second microphone is used to pick up sound signals at the second pickup hole. The wearable device also includes a first vibration sensor located within the first suspension portion, located on the side closer to the user.

[0006] By using microphones and vibration sensors in wearable devices and carefully configuring the positions of the vibration sensors and the microphone pickup holes (i.e., the microphone's pickup location), interference with the picked-up sound signals can be reduced. This improves the sound signal pickup capabilities of wearable devices, expands the application scenarios of the picked-up sound signals, and enhances the quality of the sound signals and the tolerance and upper limit of algorithm design. For example, it can improve the signal-to-noise ratio of the sound signal, provide more accurate information on the directionality of external sounds, and offer greater freedom for the design and development of algorithms for sound source localization, spatial audio, sound leakage prevention, active noise reduction, and speech enhancement.

[0007] In one possible implementation, the first pickup hole is located at the end of the first suspension portion, which is the end of the first suspension portion away from the second pickup hole.

[0008] The opening of the first pickup hole is oriented towards the user's throat in order to pick up the user's speech signal.

[0009] In another possible implementation, the wearable device also includes a third microphone and a third pickup hole. The third microphone is used to pick up the sound signal at the third pickup hole. The third microphone is located inside the first suspension part. The third pickup hole is opened on the wall of the first suspension part and is located on the side away from the user. The first pickup hole is closer to the end of the first suspension part than the third pickup hole.

[0010] With the first and third pickup holes located behind the auricle, they are designed to pick up sound signals from behind the auricle. The first and third pickup holes form an array to reduce external noise and airflow interference in the picked-up sound signals.

[0011] In another possible implementation, the third pickup hole is closer to the user than the second pickup hole. For example, the opening direction of the third pickup hole is oriented in a direction approximately parallel to the ear projection plane.

[0012] The third pickup hole on the wall of the first suspension part is closer to the user in order to pick up sound signals behind the auricle.

[0013] In another possible implementation, the wearable device also includes a fourth microphone and a fourth pickup hole. The fourth microphone is used to pick up the sound signal at the fourth pickup hole. The fourth microphone is located inside the second suspension part, and the fourth pickup hole is opened on the wall of the second suspension part and is located on the side away from the user. The fourth pickup hole is farther away from the first suspension part than the second pickup hole.

[0014] When the second and fourth microphone holes are located in front of the auricle, they are used to pick up sound signals in front of the auricle. The second and fourth microphone holes form an array, which can improve the recognition of the direction of the sound signal.

[0015] In another possible implementation, the fourth pickup hole is closer to the user than the second pickup hole. The fourth pickup hole on the wall of the second suspension is closer to the user to pick up sound signals in front of the auricle.

[0016] In another possible implementation, the opening direction of the pickup hole has an angle with the ear projection plane, and the pickup hole includes a first pickup hole, a second pickup hole, a third pickup hole, and a fourth pickup hole.

[0017] In another possible implementation, the first microphone is a directional microphone. The opening of the first pickup hole is oriented towards the user's throat.

[0018] In another possible implementation, the fourth microphone is a directional microphone. The opening of the fourth pickup hole faces the direction in front of the user's eyes.

[0019] Using directional microphones effectively improves the ability of microphone arrays to pick up valid sound signals and determine the direction of the sound signals.

[0020] In another possible implementation, the wearable device further includes a bone conduction oscillator and a second vibration sensor, the second vibration sensor being used to pick up the vibration signal from the bone conduction oscillator. The bone conduction oscillator and the second vibration sensor are located within a second suspension unit. Vibration signals within the wearable device are picked up by the vibration sensor inside the wearable device.

[0021] In another possible implementation, the wearable device also includes an air-conducting speaker, a sound outlet, and a fifth microphone. The fifth microphone is used to pick up the sound signal from the air-conducting speaker. The air-conducting speaker and the fifth microphone are located inside the second suspension part. The sound outlet is opened on the wall of the second suspension part and is located on the side closer to the user. The sound outlet is used to output the sound signal from the air-conducting speaker.

[0022] The sound signal played by the wearable device is picked up by the microphone inside the wearable device.

[0023] In another possible implementation, the wearable device includes any one of headphones, glasses, or a head-mounted device.

[0024] Secondly, a signal processing method is provided for use in wearable devices as described in the first aspect above or in any possible implementation thereof. The wearable device processes the picked-up sound signals based on an audio algorithm to achieve functions such as active noise reduction, scene judgment, sound source localization, speech enhancement, and hearing aids.

[0025] Thirdly, a signal processing apparatus is provided for implementing the various methods described above. This signal processing apparatus includes modules, units, or means corresponding to the methods described above. These modules, units, or means can be implemented in hardware, software, or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the functions described above.

[0026] Fourthly, a wearable device is provided, comprising: a processor, a memory, a microphone, a vibration sensor, and a speaker; the memory is used to store computer instructions, which, when executed by the processor, cause the wearable device to perform the method described in the second aspect. The memory may be coupled to the processor, or may be independent of the processor; the microphone is used to pick up sound signals; the vibration sensor is used to pick up vibration signals; and the speaker is used to output a sound source.

[0027] Fifthly, a computer-readable storage medium is provided that stores a computer program or instructions that, when executed on a signal processing apparatus, enable the signal processing apparatus to perform the methods of any of the above aspects or any implementation thereof.

[0028] In a sixth aspect, a computer program product containing instructions is provided, which, when run on a signal processing apparatus, enables the signal processing apparatus to perform the method of any of the above aspects or any implementation thereof.

[0029] The technical effects of any of the implementation methods in aspects two through six can be found in the technical effects of the corresponding implementation method in aspect one, and will not be repeated here.

[0030] Among these, any possible implementation methods of any one of the above aspects can be combined, provided that the solutions do not contradict each other. Attached Figure Description

[0031] Figure 1 is a schematic diagram of a suspension part provided in this application;

[0032] Figure 2 is a schematic diagram showing the location of a pickup hole provided in this application;

[0033] Figure 3 is a schematic diagram of the position of a vibration sensor provided in this application;

[0034] Figure 4 is a schematic diagram of another pickup hole location provided in this application;

[0035] Figure 5 is a schematic diagram of another pickup hole location provided in this application;

[0036] Figure 6 is a schematic diagram showing the location of another vibration sensor provided in this application;

[0037] Figure 7 is a schematic diagram of the position of a microphone provided in this application;

[0038] Figure 8 is a schematic diagram showing the location of another microphone provided in this application;

[0039] Figure 9 is a schematic diagram of a speech signal pickup scenario provided in this application;

[0040] Figure 10 is a schematic diagram of a binaural array provided in this application;

[0041] Figure 11 is a schematic diagram of a sound signal pickup scenario played by a wearable device according to this application;

[0042] Figure 12 is a schematic diagram of a wearable device provided in this application;

[0043] Figure 13 is a schematic diagram of an application scenario of an acoustic signal provided in this application;

[0044] Figure 14 is a structural schematic diagram of a wearable device provided in this application. Detailed Implementation

[0045] To facilitate understanding, the main terms used in this application will be explained first.

[0046] Wearable devices are electronic devices that integrate computer and communication functions. Users can wear them directly on their bodies, such as on the head, wrist, or chest, and interact with the user through contact with the body or wireless communication.

[0047] The wearable devices described in this application include headphones, glasses, and head-mounted devices. Head-mounted devices include virtual reality (VR) devices, augmented reality (AR) devices, mixed reality (MR) devices, or extended reality (ER) devices, etc.

[0048] Headphone types include, but are not limited to, in-ear, semi-in-ear (or semi-open), and open-back headphones. Headphone shapes include, but are not limited to, ear-hook, neckband, and clip-on headphones.

[0049] The wearable device described in this application has sound playback and sound pickup functions. Wearable devices with sound pickup functions include acoustic sensors. Commonly used acoustic sensors include microphones (Mic) and vibration pick-up (VPU) sensors. Microphones include microelectromechanical system microphones (MEMS microphones) and electret condenser microphones (ECM).

[0050] Microphones are primarily used to pick up sound signals transmitted through the air. Vibration sensors are primarily used to pick up vibration signals transmitted through bones, soft tissues, etc., after an object vibrates.

[0051] When wearable devices use only one type of sensor to pick up sound signals, the limited capabilities of a single sensor make them susceptible to interference from external factors such as background noise and wind noise. For example, in high-noise environments, sound signals picked up by a microphone are significantly affected, resulting in a low signal-to-noise ratio. Applying these picked-up sound signals to scenarios such as calls, noise reduction, and scene recognition leads to poor performance.

[0052] In other implementations, the signal-to-noise ratio is improved by deploying multiple sensor arrays. However, due to limitations in device size and power consumption, the number of sensors that can be used is limited, thus restricting the array's capabilities.

[0053] Therefore, current wearable devices have poor sound pickup capabilities, resulting in poor quality of the sound signals they pick up.

[0054] To address the issue of poor sound pickup capabilities in wearable devices, this application provides a wearable device, specifically a scheme for arranging the sound pickup positions of various types of sensors within the wearable device. For example, the wearable device's housing includes a first suspension portion and a second suspension portion connected to the first suspension portion. The first suspension portion is used to hang behind the ear, and the second suspension portion is used to hang in front of the ear. The wearable device includes a first microphone and a first pickup hole. The first microphone is used to pick up sound signals at the first pickup hole, and the first microphone is located within the first suspension portion. The first pickup hole is opened on the wall of the first suspension portion and is located on the side away from the user. The wearable device also includes a second microphone and a second pickup hole. The second microphone is used to pick up sound signals at the second pickup hole, and the second microphone is located within the second suspension portion. The second pickup hole is opened on the wall of the second suspension portion and is located on the side away from the user. The wearable device also includes a first vibration sensor located within the first suspension portion, and the first vibration sensor is located close to the user.

[0055] By using microphones and vibration sensors in wearable devices and carefully configuring the positions of the vibration sensors and the microphone pickup holes (i.e., the microphone's pickup location), interference with the picked-up sound signals can be reduced. This improves the sound signal pickup capabilities of wearable devices, expands the application scenarios of the picked-up sound signals, and enhances the quality of the sound signals and the tolerance and upper limit of algorithm design. For example, it can improve the signal-to-noise ratio of the sound signal, provide more accurate information on the directionality of external sounds, and offer greater freedom for the design and development of algorithms for sound source localization, spatial audio, sound leakage prevention, active noise reduction, and speech enhancement.

[0056] The embodiments of the wearable device provided in this application will be described in detail below with reference to the accompanying drawings.

[0057] This application provides a wearable device, which includes a housing, a first microphone, a first pickup hole, a second microphone, a second pickup hole, and a first vibration sensor.

[0058] The housing includes a first suspension portion and a second suspension portion connected to the first suspension portion. The first suspension portion is used to suspend behind the auricle, and the second suspension portion is used to suspend in front of the auricle. The wearable device is suspended on the ear via the first and second suspension portions so that the user can hear the sound output by the wearable device. The auricle is the visible part of the ear located on the outer side of the head.

[0059] The first suspension part and the second suspension part are each part of the housing. The housing is formed by the first suspension part and the second suspension part.

[0060] In some embodiments, the first suspension portion and the second suspension portion form an integral part of the housing and are not connected by other components. For example, as shown in FIG1(a), the wearable device is an ear-hook earphone, with the portion located behind the ear being the first suspension portion 110 and the portion located in front of the ear being the second suspension portion 120. As shown in FIG1(b), the wearable device is a pair of glasses, with the portion located behind the ear being the first suspension portion 110 and the portion located in front of the ear being the second suspension portion 120.

[0061] In other embodiments, the first suspension portion and the second suspension portion are two parts on the housing. The first suspension portion and the second suspension portion are connected by other components. For example, the housing also includes a connector for connecting the first suspension portion and the second suspension portion. The connector is flexible so that the first suspension portion is located behind the auricle and the second suspension portion is located in front of the auricle. For example, as shown in FIG1(c), the wearable device is an ear clip-on earphone, and the first suspension portion and the second suspension portion are two parts on the housing, with the portion located behind the auricle being the first suspension portion 110 and the portion located in front of the auricle being the second suspension portion 120.

[0062] A first pickup hole is provided on the wall of the first suspension part, and the first pickup hole is located on the side away from the user. A first microphone is located inside the first suspension part, and the first microphone is used to pick up the sound signal at the first pickup hole.

[0063] A second pickup hole is provided on the wall of the second suspension part, and the second pickup hole is located on the side away from the user. The second microphone is located inside the second suspension part and is used to pick up the sound signal at the second pickup hole.

[0064] The wall surface refers to the surface of the casing. A first sound pickup hole is opened on the wall surface of the first suspension part, and a second sound pickup hole is opened on the wall surface of the second suspension part, so that the sound of the external environment can pass through the sound pickup hole and the wearable device can pick up the sound signal of the external environment.

[0065] The side facing away from the user refers to the side of the housing that is not in contact with the user. Understandably, the first microphone hole is not in contact with the user so that the sound signal can pass through it, and the first microphone picks up the sound signal from the first microphone hole, preventing the first microphone hole from being blocked by contact with the user. Similarly, the second microphone hole is not in contact with the user so that the sound signal can pass through it, and the second microphone picks up the sound signal from the second microphone hole, preventing the second microphone hole from being blocked by contact with the user.

[0066] Since the first suspension part is used to hang behind the auricle, a first sound pickup hole is opened on the wall of the first suspension part so that the wearable device can pick up the sound behind the auricle.

[0067] Since the second suspension part is used to hang in front of the auricle, a second sound pickup hole is opened on the wall of the second suspension part so that the wearable device can pick up the sound in front of the auricle.

[0068] The opening direction of the microphone hole forms an angle with the ear projection plane. Different wearing methods of wearable devices may cause the opening direction of the microphone hole to have different angles with the ear projection plane. The microphone hole includes a first microphone hole and a second microphone hole. The opening direction of the first microphone hole forms an angle with the ear projection plane, while the opening direction of the second microphone hole has a different angle with the ear projection plane.

[0069] As shown in Figure 2, this application provides a schematic diagram of the location of a microphone hole. Taking an ear-hook earphone as an example, as shown in Figure 2(a), the wearable device 200 includes a suspension part 210 and a suspension part 220. A microphone hole 1 is provided on the wall of the suspension part 210, and the opening direction of the microphone hole 1 is located on the side away from the user. The opening direction of the microphone hole 1 forms an angle with the ear projection plane. A microphone hole 2 is provided on the wall of the suspension part 220. The opening direction of the microphone hole 2 is located on the side away from the user. The opening direction of the microphone hole 2 is approximately perpendicular to the ear projection plane. Microphone 1 picks up the sound signal at the microphone hole 1, and microphone 2 picks up the sound signal at the microphone hole 2.

[0070] In some embodiments, the first pickup hole is located at the end of the first suspension portion. The end is the portion of the first suspension portion away from the second pickup hole. The opening direction of the first pickup hole is towards the user's throat, so that the first microphone can pick up the user's voice signal. As shown in FIG2(b), the pickup hole 1 is located at the end of the suspension portion 210, and the opening direction of the pickup hole 1 is towards the user's throat. The opening direction of the pickup hole 2 is approximately perpendicular to the ear projection plane.

[0071] Optionally, in a user call scenario, both the first and second microphones pick up the user's voice signal, improve the signal-to-noise ratio of the speech signal, and apply the picked-up sound signal to call and noise reduction to improve call quality.

[0072] The first vibration sensor is located inside the first suspension section, and is positioned close to the user, meaning it is in contact with the housing. The housing at the location of the first vibration sensor is in contact with the user. This allows the first vibration sensor to pick up vibration signals from the housing when the user speaks or when the wearable device plays audio signals.

[0073] In some embodiments, as shown in FIG3, a schematic diagram of the location of a vibration sensor provided in this application is presented. The difference from FIG2 is that, as shown in FIG3(a), the wearable device 200 includes a vibration sensor 230, which is located within the suspension portion 210. The vibration sensor 230 is attached to the side of the suspension portion 210 closest to the user. FIG3 shows both sides of the wearable device 200, namely the side away from the user and the side closest to the user. The side closest to the user refers to the side of the housing that contacts the user. The vibration sensor 230 is used to pick up vibration signals from the housing.

[0074] As shown in Figure 3(b), the possible location area of ​​the vibration sensor 230 when the wearable device 200 is suspended on the auricle.

[0075] In another possible implementation, the wearable device also includes a third microphone and a third pickup hole.

[0076] A third microphone is provided on the wall of the first suspension part, and the third microphone is located on the side away from the user. The third microphone is located inside the first suspension part and is used to pick up the sound signal at the third microphone.

[0077] Understandably, both the first and third pickup holes are located on the wall of the first suspension part. The difference between the third and first pickup holes is that the first pickup hole is closer to the end of the first suspension part, while the third pickup hole is farther away from the end of the first suspension part.

[0078] Since the first suspension part is used to suspend behind the auricle, the first and third sound pickup holes are opened on the wall of the first suspension part to facilitate the pickup of sound behind the auricle. The first and third sound pickup holes form an array to reduce interference from external noise and airflow.

[0079] As an example, Figure 4 shows another schematic diagram of the location of the microphone hole provided in this application. As shown in Figure 4(a), the difference from that shown in Figure 3 is that a microphone hole 3 is provided on the wall surface of the suspension part 210. The microphone 3 picks up the sound signal at the microphone hole 3.

[0080] In some embodiments, the third microphone hole is closer to the user than the second microphone hole. The opening direction of the third microphone hole forms an angle with the ear projection plane; different wearing methods of the wearable device may cause different angles between the opening direction of the microphone hole and the ear projection plane. When the opening direction of the second microphone hole is approximately perpendicular to the ear projection plane, the opening direction of the third microphone hole is approximately parallel to the ear projection plane. The third microphone hole is further away from the auricle and closer to the user's head, reducing the interference of the auricle on the sound signal when the third microphone picks up the sound signal from the third microphone hole.

[0081] For example, as shown in Figure 4(b), the opening direction of the pickup hole 2 is approximately perpendicular to the ear projection plane, while the opening direction of the pickup hole 3 is closer to the user's head.

[0082] In some embodiments, the first pickup hole is located at the end of the first suspension portion, and the opening direction of the first pickup hole faces the user's throat. The first pickup hole and the third pickup hole form an array facing the throat. When the user speaks, the first pickup hole and the third pickup hole are used to pick up the user's voice signal, improve the signal-to-noise ratio of the speech signal, apply the picked-up sound signal to the call and noise reduction, and improve the call quality.

[0083] For example, as shown in Figure 4(c), the pickup hole 1 is located at the end of the suspension part 210, and the opening direction of the pickup hole 1 is towards the user's throat. The opening direction of the pickup hole 3 is closer to the user's head. The pickup holes 1 and 3 form an array facing the throat, and when the user speaks, the pickup holes 1 and 3 pick up the user's voice signal.

[0084] In another possible implementation, the wearable device also includes a fourth microphone and a fourth pickup hole.

[0085] A fourth microphone hole is provided on the wall of the second suspension part, and the fourth microphone hole is located on the side away from the user. The fourth microphone is located inside the second suspension part and is used to pick up the sound signal at the fourth microphone hole.

[0086] Understandably, both the second and fourth pickup holes are located on the wall of the second suspension part. The difference between the fourth and second pickup holes is that the fourth pickup hole is farther away from the first suspension part than the second pickup hole.

[0087] Since the second suspension part is used to suspend in front of the auricle, the second and fourth sound pickup holes are opened on the wall of the second suspension part to facilitate the pickup of sound in front of the auricle. The second and fourth sound pickup holes form an array to pick up ambient sound signals and effectively identify sound signals in front of the auricle.

[0088] As an example, Figure 5 shows another schematic diagram of the location of the microphone hole provided in this application. As shown in Figure 5(a), the difference from that shown in Figure 4 is that a microphone hole 4 is provided on the wall surface of the suspension part 220. The microphone 4 picks up the sound signal at the microphone hole 4.

[0089] The fourth microphone hole has an angle between its opening direction and the ear projection plane. Different wearing methods of wearable devices may cause the angle between the microphone hole's opening direction and the ear projection plane to be different. The four microphone holes have different angles between their opening directions and the ear projection plane.

[0090] In some embodiments, the fourth microphone hole is closer to the user's head than the second microphone hole. When the opening direction of the second microphone hole is approximately perpendicular to the ear projection plane, the opening direction of the fourth microphone hole is approximately parallel to the user's eye direction, that is, the opening direction of the fourth microphone hole is approximately parallel to the direction the user is looking forward.

[0091] For example, as shown in Figure 5(b), the opening direction of the pickup hole 2 is approximately perpendicular to the ear projection plane, and the opening direction of the pickup hole 4 is approximately parallel to the user's eye direction.

[0092] In another possible implementation, the wearable device also includes a bone conduction oscillator and a second vibration sensor.

[0093] The bone conduction oscillator and the second vibration sensor are located within the second suspension section. The second vibration sensor is used to pick up the vibration signal from the bone conduction oscillator. The housing at the location of the second vibration sensor is not in contact with the user.

[0094] Optionally, the second vibration sensor is in contact with the bone conduction oscillator. Alternatively, the second vibration sensor is in contact with the housing adjacent to the bone conduction oscillator. Or, the second vibration sensor is in contact with a component connecting the bone conduction oscillator and the housing.

[0095] As an example, Figure 6 shows a schematic diagram of the location of another vibration sensor provided in this application. The difference from Figure 5 is that the wearable device 200 includes a vibration sensor 240 and a bone conduction transducer 250, both located within the suspension portion 220. The vibration sensor 240 is used to pick up vibration signals from the bone conduction transducer 250. The suspension portion 220 where the vibration sensor 240 is located is not close to the user.

[0096] In another possible implementation, the wearable device also includes an air-conducting speaker, a sound outlet, and a fifth microphone.

[0097] The sound outlet is located on the wall of the second suspension part, and the sound outlet is located on the side closer to the user.

[0098] The air-conducting loudspeaker is located inside the second suspension section, and the sound outlet is used to output the sound signal of the air-conducting loudspeaker.

[0099] A sound hole generally refers to an opening used for sound output. The main function of a sound hole is to allow sound to travel from inside the speaker to the external environment. When the diaphragm inside the air-conducting speaker vibrates, it pushes the surrounding air molecules to vibrate, thereby generating sound waves. These sound waves propagate into the air through the sound hole, and the user hears the sound.

[0100] The fifth microphone is located inside the second suspension section and is used to pick up the acoustic signal from the air-conducting speaker.

[0101] As an example, Figure 7 shows a schematic diagram of the location of a microphone provided in this application. The difference from Figure 6 is that the wearable device 200 includes a microphone 260, an air-conducting speaker 270, and a sound outlet 280. The microphone 260 and the air-conducting speaker 270 are located within the suspension portion 220. The sound outlet 280 is used to output the acoustic signal from the air-conducting speaker 270. A vibration sensor 260 is used to pick up the acoustic signal from the air-conducting speaker 270. The suspension portion 220, where the vibration sensor 260 is located, is close to the user.

[0102] In some embodiments, in a wearable device playback scenario, the fifth microphone and the second vibration sensor pick up the sound played by the device to achieve effects such as noise reduction, sound leakage prevention, feedback suppression, and playback calibration.

[0103] A directional microphone is a microphone whose pickup range is directional. A directional microphone selectively receives sound from a certain direction while suppressing sound from other directions, as needed.

[0104] In another possible implementation, the first microphone is a directional microphone. The fourth microphone is also a directional microphone.

[0105] In some embodiments, the first pickup hole is located at the end of the first suspension portion. The opening direction of the first pickup hole is towards the user's throat. When the first microphone is a directional microphone, the first microphone is pointed towards the user's throat, and the first microphone picks up the sound signal from the user's throat.

[0106] The fourth microphone hole is located in the second suspension section. The opening of the fourth microphone hole faces the user's eyes. When the fourth microphone is a directional microphone, it faces the user's eyes and picks up the sound signal from the user's throat.

[0107] By using directional microphones, the microphone array can effectively pick up sound signals and determine the direction of the sound signals.

[0108] As an example, Figure 8 shows a schematic diagram of the location of another microphone provided in this application. The opening direction of the pickup hole 1 is towards the user's throat. The opening direction of the pickup hole 2 is approximately perpendicular to the ear projection plane, and the opening direction of the pickup hole 4 is approximately parallel to the user's eye direction.

[0109] The combined use of multiple microphones and multiple vibration sensors effectively improves the signal-to-noise ratio of sound signals, enhances algorithm usability, and expands the application scenarios of wearable devices.

[0110] In some embodiments, the first to fourth microphones described in this application form a microphone array to pick up speech signals, improve the quality of the speech signals, and reduce the influence of external noise. For example, as shown in Figure 9, in a quiet scene where the user speaks softly, an array of multiple microphones pointed towards the throat can improve the signal-to-noise ratio, enhance the speech signal, and achieve a clear call.

[0111] Optionally, a first vibration sensor located close to the user can receive vibration signals to improve the signal-to-noise ratio of the voice signal. It can also be used to pick up user control signals, such as touch vibrations and sibilance.

[0112] In some embodiments, the first to fourth microphones described in this application form a microphone array for picking up ambient sound signals. To determine the direction of the sound signals, they can also pick up or suppress sound signals from specific directions. The first and third microphones are located behind the auricle, effectively identifying sound signals behind the user. For example, in outdoor scenarios where the user is not making a call, the microphones adaptively transmit sound signals such as human voices and vehicle sounds behind the auricle for identification and alerting the user. Furthermore, in high-noise scenarios, while the user is making a call, the microphones assist in picking up ambient sound signals, further reducing noise in the speech signal, and improving its quality.

[0113] In some embodiments, an array of binaural microphones can be used to pick up external sound signals and obtain more accurate sound signal direction information. For example, user head width information can be obtained through inertial measurement unit (IMU) information, and spatial audio acquisition can be achieved through a binaural array via algorithm adaptation.

[0114] In some embodiments, the wearable device includes two earpieces, namely a left earpiece and a right earpiece. Each earpiece includes a pickup hole, a microphone, and a vibration sensor as described in the above embodiments. The microphone and vibration sensor form an array to pick up the user's voice signal, which, after processing by an algorithm, can obtain voice information with a higher signal-to-noise ratio.

[0115] As shown in Figure 10, this application provides a schematic diagram of a binaural array. The binaural array, composed of multiple microphones and vibration sensors on a wearable device, is used in conjunction with the device to pick up ambient sound signals, voice signals, and sound signals played by the device, effectively identifying the direction of external sound sources. Further processing of the sound signals can achieve functions such as noise suppression, directional sound source enhancement, sound transmission, and specific directional sound alerts.

[0116] As shown in Figure 11, this is a schematic diagram of a sound signal pickup scenario provided by this application for a wearable device. In the listening scenario, the sound signal conducted through the air of the air-conducting speaker 270 is picked up by the fifth microphone, and the vibration signal of the bone conduction vibrator 250 is picked up by the second vibration sensor. The picked-up signals can be used as feedback signals to adaptively adjust algorithms such as active noise reduction and sound leakage prevention.

[0117] In some embodiments, as shown in FIG12, the wearable device described in this application includes any one of headphones, glasses, or a head-mounted device. The head-mounted device includes VR devices, AR devices, MR devices, or ER devices, etc. For example, as shown in FIG12(a), the wearable device is an ear-hook headphone. As shown in FIG12(b), the wearable device is glasses. As shown in FIG12(c), the wearable device is a clip-on headphone.

[0118] This application primarily relates to sound pickup scenarios using microphones and vibration sensors. The picked-up sound signals mainly include user voice signals, ambient sound signals, air conduction sound signals played by the device, and vibration signals. Through sensor layout design, these signals are applied to various audio algorithms, expanding and improving algorithm capabilities and enhancing the user's audio experience.

[0119] As shown in Figure 13, the sound signals picked up by the microphone and vibration sensor included in the wearable device are used for active noise reduction, speech enhancement, scene judgment, hearing aids, sound source localization, spatial audio, and sound leakage prevention.

[0120] In some embodiments, the wearable device includes a processor for running audio processing algorithms such as active noise reduction, speech enhancement, scene judgment, hearing aids, sound source localization, spatial audio, and sound leakage prevention. The processor uses the sound signals picked up by the wearable device to achieve effects such as active noise reduction, speech enhancement, scene judgment, hearing aids, sound source localization, spatial audio, and sound leakage prevention.

[0121] The device status includes call status, sound playback status, and radio reception status. Uplink signals refer to signals received from outside the device, such as signals input to the device's processor. Downlink signals refer to signals played by the device, such as signals output from the device's processor.

[0122] Optionally, the sensor position of the wearable device can vary depending on the sensor type, function, human body shape, and sound generation principle. It can also take into account changes in the external environment, such as strong airflow or water exposure, to adjust the sensor position and improve the sound pickup capability of the wearable device.

[0123] Optionally, the microphone array pointing towards the larynx can be applied not only to wearable devices but also to devices with voice pickup capabilities such as telephones and walkie-talkies. During a call, the microphone array beam on the device is identified and adaptively adjusted to point the microphone towards the larynx, thereby improving the quality of the picked-up voice signal.

[0124] The use of directional microphones can also be applied to devices with video recording and audio pickup capabilities, such as cameras and smartphones. By integrating directional microphones, the device's ability to pick up sound in a directional manner can be improved.

[0125] The above embodiments illustrate the positions of the microphone and vibration sensor in the wearable device.

[0126] In some embodiments, the wearable device further includes a processor and a memory for processing the acoustic signals picked up by the wearable device.

[0127] Figure 14 is a schematic diagram of a wearable device provided in this application. As shown in Figure 14, the wearable device 1400 includes a processor 1410, a microphone 1420, a storage medium 1430, an interface circuit 1440, and a vibration sensor 1450. The processor 1410, microphone 1420, storage medium 1430, interface circuit 1440, and vibration sensor 1450 are connected together.

[0128] In this embodiment, the wearable device 1400 includes multiple microphones. The multiple microphones are oriented differently, as described in the embodiments above.

[0129] In some embodiments, processor 1410 is a CPU, but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor is a microprocessor or any conventional processor.

[0130] In this application, the processor 1410 is used to process the sound signals picked up by the wearable device to achieve effects such as sound source localization, spatial audio, sound leakage prevention, active noise reduction, and voice enhancement.

[0131] As an example, wearable device 1400 includes multiple processors. The processor is a multi-core (multi-CPU) processor, such as CPU0 and CPU1 in Figure 14. Here, "processor" can refer to one or more devices, circuits, and / or computing units used to process data (e.g., computer program instructions).

[0132] As an example, the wearable device 1400 also includes an NPU, a DPU, or one or more integrated circuits for controlling the execution of programs according to the present application. For example, the wearable device 1400 also calculates the gain required to eliminate wind noise from signals collected by multiple microphones based on an artificial neural network.

[0133] It is worth noting that Figure 14 only shows an example of a wearable device 1400 including one processor 1410. Here, the processor 1410 is used to indicate a type of device or equipment. In specific embodiments, the number of each type of device or equipment can be determined according to business needs.

[0134] Interface circuit 1440 is used to enable communication between wearable device 1400 and external devices or components.

[0135] Storage medium 1430 can be used to store relevant information during signal processing, such as gain, signal energy, amplitude, etc., for example, a disk, such as a solid-state drive.

[0136] Optionally, the wearable device may also include a Wi-Fi module, a Bluetooth module, an input unit, a display unit, sensors, etc., which will not be described in detail in the embodiments of this application. Those skilled in the art will understand that the wearable device structure shown in FIG14 does not constitute a limitation on the device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0137] The input unit can be used to receive input numerical or character information, and to generate key signal inputs related to user settings and function control of the device. The input unit may include a touchscreen and other input devices. The touchscreen can collect user touch operations on or near it and drive corresponding connected devices according to a pre-set program. For example, touch operations may include operations performed by the user using their fingers, stylus, or any suitable object or accessory on or near the touchscreen. Optionally, other input devices may include, but are not limited to, one or more of a physical keyboard, function keys, trackball, mouse, joystick, etc., such as volume control buttons, power switch buttons, etc.

[0138] The display unit can be used to display information input by the user or information provided to the user, as well as various menus of the device. In one example, the display unit may include a display screen, which may be configured as a liquid crystal display (LCD), organic light-emitting diode (OLED), or similar type. Furthermore, a touchscreen may cover the display screen. When the touchscreen detects a touch operation on or near it, it transmits the information to a processor to determine the type of touch event. The processor then provides corresponding visual output on the display screen based on the type of touch event. Although the touchscreen and display screen are implemented as two separate components to achieve the input and output functions of the device, in some embodiments, the touchscreen and display screen can be integrated to achieve the input and output functions of the device.

[0139] This application also provides a computer program product that, when executed by a computer, can implement the functions of any of the above method embodiments.

[0140] This application also provides a computer program that, when executed by a computer, can implement the functions of any of the above method embodiments.

[0141] This application also provides a computer-readable storage medium. All or part of the processes in the above method embodiments can be implemented by a computer program instructing related hardware. This program can be stored in the computer-readable storage medium, and when executed, it can include the processes of the above method embodiments. The computer-readable storage medium can be an internal storage unit of the terminal (including a data sending end and / or a data receiving end) of any of the foregoing embodiments, such as the terminal's hard disk or memory. The computer-readable storage medium can also be an external storage device of the terminal, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the terminal. Further, the computer-readable storage medium can include both the terminal's internal storage unit and external storage devices. The computer-readable storage medium is used to store the computer program and other programs and data required by the terminal. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0142] The terms "first" and "second," etc., used in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. "First" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first" and "second" may explicitly or implicitly include one or more of that feature. In the description of this embodiment, unless otherwise stated, "a plurality of" means two or more.

[0143] Furthermore, the terms “comprising” and “having”, and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.

[0144] It should be understood that in this application, "at least one (item)" means one or more. "More than one" means two or more. "At least two (items)" means two or three or more. "And / or" is used to describe the relationship between related objects, indicating that there can be three relationships. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the related objects before and after are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple. Both "...when" and "if" indicate that a corresponding action will be taken under certain objective circumstances. They are not time limits, nor do they require a judgment action to be taken when the action is taken, nor do they imply any other limitations.

[0145] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner to facilitate understanding.

[0146] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0147] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0148] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0149] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0150] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of this application embodiment, or all or part of the technical solution, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

Claims

1. A wearable device, characterized in that, include: The housing includes a first suspension part and a second suspension part connected to the first suspension part. The first suspension part is used to suspend behind the auricle, and the second suspension part is used to suspend in front of the auricle. A first microphone and a first pickup hole, wherein the first microphone is used to pick up the sound signal at the first pickup hole, the first microphone is located inside the first suspension part, the first pickup hole is opened on the wall of the first suspension part, and the first pickup hole is located on the side away from the user. The second microphone and the second pickup hole are used to pick up the sound signal at the second pickup hole. The second microphone is located inside the second suspension part, and the second pickup hole is opened on the wall of the second suspension part and is located on the side away from the user. The first vibration sensor is located inside the first suspension part, and the first vibration sensor is located on the side closer to the user.

2. The wearable device according to claim 1, characterized in that, The first pickup hole is located at the end of the first suspension part, and the end is the end of the first suspension part that is away from the second pickup hole.

3. The wearable device according to claim 1 or 2, characterized in that, The wearable device also includes: A third microphone and a third pickup hole, wherein the third microphone is used to pick up the sound signal at the third pickup hole, the third microphone is located inside the first suspension part, the third pickup hole is opened on the wall of the first suspension part, and the third pickup hole is located on the side away from the user; the first pickup hole is closer to the end of the first suspension part than the third pickup hole.

4. The wearable device according to any one of claims 1-3, characterized in that, The wearable device also includes: A fourth microphone and a fourth pickup hole are provided. The fourth microphone is used to pick up the sound signal at the fourth pickup hole. The fourth microphone is located inside the second suspension part. The fourth pickup hole is opened on the wall of the second suspension part and is located on the side away from the user. The fourth pickup hole is farther away from the first suspension part than the second pickup hole.

5. The wearable device according to any one of claims 1-4, characterized in that, The opening direction of the pickup hole forms an angle with the ear projection plane, and the pickup hole includes a first pickup hole, a second pickup hole, a third pickup hole, and a fourth pickup hole.

6. The wearable device according to any one of claims 1-5, characterized in that, The first microphone is a directional microphone, and the opening of the first pickup hole faces the user's throat.

7. The wearable device according to any one of claims 1-6, characterized in that, The fourth microphone is a directional microphone, and the opening of the fourth pickup hole faces the direction in front of the user's eyes.

8. The wearable device according to any one of claims 1-7, characterized in that, The wearable device also includes: A bone conduction oscillator and a second vibration sensor are provided, the second vibration sensor being used to pick up the vibration signal of the bone conduction oscillator, and the bone conduction oscillator and the second vibration sensor being located within the second suspension section.

9. The wearable device according to any one of claims 1-8, characterized in that, The wearable device also includes: The device includes an air-conducting speaker, a sound outlet, and a fifth microphone. The fifth microphone is used to pick up the sound signal from the air-conducting speaker. The air-conducting speaker and the fifth microphone are located inside the second suspension part. The sound outlet is opened on the wall of the second suspension part and is located on the side closer to the user. The sound outlet is used to output the sound signal from the air-conducting speaker.

10. The wearable device according to any one of claims 1-9, characterized in that, The wearable device includes any one of headphones, glasses, or head-mounted devices.

Images