Wearable device, audio playing method of wearable device, and medium

Abstract

The present disclosure relates to a wearable device, an audio playing method of the wearable device and a medium, and relates to the technical field of wearable devices. The wearable device comprises a wearable body, a posture sensor, a speaker array and a processor. The wearable body is used to carry the posture sensor, the speaker array and the processor. The posture sensor is used to collect posture data of the wearable device. The speaker array is used to play audio signals. The processor is used to divide the audio signals to be played into at least two adjacent frequency band sub-audio signals. For each sub-audio signal, according to the geometric parameters of the speaker array, the reference offset, the current posture data collected by the posture sensor, the preset at least one sound leakage direction and the frequency band corresponding to the sub-audio signal, the current sound production parameter of each speaker in the speaker array that satisfies the constraint condition is determined. According to each sub-audio signal and the current sound production parameter of each speaker corresponding to each sub-audio signal, the sound production of the speaker array is controlled.

Description

Technical Field

[0001] This disclosure relates to the field of wearable device technology, and more specifically, to a wearable device, an audio playback method for a wearable device, and a computer-readable storage medium. Background Technology

[0002] With economic and technological development, wearable devices such as smart pendants and smart brooches are becoming increasingly popular. Existing wearable devices typically use a traditional single speaker for audio output. However, wearable devices using a single speaker suffer from serious sound leakage problems, which can lead to privacy breaches when outputting audio. Summary of the Invention

[0003] One objective of this disclosure is to provide a new technical solution for wearable devices that enables directional sound generation while preventing audio signal leakage.

[0004] According to a first aspect of this disclosure, a wearable device is provided, comprising: a wearable body, a posture sensor, a speaker array, and a processor, wherein: The wearable body is used to carry the posture sensor, the speaker array, and the processor. The posture sensor is used to collect posture data of the wearable device, the speaker array is used to play audio signals, and the processor is connected to the posture sensor and each speaker in the speaker array. The processor is used to: Divide the audio signal to be played into sub-audio signals of at least two adjacent frequency bands; For each sub-audio signal, based on the geometric parameters of the speaker array, the reference offset, the current attitude data collected by the attitude sensor, at least one preset sound leakage direction, and the frequency band corresponding to the sub-audio signal, the current sound output parameters of each speaker in the speaker array that satisfy the constraints are determined. The speaker array is controlled to emit sound based on each of the sub-audio signals and the current sound parameters of each speaker corresponding to each sub-audio signal; Wherein, the reference offset is the relative offset between the wearable device and the target listening position when the wearable device is in the standard wearing state, the constraint condition is that the total sound pressure energy in the leakage direction is minimized under the constraint that the sound pressure gain in the direction of the target listening position is 1, and the current sound parameters include the current amplitude and the current phase.

[0005] Optionally, the loudspeaker is a miniature dynamic loudspeaker, and / or the loudspeakers are arranged in a parallel array or a ring array.

[0006] Optionally, the wearable device further includes: An image sensor, connected to the processor, is used to acquire environmental images; The processor is further configured to: identify the environmental image and generate an audio signal to be played for the environmental image; The wearable body also serves to support the image sensor.

[0007] Optionally, the wearable device further includes: A microphone module, connected to the processor, is used to collect ambient audio signals; The processor is further configured to identify the ambient audio signal and generate an audio signal to be played for the ambient audio signal; The wearable body also serves to support the microphone module.

[0008] Optionally, the wearable body includes a wearing structure located inside or on the back of the wearable body, the wearing structure being used to secure the wearable body to the wearer.

[0009] Optionally, the wearing structure is a clamping member; Alternatively, the wearing structure may be a first magnetic element, and the wearable device may also include a second magnetic element, with the wearing body being detachably connected to the second magnetic element via the first magnetic element.

[0010] According to a second aspect of this disclosure, an audio playback method for a wearable device is also provided, the method being applied to a wearable device as described in any one of the first aspects, the method comprising: Divide the audio signal to be played into sub-audio signals of at least two adjacent frequency bands; For each sub-audio signal, based on the geometric parameters of the speaker array in the wearable device, the reference offset, the current posture data collected by the posture sensor in the wearable device, at least one preset sound leakage direction, and the frequency band corresponding to the sub-audio signal, the current sound output parameters of each speaker in the speaker array that satisfy the constraints are determined. The speaker array is controlled to emit sound based on each of the sub-audio signals and the current sound parameters of each speaker corresponding to each sub-audio signal; Wherein, the reference offset is the relative offset between the wearable device and the target listening position when the wearable device is in the standard wearing state, the constraint condition is that the total sound pressure energy in the leakage direction is minimized under the constraint that the sound pressure gain in the direction of the target listening position is 1, and the current sound parameters include the current amplitude and the current phase.

[0011] Optionally, determining the current sound output parameters of each speaker in the speaker array that satisfy the constraints based on the geometric parameters of the speaker array in the wearable device, the reference offset, the current posture data collected by the posture sensor in the wearable device, at least one preset sound leakage direction, and the frequency band corresponding to the sub-audio signal includes: The phase pre-compensation amount of the speaker array is determined based on the geometric parameters of the speaker array in the wearable device, the reference offset, the current attitude data collected by the attitude sensor in the wearable device, and the frequency band corresponding to the sub-audio signal. Based on the frequency band, determine the leakage direction constraint weight coefficient corresponding to the frequency band. The leakage direction constraint weight coefficient corresponding to a higher frequency band is smaller than that corresponding to a lower frequency band. The audio spatial propagation characteristics are determined based on at least one preset sound leakage direction and the sound leakage direction constraint weight coefficient corresponding to the frequency band. Based on the phase pre-compensation amount and the audio spatial propagation characteristics, determine the current sound parameters of each loudspeaker in the loudspeaker array that satisfy the constraints. Wherein, the phase pre-compensation amount is the phase compensation amount required to be applied when the audio output of each speaker in the speaker array simultaneously reaches the target listening position; the audio spatial propagation characteristics are used to characterize the comprehensive quantization value of the relative phase relationship distribution of the audio radiated by each speaker in each leakage direction; the constraint condition is that the total sound pressure energy in the leakage direction is minimized under the constraint that the sound pressure gain in the direction of the target listening position is 1; and the current sound parameters include the current amplitude and the current phase.

[0012] Optionally, before controlling the speaker array to emit sound based on each of the sub-audio signals and the current sound parameters of each speaker corresponding to each sub-audio signal, the method further includes: A smooth transition process is performed on the current sound parameters of each speaker based on the historical sound parameters of each speaker, and the current sound parameters of each speaker are updated.

[0013] Optionally, before dividing the audio signal to be played into sub-audio signals of at least two adjacent frequency bands, the method further includes: Acquire environmental images captured by the image sensor in the wearable device; The environmental image is identified, and an audio signal to be played is generated for the environmental image.

[0014] Optionally, before dividing the audio signal to be played into sub-audio signals of at least two adjacent frequency bands, the method further includes: Acquire ambient audio signals collected by the microphone module in the wearable device; Identify the ambient audio signal and generate an audio signal to be played for the ambient audio signal.

[0015] According to a third aspect of the present disclosure, a computer-readable storage medium is also provided, on which a computer program is stored, the computer program implementing the method according to any one of the first aspects when executed by a processor.

[0016] Based on the wearable device disclosed herein, even if the wearable device is oscillating, it can still ensure that the speaker array's sound direction is focused on the target listening position so that the wearer can hear clear sound at the target listening position, while also preventing audio signal leakage.

[0017] Other features and advantages of the embodiments of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the disclosure.

[0019] Figure 1 These are schematic diagrams of the structure of a wearable device according to some embodiments; Figure 2 These are schematic diagrams of the structure of a wearable device according to other embodiments; Figure 3 This is a flowchart illustrating an audio playback method for a wearable device according to some embodiments; Figure 4 This is a structural schematic diagram of an audio playback device for a wearable device according to some embodiments. Detailed Implementation

[0020] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present disclosure.

[0021] The following description of at least one exemplary embodiment is merely illustrative and is not intended to limit the scope of this disclosure or its application or use.

[0022] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0023] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0024] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0025] This disclosure provides a wearable device, exemplified by a smart pendant, smart brooch, etc. It should be noted that this disclosure does not limit the specific form of the wearable device. Figure 1 As shown, the wearable device provided in this disclosure includes: a wearable body 101, a posture sensor 102, a speaker array 103, and a processor 104, wherein: The wearable body 101 houses the attitude sensor 102, the speaker array 103, and the processor 104. The attitude sensor 102 collects attitude data of the wearable device. The speaker array 103 plays audio signals. The processor 104 is connected to each speaker in the attitude sensor 102 and the speaker array 103, respectively.

[0026] In this embodiment of the disclosure, the wearable body 101 may be made of a lightweight material, such as ultralight plastic or carbon fiber. The wearable body 101 serves as a carrier for the attitude sensor 102, the speaker array 103, and the processor 104, providing mounting space for the attitude sensor 102, the speaker array 103, and the processor 104.

[0027] In some embodiments, the wearable body 101 includes a wearing structure located inside or on the back of the wearable body, the wearing structure being used to secure the wearable body 101 to the wearer. In this embodiment, the wearable body 101 includes a wearing structure for securing the wearable device to the wearer's clothing, which reduces the swaying frequency of the wearable device caused by the wearer's movement, thereby providing a basis for stable sound output from the speaker array on the wearable device.

[0028] In some embodiments, the wearing structure is a clamp that, when clamped to the wearer's clothing, enables the wearing of the device.

[0029] In other embodiments, the wearing structure is a first magnetic attachment. Based on this, the wearable device also includes a second magnetic attachment, and the wearing body is detachably connected to the first magnetic attachment via the second magnetic attachment. For example, when wearing the device, the second magnetic attachment is placed inside clothing, and the wearing body is placed outside the clothing, or the first magnetic attachment is fixed inside clothing, and the first and second magnetic attachments are magnetically attracted together, completing the wearing of the device. Correspondingly, separating the first and second magnetic attachments completes the detachment of the device.

[0030] In this embodiment of the disclosure, the attitude sensor 102 can be an inertial measurement unit (IMU). Based on this, the attitude data includes triaxial acceleration data and triaxial angular velocity data. It is understood that the attitude data collected by the attitude sensor 102 changes when the wearable device swings.

[0031] Furthermore, the speaker array 103 includes multiple speakers. In one example, the speaker array 103 includes 4-8 speakers, and each speaker has a projected area of ​​at least 100 square millimeters and a thickness of 2-4 mm. In some embodiments, at least one speaker in the speaker array 103 is a miniature dynamic speaker. Thus, the speaker array 103 has low power consumption and a small size. In some embodiments, the speakers are arranged in a parallel array or a ring array. In the case of a parallel array, the speaker array 103 has high beam directivity and a long projection distance. In the case of a ring array, the speaker array 103 has spatial symmetry, thus allowing for flexible adjustment of the beam directivity of the speaker array.

[0032] Processor 104 may be one or at least two, which may execute instructions individually or jointly. In some embodiments, processor 104 is a central processing unit (CPU) and a digital signal processor (DSP). In some embodiments, processor 104 is a CPU, a DSP, and an artificial intelligence (AI) processor.

[0033] In this embodiment of the disclosure, the processor 104 is used to perform the following steps S110 to S130.

[0034] Step S110: Divide the audio signal to be played into sub-audio signals of at least two adjacent frequency bands.

[0035] In some embodiments, the audio signal may be a multimedia audio signal, a recorded audio signal, a call audio signal, etc., stored or received from other terminal devices and to be output by the wearable device.

[0036] In other embodiments, the audio signal to be played is an audio signal generated by the wearable device. In one embodiment of this disclosure, such as... Figure 2 As shown, the wearable device also includes an image sensor 105. The image sensor 105 is connected to the processor 104 and is used to acquire environmental images. The wearable body 101 also serves to carry the image sensor 105. Furthermore, the processor 104 is used to perform the following step S1111.

[0037] Step S1111: Identify the environmental image and generate an audio signal to be played for the environmental image.

[0038] In some embodiments, when a wearer is unfamiliar with an object in the environment or wants to learn about an object, they can trigger the wearable device to take a picture. At this time, the image sensor in the wearable device captures an image of the object in the environment. The processor obtains the environmental image from the image sensor and further calls an AI algorithm to recognize the environmental image in order to generate an audio signal to be played for that environmental image. In one example, the processor calls the AI ​​algorithm to recognize the environmental image in the following way: the central processing unit inputs the environmental image into the AI ​​processor, which generates an audio signal to be played for the environmental image and feeds it back to the central processing unit.

[0039] In one embodiment of this disclosure, such as Figure 2 As shown, the wearable device also includes a microphone module 106. This microphone module 106 is connected to the processor 104 and is used to collect ambient audio signals. Furthermore, the wearable body 101 is also used to carry the microphone module 106. In addition, the processor 104 is used to perform the following step S1112.

[0040] Step S1112: Identify ambient audio signals and generate an audio signal to be played that is specific to the ambient audio signals.

[0041] In some embodiments, when a wearer has a question that needs answering, the wearable device can be triggered to collect ambient audio. For example, pressing a specific button on the wearable device or saying the wake word. At this time, the microphone module in the wearable device picks up the audio signal in the environment. The processor obtains the ambient audio signal from the microphone module and further calls the AI ​​algorithm to identify the ambient audio signal in order to generate an audio signal to be played for that ambient audio signal. In one example, the specific way the processor calls the AI ​​algorithm to identify the ambient image is as follows: the central processing unit inputs the ambient image into the AI ​​processor, the AI ​​processor generates an audio signal to be played for the ambient audio signal and feeds it back to the central processing unit.

[0042] After receiving the audio signal to be played, the processor can use a digital bandpass filter to divide the audio signal into sub-audio signals of at least two adjacent frequency bands. Alternatively, other methods can also be used to divide the audio signal into sub-audio signals of at least two adjacent frequency bands. It should be noted that adjacent frequency bands refer to frequency bands with adjacent cutoff frequencies, and at least two adjacent frequency bands must cover the frequency range corresponding to the audio signal.

[0043] Audio signals of different frequency bands have different propagation characteristics. Specifically, low-frequency audio signals have long wavelengths and strong diffraction capabilities, making them difficult to direct and prone to spreading in all directions. Mid-to-high frequency audio signals have shorter wavelengths and better directionality, thus exhibiting strong directivity. Based on this, this disclosure first divides the audio signal to be played into frequency bands through step S110 to obtain sub-audio signals of different frequency bands, and then uses steps S120 and S130 to perform differentiated sound leakage suppression and directional sound transmission for the sub-audio signals of different frequency bands.

[0044] In one example, the processor specifically divides the audio signal into three adjacent frequency bands: a low-frequency sub-audio signal, a mid-frequency sub-audio signal, and a high-frequency sub-audio signal. The low-frequency sub-audio signal corresponds to a frequency band of (200-1000) Hz, the mid-frequency sub-audio signal corresponds to a frequency band of (1000-4000) Hz, and the high-frequency sub-audio signal corresponds to a frequency band of (4000-8000) Hz.

[0045] Step S120: For each sub-audio signal, determine the current sound output parameters of each speaker in the speaker array that meet the constraints based on the geometric parameters of the speaker array, the reference offset, the current attitude data collected by the attitude sensor, at least one preset sound leakage direction, and the frequency band corresponding to the sub-audio signal.

[0046] Among them, the reference offset is the relative offset between the wearable device and the target listening position when the wearable device is in the standard wearing state. The constraint condition is that the total sound pressure energy in the direction of sound leakage is minimized under the constraint that the sound pressure gain in the direction of the target listening position is 1. The current sound parameters include the current amplitude and the current phase.

[0047] In this embodiment of the disclosure, the target hearing position is the wearer's ear position. The geometric parameters of the speaker array include the local spatial position of each speaker in the speaker array in the wearable device coordinate system, and the local spatial position of a reference point in the speaker array in the wearable device coordinate system. The geometric parameters and reference offset of the speaker array are pre-stored in the wearable device.

[0048] In some embodiments, step S120 specifically includes steps S1211 to S1214.

[0049] Step S1211: Determine the phase pre-compensation amount of the speaker array based on the geometric parameters of the speaker array in the wearable device, the reference offset, the current attitude data collected by the attitude sensor in the wearable device, and the frequency band corresponding to the sub-audio signal.

[0050] The phase pre-compensation amount is the amount of phase compensation required when the audio output from each speaker in the speaker array simultaneously reaches the target listening position.

[0051] In some embodiments, step S1211 can be implemented by referring to the target direction guidance vector calculation method in the constrained minimum variance beamforming algorithm. Specifically, step S1211 is implemented by the following steps S1211-1 to S1211-10.

[0052] Step S1211-1: Determine the current attitude change of the wearable device based on the current attitude data and the reference attitude data collected by the attitude sensor when the wearable device is in a standard wearing state.

[0053] Step S1211-2: For each speaker in the speaker array, determine the current spatial position of the speaker in the world coordinate system based on the speaker's local spatial position in the wearable device and the current attitude change.

[0054] Step S1211-3: Based on the current attitude change and reference offset, determine the current relative offset between the wearable device and the target listening position in the world coordinate system.

[0055] Step S1211-4: Determine the current spatial position of the reference point in the world coordinate system based on the local spatial position of the reference point in the loudspeaker array and the current attitude change.

[0056] Step S1211-5: Determine the current spatial position of the target listening position in the world coordinate system based on the current relative offset and the current spatial position of the reference point in the world coordinate system.

[0057] Step S1211-6: Determine the distance between each speaker and the target listening position based on the current spatial position of each speaker in the world coordinate system and the current spatial position of the target listening position in the world coordinate system.

[0058] Step S1211-7: Determine the distance between the reference point and the target listening position based on the current spatial position of the reference point in the world coordinate system and the current spatial position of the target listening position in the world coordinate system.

[0059] Step S1211-8: Determine the sound path difference of each loudspeaker based on the distance between each loudspeaker and the target listening position, and the distance between the reference point and the target listening position.

[0060] Step S1211-9: Determine the wavenumber of the sub-audio signal based on the center frequency of the frequency band corresponding to the sub-audio signal.

[0061] Step S1211-10: Determine the target direction steering vector, which includes the phase pre-compensation amount of each loudspeaker in the loudspeaker array, based on the path difference of each loudspeaker and the wavenumber corresponding to the sub-audio signal.

[0062] Step S1212: Determine the leakage direction constraint weight coefficient corresponding to the frequency band based on the frequency band.

[0063] Among them, the weighting coefficient of the leakage direction constraint is smaller for higher frequency bands compared to the weighting coefficient of the leakage direction constraint for lower frequency bands.

[0064] In one example, when the sub-audio signals are low-frequency, mid-frequency, and high-frequency sub-audio signals respectively, the leakage direction constraint weighting coefficient for the corresponding frequency band of the low-frequency sub-audio signal is 5, the leakage direction constraint weighting coefficient for the corresponding frequency band of the mid-frequency sub-audio signal is 1, and the leakage direction constraint weighting coefficient for the corresponding frequency band of the high-frequency sub-audio signal is 0.1.

[0065] Based on the explanation of the different propagation characteristics of audio signals in different frequency bands in step S110, step S1212 achieves the following: For low-frequency sub-audio signals, the maximum leakage direction constraint weighting coefficient is used to form a deeper acoustic null point in the leakage direction to suppress low-frequency diffusion, overcoming the physical defect that low-frequency sub-audio signal sound waves are difficult to direct due to their longer wavelength and stronger diffraction ability; for mid-frequency audio signals, a moderate leakage direction constraint weighting coefficient is used to achieve a balance between directional sound transmission and leakage suppression, ensuring that the main speech is clearly identifiable and that privacy is not compromised; for high-frequency audio signals, a smaller leakage direction constraint weighting coefficient is used, allowing more degrees of freedom to form a narrow beam, fully utilizing the physical advantages of shorter wavelengths and stronger directivity of high-frequency sound waves to improve the clarity of speech consonants. This provides a foundation for achieving optimal directional sound transmission and leakage suppression effects across the entire frequency band of the audio signal.

[0066] Step S1213: Determine the audio spatial propagation characteristics based on at least one preset sound leakage direction and the sound leakage direction constraint weight coefficient corresponding to the frequency band.

[0067] Among them, the audio spatial propagation characteristics are used to characterize the comprehensive quantitative value of the relative phase relationship distribution of the audio radiated by each loudspeaker in each leakage direction.

[0068] In some embodiments, step S1213 can be implemented by referring to the method for calculating the noise covariance matrix in the constrained minimum variance beamforming algorithm. Specifically, step S1213 is implemented through steps S1213-1 to S1213-3.

[0069] Step S1213-1: Based on the local spatial position of each speaker in the speaker array in the wearable device coordinate system and the direction vector of the sound leakage direction in the local coordinate system, determine the projection distance of each speaker in each sound leakage direction.

[0070] In this embodiment, at least one preset sound leakage direction is defined and stored in the wearable device. In some embodiments, the at least one sound leakage direction includes, but is not limited to, one or more of the following: 30° to the side, 60° to the side, and below the wearable device. The sound leakage direction refers to the spatial direction from which sound is not desired to propagate.

[0071] Step S1213-2: Determine the wavenumber of the sub-audio signal based on the center frequency of the frequency band corresponding to the sub-audio signal.

[0072] Step S1213-3: Determine the leakage direction guide vector based on the projection distance and wave number of each loudspeaker in each leakage direction.

[0073] Step S1213-4: Based on the leakage direction constraint weight coefficient and leakage direction steering vector corresponding to the frequency band, construct the noise covariance matrix as the audio spatial propagation feature.

[0074] Step S1214: Based on the phase pre-compensation amount and audio spatial propagation characteristics, determine the current sound parameters of each loudspeaker in the loudspeaker array that meet the constraints.

[0075] In some embodiments, step S1214 can be implemented by referring to the optimal weight vector solution method in the constrained minimum variance beamforming algorithm. Specifically: based on the phase pre-compensation amount and audio spatial propagation characteristics, that is, based on the target direction steering vector and the noise covariance matrix, the current optimal weight vector of each speaker in the speaker array is solved to satisfy the constraint that the total sound pressure energy in the leakage direction is minimized under the constraint that the sound pressure gain in the direction of the target listening position is 1. This is used as the current sound emission parameter. Here, the sound pressure gain in the direction of the target listening position is 1, which ensures that the sound emission direction of the speaker array is focused on the target listening position so that the wearer can hear clear sound at the target listening position. Minimizing the total sound pressure energy in the leakage direction ensures that the total sound pressure energy in the leakage direction is suppressed to the minimum to avoid audio signal leakage.

[0076] In some embodiments, since the current posture data may change in real time, the processor executes step S120 in real time. In other embodiments, the current posture data changes little in a short period of time, and the current vocal parameters obtained based on the little-changing current posture data also change little, resulting in a small difference in auditory effect. Therefore, in order to reduce the processor's computational overhead, the processor may execute step S120 at set time intervals.

[0077] Step S130: Control the speaker array to emit sound according to each sub-audio signal and the current sound parameters of each speaker corresponding to each sub-audio signal.

[0078] In this embodiment, the specific implementation of step S130 can be as follows: for each frequency band sub-audio signal, the sub-audio signal is weighted according to the current sound output parameters of each loudspeaker corresponding to the sub-audio signal to obtain the weighted signal of each loudspeaker in the frequency band; for each loudspeaker in the loudspeaker array, the weighted signals of the sub-audio signals of the loudspeaker in each frequency band are superimposed to generate the final driving signal of the loudspeaker, and the final driving signals of each loudspeaker are output to the corresponding loudspeakers to control the loudspeaker array to produce sound.

[0079] In some embodiments, prior to step S130 described above, the processor is further configured to perform step S1311 described below.

[0080] Step S1311: Perform smooth transition processing on the current sound parameters of each speaker based on the historical sound parameters of each speaker, and update the current sound parameters of each speaker.

[0081] In this process, the historical sound parameters of each speaker are the current sound parameters obtained by the processor in the previous execution of step S120. Through step S1311, the problem of reduced auditory experience caused by the sudden change in the current sound parameters before the update compared to the historical sound parameters can be avoided.

[0082] Based on the above, this disclosure provides a wearable device, which includes a wearable body, a posture sensor, a speaker array, and a processor. The wearable body carries the posture sensor, speaker array, and processor. The posture sensor collects posture data of the wearable device. The speaker array plays audio signals. The processor is connected to the posture sensor and each speaker in the speaker array. The processor is used to: divide the audio signal to be played into at least two adjacent frequency bands as sub-audio signals; for each sub-audio signal, determine the current sound output parameters of each speaker in the speaker array that satisfy the constraints based on the geometric parameters of the speaker array, the reference offset, the current posture data collected by the posture sensor, at least one preset sound leakage direction, and the frequency band corresponding to the sub-audio signal; and control the speaker array to emit sound based on each sub-audio signal and the current sound output parameters of each speaker corresponding to each sub-audio signal. The reference offset is the relative offset between the wearable device and the target listening position when the wearable device is in a standard wearing state. The constraint condition is that the total sound pressure energy in the sound leakage direction is minimized under the constraint that the sound pressure gain in the direction of the target listening position is 1. The current sound output parameters include the current amplitude and the current phase. Based on the wearable device disclosed herein, even if the wearable device is oscillating, it can still ensure that the speaker array's sound direction is focused on the target listening position so that the wearer can hear clear sound at the target listening position, while also preventing audio signal leakage.

[0083] This disclosure also provides an audio playback method for a wearable device, which can be applied to the wearable device provided in any of the above embodiments, such as... Figure 3 As shown, the method includes: Step S310: Divide the audio signal to be played into sub-audio signals of at least two adjacent frequency bands; Step S320: For each sub-audio signal, based on the geometric parameters of the speaker array in the wearable device, the reference offset, the current posture data collected by the posture sensor in the wearable device, at least one preset sound leakage direction, and the frequency band corresponding to the sub-audio signal, determine the current sound output parameters of each speaker in the speaker array that satisfy the constraints. Step S330: Control the speaker array to emit sound according to each sub-audio signal and the current sound emission parameters of each speaker corresponding to each sub-audio signal; Wherein, the reference offset is the relative offset between the wearable device and the target listening position when the wearable device is in the standard wearing state, the constraint condition is that the total sound pressure energy in the leakage direction is minimized under the constraint that the sound pressure gain in the direction of the target listening position is 1, and the current sound parameters include the current amplitude and the current phase.

[0084] In some embodiments, step S320 includes: Step S3211: Determine the phase pre-compensation amount of the speaker array based on the geometric parameters of the speaker array in the wearable device, the reference offset, the current attitude data collected by the attitude sensor in the wearable device, and the frequency band corresponding to the sub-audio signal. Step S3212: Determine the leakage direction constraint weight coefficient corresponding to the frequency band based on the frequency band. The leakage direction constraint weight coefficient corresponding to a higher frequency band is smaller than that corresponding to a lower frequency band. Step S3213: Determine the audio spatial propagation characteristics based on at least one preset sound leakage direction and the sound leakage direction constraint weight coefficient corresponding to the frequency band. Step S3214: Based on the phase pre-compensation amount and the audio spatial propagation characteristics, determine the current sound parameters of each loudspeaker in the loudspeaker array that satisfy the constraints. Wherein, the phase pre-compensation amount is the phase compensation amount required to be applied when the audio output of each speaker in the speaker array simultaneously reaches the target listening position; the audio spatial propagation characteristics are used to characterize the comprehensive quantization value of the relative phase relationship distribution of the audio radiated by each speaker in each leakage direction; the constraint condition is that the total sound pressure energy in the leakage direction is minimized under the constraint that the sound pressure gain in the direction of the target listening position is 1; and the current sound parameters include the current amplitude and the current phase.

[0085] In some embodiments, the audio playback method for wearable devices provided in this disclosure further includes, before step S330: Step S3311: Perform a smooth transition process on the current sound parameters of each loudspeaker based on the historical sound parameters of each loudspeaker, and update the current sound parameters of each loudspeaker.

[0086] In some embodiments, the audio playback method for wearable devices provided in this disclosure further includes, before step S310: Step S3111: Obtain the environmental image captured by the image sensor in the wearable device; Step S3112: Identify the environmental image and generate an audio signal to be played for the environmental image.

[0087] In some embodiments, the audio playback method for wearable devices provided in this disclosure further includes, before step S310: Step S3121: Acquire the ambient audio signal collected by the microphone module in the wearable device; Step S3122: Identify the ambient audio signal and generate an audio signal to be played for the ambient audio signal.

[0088] This disclosure also provides an audio playback device 400 for a wearable device, applicable to any of the wearable devices provided in the above embodiments. For example... Figure 4 As shown, the audio playback device 400 of the wearable device includes: The segmentation module 410 is used to divide the audio signal to be played into sub-audio signals of at least two adjacent frequency bands; The determining module 420 is used to determine, for each of the sub-audio signals, the current sound parameters of each speaker in the speaker array that satisfy the constraints, based on the geometric parameters of the speaker array in the wearable device, the reference offset, the current posture data collected by the posture sensor in the wearable device, at least one preset sound leakage direction, and the frequency band corresponding to the sub-audio signal. The control module 430 is used to control the speaker array to emit sound according to each of the sub-audio signals and the current sound emission parameters of each speaker corresponding to each of the sub-audio signals; Wherein, the reference offset is the relative offset between the wearable device and the target listening position when the wearable device is in the standard wearing state, the constraint condition is that the total sound pressure energy in the leakage direction is minimized under the constraint that the sound pressure gain in the direction of the target listening position is 1, and the current sound parameters include the current amplitude and the current phase.

[0089] In some embodiments, the determining module 420 is specifically used to: determine the phase pre-compensation amount of the speaker array based on the geometric parameters of the speaker array in the wearable device, the reference offset, the current posture data collected by the posture sensor in the wearable device, and the frequency band corresponding to the sub-audio signal; Based on the frequency band, determine the leakage direction constraint weight coefficient corresponding to the frequency band. The leakage direction constraint weight coefficient corresponding to a higher frequency band is smaller than that corresponding to a lower frequency band. The audio spatial propagation characteristics are determined based on at least one preset sound leakage direction and the sound leakage direction constraint weight coefficient corresponding to the frequency band. Based on the phase pre-compensation amount and the audio spatial propagation characteristics, determine the current sound parameters of each loudspeaker in the loudspeaker array that satisfy the constraints. Wherein, the phase pre-compensation amount is the phase compensation amount required to be applied when the audio output of each speaker in the speaker array simultaneously reaches the target listening position; the audio spatial propagation characteristics are used to characterize the comprehensive quantization value of the relative phase relationship distribution of the audio radiated by each speaker in each leakage direction; the constraint condition is that the total sound pressure energy in the leakage direction is minimized under the constraint that the sound pressure gain in the direction of the target listening position is 1; and the current sound parameters include the current amplitude and the current phase.

[0090] In some embodiments, the audio playback device 400 of the wearable device provided in this disclosure further includes: An update module is used to perform a smooth transition process on the current sound parameters of each speaker based on the historical sound parameters of each speaker, and update the current sound parameters of each speaker.

[0091] In some embodiments, the audio playback device 400 of the wearable device provided in this disclosure further includes: The first generation module is used to acquire environmental images collected by the image sensor in the wearable device; The environmental image is identified, and an audio signal to be played is generated for the environmental image.

[0092] In some embodiments, the audio playback device 400 of the wearable device provided in this disclosure further includes: The second generation module is used to acquire the ambient audio signal collected by the microphone module in the wearable device; Identify the ambient audio signal and generate an audio signal to be played for the ambient audio signal.

[0093] This disclosure also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the audio playback method of the wearable device in any embodiment of this disclosure. Optionally, the computer-readable storage medium may be a non-transitory storage medium, but is not limited thereto, and may also be a temporary storage medium.

[0094] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0095] This disclosure may be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having a computer-readable program loaded thereon for causing a processor to implement any of the methods in the foregoing embodiments of this disclosure.

[0096] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media may include, for example, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), compact disc-read-only memory (CD-ROM), digital versatile disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any combination thereof. The computer-readable storage medium used herein is not to be interpreted as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0097] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include one or more of copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to computer-readable storage media in the respective computing / processing device.

[0098] The computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source or object programs written in any combination of one or more programming languages, including object-oriented programming languages ​​(such as Smalltalk, C++, etc.) and conventional procedural programming languages ​​(such as the "C" language or similar programming languages). The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network (e.g., a local area network or a wide area network), or it may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays, or programmable logic arrays, is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of the embodiments of this disclosure.

[0099] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus, and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0100] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0101] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0102] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It should be noted that implementation in hardware, implementation in software, and implementation using a combination of software and hardware are all equivalent.

[0103] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of this disclosure is defined by the appended claims.

Claims

1. A wearable device, characterized in that, include: Wearable device, posture sensor, speaker array and processor, including: The wearable body is used to carry the posture sensor, the speaker array, and the processor. The posture sensor is used to collect posture data of the wearable device, the speaker array is used to play audio signals, and the processor is connected to the posture sensor and each speaker in the speaker array. The processor is used to: Divide the audio signal to be played into sub-audio signals of at least two adjacent frequency bands; Based on the characteristic parameters of the loudspeaker array, the current attitude data collected by the attitude sensor, at least one preset sound leakage direction, and the frequency band corresponding to each sub-audio signal, the loudspeaker array is controlled to emit sound according to the current sound emission parameters that satisfy the constraint condition, which is that the total sound pressure energy in the sound leakage direction is minimized under the constraint that the sound pressure gain in the direction of the target listening position is 1.

2. The wearable device according to claim 1, characterized in that, The loudspeaker is a miniature dynamic loudspeaker, and / or the loudspeakers are arranged in a parallel array or a ring array.

3. The wearable device according to claim 1, characterized in that, The wearable device also includes: An image sensor, connected to the processor, is used to acquire environmental images; The processor is further configured to: identify the environmental image and generate an audio signal to be played for the environmental image; The wearable body also serves to support the image sensor.

4. The wearable device according to claim 1, characterized in that, The wearable device also includes: A microphone module, connected to the processor, is used to collect ambient audio signals; The processor is further configured to identify the ambient audio signal and generate an audio signal to be played for the ambient audio signal; The wearable body also serves to support the microphone module.

5. The wearable device according to any one of claims 1-4, characterized in that, The wearable body includes a wearing structure located inside or on the back of the wearable body, and the wearing structure is used to fix the wearable body to the wearer.

6. An audio playback method for a wearable device, characterized in that, The method is applied to a wearable device as described in any one of claims 1-5, and the method includes: Divide the audio signal to be played into sub-audio signals of at least two adjacent frequency bands; Based on the characteristic parameters of the speaker array in the wearable device, the current posture data collected by the posture sensor in the wearable device, at least one preset sound leakage direction, and the frequency band corresponding to each sub-audio signal, the speaker array is controlled to emit sound according to the current sound emission parameters that satisfy the constraint condition, which is that the total sound pressure energy in the sound leakage direction is minimized under the constraint that the sound pressure gain in the direction of the target listening position is 1.

7. The method according to claim 6, characterized in that, The step of controlling the speaker array to emit sound according to the current sound parameters that satisfy the constraints, based on the characteristic parameters of the speaker array in the wearable device, the current posture data collected by the posture sensor in the wearable device, at least one preset sound leakage direction, and the frequency band corresponding to each sub-audio signal, includes: For each sub-audio signal, based on the characteristic parameters of the speaker array in the wearable device, the current posture data collected by the posture sensor in the wearable device, at least one preset sound leakage direction, and the frequency band corresponding to the sub-audio signal, the current sound output parameters of each speaker in the speaker array that satisfy the constraints are determined. The speaker array is controlled to emit sound based on each of the sub-audio signals and the current sound parameters of each speaker corresponding to each sub-audio signal; The characteristic parameters include the geometric parameters of the speaker array and the reference offset. The reference offset is the relative offset between the wearable device and the target listening position when the wearable device is in a standard wearing state. The current sound parameters include the current amplitude and the current phase.

8. The method according to claim 7, characterized in that, The step of determining the current sound parameters of each speaker in the speaker array that satisfy the constraints based on the characteristic parameters of the speaker array in the wearable device, the current posture data collected by the posture sensor in the wearable device, at least one preset sound leakage direction, and the frequency band corresponding to the sub-audio signal includes: The phase pre-compensation amount of the speaker array is determined based on the characteristic parameters of the speaker array in the wearable device, the current posture data collected by the posture sensor in the wearable device, and the frequency band corresponding to the sub-audio signal. Based on the frequency band, determine the leakage direction constraint weight coefficient corresponding to the frequency band. The leakage direction constraint weight coefficient corresponding to a higher frequency band is smaller than that corresponding to a lower frequency band. The audio spatial propagation characteristics are determined based on at least one preset sound leakage direction and the sound leakage direction constraint weighting coefficient corresponding to the frequency band. Based on the phase pre-compensation amount and the audio spatial propagation characteristics, determine the current sound parameters of each loudspeaker in the loudspeaker array that satisfy the constraints. Wherein, the phase pre-compensation amount is the phase compensation amount required to be applied when the audio output of each speaker in the speaker array simultaneously reaches the target listening position; the audio spatial propagation characteristics are used to characterize the comprehensive quantization value of the relative phase relationship distribution of the audio radiated by each speaker in each leakage direction; the constraint condition is that the total sound pressure energy in the leakage direction is minimized under the constraint that the sound pressure gain in the direction of the target listening position is 1; and the current sound parameters include the current amplitude and the current phase.

9. The method according to claim 6, characterized in that, Before controlling the loudspeaker array to emit sound according to the current sound parameters that satisfy the constraints, the method further includes: The current sound parameters of the speaker array are updated by performing a smooth transition process based on the previous historical sound parameters of the speaker array.

10. The method according to claim 7, characterized in that, Before dividing the audio signal to be played into sub-audio signals of at least two adjacent frequency bands, the method further includes: Acquire environmental images captured by the image sensor in the wearable device; Identify the environmental image and generate an audio signal to be played for the environmental image; Alternatively, acquire ambient audio signals collected by the microphone module in the wearable device; Identify the ambient audio signal and generate an audio signal to be played that is specific to the ambient audio signal; Alternatively, acquire environmental images captured by the image sensor in the wearable device and environmental audio signals captured by the microphone module in the wearable device; The environmental image and the environmental audio signal are identified, and an audio signal to be played is generated for the environmental image and the environmental audio signal.

Images