Audio signal processing method and apparatus

Abstract

This disclosure relates to the field of spatial audio technology, specifically to an audio signal processing method and apparatus. An audio signal processing method includes: acquiring a first distance between the current position and an initial position of a mobile device, and a second distance between the current position of the mobile device and a wearable device; determining a first deflection angle of the mobile device based on the first distance, the second distance, and the initial distance between the mobile device and the wearable device; acquiring a second deflection angle representing a change in the posture of the wearable device; determining relative position information between the mobile device and the wearable device based on the first deflection angle, the second deflection angle, and the second distance; and processing the audio signal based on the relative position information to obtain playback audio played by the wearable device. This disclosure improves the spatial audio effect through its implementation method.

Description

Technical Field

[0001] This disclosure relates to the field of spatial audio technology, specifically to an audio signal processing method and apparatus. Background Technology

[0002] Spatial audio, also known as surround audio, refers to the precise placement of surround sound channels in the appropriate locations, allowing users to experience an immersive surround sound experience simply by turning their heads.

[0003] In related technologies, some manufacturers have introduced spatial audio technology into headphones. When users watch videos while wearing headphones, they can feel the changes in the sound source of both ear channels by turning their heads, thus creating a spatial surround audio effect and improving the user experience.

[0004] However, spatial audio technology in this field is only applicable in scenarios with fixed audio source playback devices, and the effect is not good. Summary of the Invention

[0005] To improve the effect of spatial audio, this disclosure provides an audio signal processing method, apparatus, electronic device, and storage medium.

[0006] In a first aspect, embodiments of this disclosure provide an audio signal processing method, including:

[0007] The first distance between the current location and the initial location of the mobile device, and the second distance between the current location of the mobile device and the wearable device are obtained; wherein the mobile device and the wearable device are communicatively connected.

[0008] The first deflection angle of the mobile device is determined based on the first distance, the second distance, and the initial distance between the mobile device and the wearable device;

[0009] Obtain the second deflection angle of the wearable device's posture change;

[0010] The relative position information of the mobile device and the wearable device is determined based on the first deflection angle, the second deflection angle, and the second distance;

[0011] The audio signal is processed based on the relative position information to obtain the playback audio played by the wearable device.

[0012] In some implementations, obtaining the first distance between the current location and the initial location of the mobile device, and the second distance between the current location of the mobile device and the current location of the wearable device, includes:

[0013] The first distance is obtained by a distance measuring device installed on the mobile device;

[0014] And / or,

[0015] The second distance is obtained by detecting a distance measuring device located on the mobile device and / or the wearable device.

[0016] In some implementations, determining the first deflection angle of the mobile device based on the first distance, the second distance, and the initial distance between the mobile device and the wearable device includes:

[0017] Based on the first distance, the second distance, and the initial distance, the first deflection angle is determined using the law of cosines; the first deflection angle is the angle between the current position and the initial position of the mobile device and the line connecting the wearable device.

[0018] In some implementations, obtaining the second deflection angle of the wearable device's posture change includes:

[0019] The second deflection angle is detected by an angle measuring device installed on the wearable device;

[0020] And / or,

[0021] The second deflection angle is obtained by calculating the measurement signal from the inertial measurement device installed on the wearable device.

[0022] In some implementations, determining the relative position information between the mobile device and the wearable device based on the first deflection angle, the second deflection angle, and the second distance includes:

[0023] The relative azimuth angle between the mobile device and the wearable device is obtained based on the first deflection angle and the second deflection angle;

[0024] The relative position information is determined based on the relative azimuth angle and the second distance.

[0025] In some embodiments, processing the audio signal based on the relative position information to obtain the playback audio to be played by the wearable device includes:

[0026] The head-related parameters of the wearable device and the mobile device are determined based on the relative position information;

[0027] The audio signal is remapped based on the head-related parameters to obtain the playback audio.

[0028] In some embodiments, before obtaining the first distance between the current location and the initial location of the mobile device, and the second distance between the current location of the mobile device and the wearable device, the method further includes:

[0029] The state of the trigger switch on the mobile device is detected, and in response to the trigger switch being turned on, the steps of obtaining the first distance and the second distance are performed.

[0030] In some embodiments, the wearable device includes headphones; and / or, the mobile device includes a mobile phone.

[0031] Secondly, embodiments of this disclosure provide an audio signal processing apparatus, comprising:

[0032] The first acquisition module is configured to acquire a first distance between the current position and the initial position of the mobile device, and a second distance between the current position of the mobile device and the wearable device; wherein the mobile device and the wearable device are communicatively connected.

[0033] The first determining module is configured to determine a first deflection angle of the mobile device based on the first distance, the second distance, and the initial distance between the mobile device and the wearable device;

[0034] The second acquisition module is configured to acquire the second deflection angle of the wearable device's posture change;

[0035] The second determining module is configured to determine the relative position information between the mobile device and the wearable device based on the first deflection angle, the second deflection angle, and the second distance;

[0036] The processing module is configured to process the audio signal based on the relative position information to obtain the playback audio to be played by the wearable device.

[0037] In some implementations, the first acquisition module is specifically configured as follows:

[0038] The first distance is obtained by a distance measuring device installed on the mobile device;

[0039] And / or,

[0040] The second distance is obtained by detecting a distance measuring device located on the mobile device and / or the wearable device.

[0041] In some implementations, the first determining module is specifically configured as follows:

[0042] Based on the first distance, the second distance, and the initial distance, the first deflection angle is determined using the law of cosines; the first deflection angle is the angle between the current position and the initial position of the mobile device and the line connecting the wearable device.

[0043] In some implementations, the second acquisition module is specifically configured as follows:

[0044] The second deflection angle is detected by an angle measuring device installed on the wearable device;

[0045] And / or,

[0046] The second deflection angle is obtained by calculating the measurement signal from the inertial measurement device installed on the wearable device.

[0047] In some implementations, the second determining module is specifically configured as follows:

[0048] The relative azimuth angle between the mobile device and the wearable device is obtained based on the first deflection angle and the second deflection angle;

[0049] The relative position information is determined based on the relative azimuth angle and the second distance.

[0050] In some implementations, the processing module is specifically configured as follows:

[0051] The head-related parameters of the wearable device and the mobile device are determined based on the relative position information;

[0052] The audio signal is remapped based on the head-related parameters to obtain the playback audio.

[0053] In some embodiments, the apparatus described in this disclosure further includes:

[0054] The detection module is configured to detect the state of the trigger switch on the mobile device, and in response to the trigger switch being turned on, to perform the steps of acquiring the first rotation information and the second rotation information.

[0055] Thirdly, embodiments of this disclosure provide an electronic device, including:

[0056] processor; and

[0057] The memory stores computer instructions that can be read by the processor, and when the computer instructions are read, the processor executes the method according to any embodiment of the first aspect.

[0058] Fourthly, embodiments of this disclosure provide a storage medium for storing computer-readable instructions for causing a computer to perform the method according to any embodiment of the first aspect.

[0059] The audio signal processing method of this disclosure includes obtaining a first distance between the current position and the initial position of a mobile device and a second distance between the current position of the mobile device and a wearable device; determining a first deflection angle based on the first distance, the second distance, and the initial distance between the mobile device and the wearable device; obtaining a second deflection angle due to a change in the posture of the wearable device; determining relative position information between the mobile device and the wearable device based on the first deflection angle, the second deflection angle, and the second distance; and processing the audio signal based on the relative position information to obtain the audio to be played. This disclosure achieves spatial surround audio, improves audio playback effect, and avoids the defect of inconsistency between the sound source position and the actual position of the device when the mobile device's position changes, thereby improving the user experience. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in the specific embodiments of this disclosure or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0061] Figure 1 These are schematic diagrams illustrating application scenarios based on some embodiments of this disclosure.

[0062] Figure 2 This is a flowchart of an audio signal processing method according to some embodiments of the present disclosure.

[0063] Figure 3 This is a schematic diagram of an audio signal processing method according to some embodiments of this disclosure.

[0064] Figure 4 This is a flowchart of an audio signal processing method according to some embodiments of the present disclosure.

[0065] Figure 5 This is a schematic diagram of an audio signal processing method according to some embodiments of this disclosure.

[0066] Figure 6 This is a flowchart of an audio signal processing method according to some embodiments of the present disclosure.

[0067] Figure 7 This is a flowchart of an audio signal processing method according to some embodiments of the present disclosure.

[0068] Figure 8 This is a flowchart of an audio signal processing method according to some embodiments of the present disclosure.

[0069] Figure 9 This is a structural block diagram of an audio signal processing apparatus according to some embodiments of the present disclosure.

[0070] Figure 10 It is a block diagram of an electronic device suitable for implementing the audio signal processing method of this disclosure. Detailed Implementation

[0071] The technical solutions of this disclosure will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure. Furthermore, the technical features involved in the different embodiments of this disclosure described below can be combined with each other as long as they do not conflict with each other.

[0072] Currently, some manufacturers are incorporating spatial audio technology into Bluetooth headsets. By installing inertial measurement devices such as gyroscopes and accelerometers within the headsets, they can calculate and obtain changes in the headset's posture, thus tracking the wearer's head. When the wearer's head posture changes, the sound source can be remapped, ensuring that the sound source heard by the wearer remains in a fixed position, achieving a spatial surround sound effect.

[0073] In related technologies, the hypothetical sound source is positioned fixed when achieving spatial audio effects. However, mobile terminals differ from fixed-position sound source devices (such as televisions). In real-world scenarios, such as when a user holds a mobile device to watch a video, the device rotates with the user's body. Because the sound source position is fixed, the user's subjective perception of the sound source direction remains at its original location. Similarly, when the mobile device's position changes, because the headphone's orientation remains unchanged, the sound source heard by the user still seems to be in its original position. This discrepancy between the user's subjective perception of the sound source location and the actual location of the mobile device results in a poor user experience.

[0074] Based on the deficiencies existing in related technologies, this disclosure provides an audio signal processing method, apparatus, electronic device, and storage medium, aiming to improve the spatial audio effect and user experience of wearable devices.

[0075] first, Figure 1 The diagram shows system schematics of some embodiments of this disclosure. For example... Figure 1 As shown, the system of this embodiment includes a wearable device 100 and a mobile device 200.

[0076] Wearable device 100 can be any device suitable for playing audio. It can be worn on the human body and its angle deflects with the movement of the user's limbs, i.e., the posture change described in this embodiment. In some embodiments, wearable device 100 can be headphones, TWS (True Wireless Stereo) headphones, etc.

[0077] The mobile device 200 can be any mobile device suitable for generating sound sources. The mobile device 200 can send audio signals to the wearable device 100 and has the ability to be portable, such as a mobile phone, tablet computer, music player, smart wearable device, etc.

[0078] In this embodiment, both the wearable device 100 and the mobile device 200 are equipped with a wireless communication module for establishing a communicable connection between them to enable data transmission. The wireless communication module includes, but is not limited to, a Bluetooth transmission module and a WiFi transmission module. Of course, the wearable device 100 and the mobile device 200 can also establish a communicable connection via a wired connection; this disclosure does not impose any limitations on this.

[0079] Both the wearable device 100 and the mobile device 200 are equipped with an angle measurement device, a distance measurement device, and an inertial measurement unit (IMU). The angle measurement device detects changes in the device's own attitude angles to obtain relevant information generated by the device's rotation. Examples of angle measurement devices include a three-axis gyroscope and an accelerometer. The distance measurement device detects the distance the device moves and the distance between two devices. For example, when the mobile device 200 moves, the distance measurement device detects the distance it moves; or it can detect the distance between the mobile device 200 and the wearable device 100. The distance measurement device can be, for example, a UWB (Ultra Wide Band) ranging module or a laser ranging module. The inertial measurement unit, also known as an IMU, is a high-precision inertial sensor used to detect changes in the device's own pose and calculate the device's pose information based on relevant inertial navigation algorithms.

[0080] Figure 2 This disclosure illustrates audio signal processing methods in some embodiments, which can be applied to, for example... Figure 1 In the system scenario shown, in some embodiments, considering that the computing power of the mobile device 200 is often stronger than that of the wearable device 100, the method of this disclosure can be executed by the processor of the mobile device 200. However, those skilled in the art will understand that the method of this disclosure can also be executed by the processor of the wearable device 100, or jointly executed by the mobile device 200 and the wearable device 100, and this disclosure does not limit it in this way.

[0081] like Figure 2 As shown, in some embodiments, the audio signal processing method of this disclosure includes:

[0082] S210. Obtain a first distance between the current location and the initial location of the mobile device, and a second distance between the current location of the mobile device and the wearable device.

[0083] like Figure 3 As shown, a user wears a wearable device 100 on their head to watch a video played on a mobile device 200. The mobile device 200 sends an audio signal to the wearable device 100 via Bluetooth, so that the wearable device 100 can play the audio signal.

[0084] In an example scenario, suppose mobile device 200 moves from initial position O to current position O1, while wearable device 100 maintains the same position and posture. In existing solutions, since the hypothetical sound source is fixed at the initial position O, after mobile device 200 moves to current position O1, the user's subjective perception of the audio heard through wearable device 100 remains at the initial position O. This results in a discrepancy between the spatial audio effect and the actual device position, degrading the user experience.

[0085] In another example scenario, suppose a user holds a mobile device 200 and rotates it synchronously with the wearable device 100 by a certain angle, meaning their relative positions remain unchanged. In existing solutions, because the user's head rotation causes a change in the posture of the wearable device 100, the system processes the audio signal, making the user subjectively perceive the sound source location as remaining in its initial position. However, the actual relative positions of the two devices do not change, resulting in a discrepancy between the spatial audio effect and the actual device positions, thus degrading the user experience.

[0086] Therefore, improvements are made to address the deficiencies existing in the prior art in this disclosure. (Still using...) Figure 3 As shown in the example, when the mobile device 200 moves from the initial position O to the current position O1, the distance measuring device inside the mobile device 200 can detect the moving distance of the mobile device 200, that is... Figure 3 The midline segment OO1 is denoted as the first distance s1. At the initial position O, the distance measuring device within the mobile device 200 and / or wearable device 100 can detect the initial distance between them, i.e. Figure 3 The midline segment OP is denoted as the initial distance d. At the current position O1, the distance measuring device within the mobile device 200 and / or wearable device 100 can detect the current distance between them, that is... Figure 3 The midline segment O1P is denoted as the second distance d1.

[0087] S220. Determine the first deflection angle of the mobile device based on the first distance, the second distance, and the initial distance between the mobile device and the wearable device.

[0088] Continue to refer to Figure 3 Line segments OO1, OP, and O1P can be connected end to end to form triangle △OO1P. Here, O1P represents the line connecting the current position of the mobile device 200 and the wearable device 100, and OP represents the line connecting the initial position of the mobile device 200 and the wearable device 100. Therefore, the angle ∠OPO1 between O1P and OP represents the deflection angle α of the mobile device's movement, which is also the first deflection angle α of this embodiment.

[0089] In some embodiments of this disclosure, the first deflection angle α can be calculated based on the first distance s1, the second distance d1, and the initial distance d. These embodiments are described in detail below and will not be elaborated upon here.

[0090] S230, Obtain the second deflection angle of the wearable device's posture change.

[0091] In the embodiments disclosed herein, attention is paid not only to the positional changes of the mobile device 200, but also to the posture changes of the wearable device 100.

[0092] For example in Figure 3 In the example, only the mobile device 200 moves from the initial position O to the current position O1, while the user's head posture remains unchanged; that is, the posture of the wearable device 100 does not change. In this case, the second deflection angle for the change in the wearable device's posture only needs to be zero or unchanged.

[0093] In another example, suppose that as the mobile device 200 moves from its initial position O to its current position O1, the user's head turns to the left by a certain angle. Thus, the wearable device 100 undergoes a corresponding posture change with the user's head, and the second deflection angle of the wearable device 100's posture change can then be obtained.

[0094] In some embodiments, the second deflection angle of the wearable device's attitude change can be detected by an angle measuring device located within the wearable device 100. In other embodiments, the attitude change can be detected by an inertial measurement unit located within the wearable device 100, and the second deflection angle of the wearable device 100's attitude change can be calculated based on the measurement signal from the inertial measurement unit. In still other embodiments, the second deflection angle of the wearable device 100's attitude change can be obtained by combining the angle measuring device and the inertial measurement unit within the wearable device 100. The following embodiments of this disclosure will be described in detail, and will not be elaborated further here.

[0095] S240. Determine the relative position information of the mobile device and the wearable device based on the first deflection angle, the second deflection angle, and the second distance.

[0096] It can be understood that the first deflection angle represents the positional deflection angle of the mobile device 200, and the second deflection angle represents the attitude deflection angle of the wearable device 100. Therefore, the relative azimuth angle between the mobile device 200 and the wearable device 100 can be obtained based on the first and second deflection angles. The relative azimuth angle represents the orientation information between the two, while the second distance represents the current distance information between them. Therefore, the relative position information of the two can be accurately defined based on the relative azimuth angle and the second distance. The following will combine... Figure 4 The implementation method will be described in detail.

[0097] like Figure 4 As shown, in some embodiments, the process of determining the relative position information between the mobile device and the wearable device in the audio signal processing method of this disclosure includes:

[0098] S241. Obtain the relative azimuth angle between the mobile device and the wearable device based on the first deflection angle and the second deflection angle.

[0099] For example Figure 5 In the example scenario, the mobile device 200 moves from its initial position O to its current position O1, while the user turns their head to the left by a certain angle from facing the initial position O. The arrow in the figure indicates the direction the user is currently facing. During this process, the second deflection angle β of the pose change of the wearable device 100 can be detected by the angle measuring device and the inertial measuring device within the wearable device 100. Furthermore, the first deflection angle α of the mobile device 200 can be calculated based on the process described in S230.

[0100] After obtaining the first deflection angle α and the second deflection angle β, the relative azimuth angle between the mobile device and the wearable device can be obtained based on the difference between the two. For example Figure 5 In the example, assuming the first deflection angle α is 65° and the second deflection angle β is 35°, the calculated relative azimuth angle is (65°-35°) = 30°. This indicates that the current mobile device 200 is located 30° to the left and in front of the wearable device 100.

[0101] S242. Determine the relative position information based on the relative azimuth and the second distance.

[0102] It can be understood that the relative azimuth angle represents the relative orientation of the mobile device 200 and the wearable device 100, and the relative position of the two can be accurately represented by combining the distance between them.

[0103] In this embodiment of the disclosure, the distance between the mobile device 200 and the wearable device 100 is the second distance d1. The method for determining the second distance d1 is the same as described in the aforementioned S210, and will not be repeated here.

[0104] The relative position information between the mobile device 200 and the wearable device 100 can be determined based on the relative azimuth angle and the second distance.

[0105] S250: Process the audio signal based on the relative position information to obtain the playback audio played by the wearable device.

[0106] HRTF (Head Related Transfer Functions) is an audio localization algorithm. In existing applications, based on tracking the user's head movements, HRTF can be used to remap the sound source, enabling headphones to produce various spatial auditory effects. The basic principle of HRTF can be viewed as: based on different head-related parameters, the audio signal is remapped accordingly to obtain the corresponding auditory effect. Head-related parameters represent the relative pose information between the wearable device and the sound source.

[0107] Figure 6 This disclosure illustrates the process of obtaining audio playback based on relative position information in some embodiments. The following is a detailed explanation. Figure 6 Please provide a detailed explanation.

[0108] like Figure 6 As shown, in some embodiments, the audio signal processing method of this disclosure includes the following process for processing the audio signal:

[0109] S251. Determine the head-related parameters of the wearable device and the mobile device based on the relative position information.

[0110] S252. Remap the audio signal according to the head-related parameters to obtain the playback audio.

[0111] Based on the above principle, after determining the relative position information of the wearable device 100 and the mobile device 200, the head-related parameters between the wearable device 100 and the mobile device 200 can be determined according to the relative position information.

[0112] After determining the relevant parameters, the audio signal transmitted by the mobile device 200 can be remapped based on the HRTF algorithm to obtain the processed playback audio. The mobile device 200 can then send the processed playback audio to the wearable device 100 for playback, allowing the user to hear a spatial surround sound effect.

[0113] It is worth noting that, in this embodiment, the relative position information represents the relative positional relationship between the wearable device 100 and the mobile device 200. Unlike related technologies that assume a fixed location for the audio source, this embodiment utilizes the relative positional relationship between the wearable device 100 and the mobile device 200 to perform audio signal remapping processing.

[0114] For example Figure 3 In the scenario shown, when the user rotates the mobile device 200, that is, the mobile device 200 and the wearable device 100 rotate synchronously, although the pose of the wearable device 100 changes, the relative position information of the mobile device 200 and the wearable device 100 remains unchanged. Therefore, the sound source after the remapping processing in this embodiment can still remain in front of the user, so that the position of the sound source in hearing is consistent with the actual position of the mobile device 200, thereby improving the auditory experience.

[0115] For example Figure 3 In the scenario shown, when the user's head posture remains unchanged and the mobile device 200 moves from the initial position O to the current position O1, the first deflection angle of the mobile device 200 can be detected in this embodiment. This allows for the acquisition of the change in relative position information caused by the movement of the mobile device 200, thereby enabling the processing of the audio signal. This ensures that the perceived location of the sound source is consistent with the actual location of the mobile device 200, improving the auditory experience.

[0116] Furthermore, it is worth noting that in some embodiments of this disclosure, the relative position information of the wearable device 100 and the mobile device 200 is represented by their relative azimuth angle and distance. This is because the pose changes of the wearable device 100 and the mobile device 200 are relatively small. Traditional inertial navigation technology has poor observability for such small pose changes, making it difficult to calculate pose information with high accuracy. If the relative position relationship between the two is calculated directly using the calculated position information of the wearable device and the mobile device, the error will be large, making it difficult to implement. Therefore, in some embodiments of this disclosure, the relative position information of the wearable device and the mobile device is represented by their relative azimuth angle and distance to improve detection accuracy.

[0117] As described above, in this embodiment, audio signal processing is based on the relative position information of the wearable device and the mobile device to achieve spatial surround audio and improve audio playback quality. Furthermore, compared to related technologies where the sound source position is fixed, this method is more suitable for mobile device scenarios, avoiding the inconsistency between the sound source position and the actual device position when the mobile device's position changes, thus improving user experience. Moreover, using the relative azimuth angle and distance between the wearable device and the mobile device to represent their relative position information improves detection accuracy.

[0118] In some embodiments, the process of determining the first deflection angle of the mobile device in the audio signal processing method of this disclosure includes:

[0119] The first deflection angle is determined based on the first distance, the second distance, and the initial distance, using the law of cosines.

[0120] Specifically, with Figure 3Taking an example scenario, the mobile device 200 moves from its initial position O to its current position O1. Line segments OO1, OP, and O1P are connected end to end to form a triangle △OO1P, where O1P represents the line connecting the current position of the mobile device 200 and the wearable device 100, and OP represents the line connecting the initial position of the mobile device 200 and the wearable device 100. Therefore, the angle ∠OPO1 between O1P and OP represents the deflection angle α of the mobile device's movement, which is also the first deflection angle α of this embodiment.

[0121] Based on the Law of Cosines formula:

[0122]

[0123] It can be deduced that:

[0124]

[0125] In the above formula, d represents the initial distance, d1 represents the second distance, and s1 represents the first distance. Therefore, the first deflection angle α can be calculated using the above formula.

[0126] The second deflection angle β can be obtained either by detecting the angle measuring device of the wearable device 100, by calculating the inertial measuring device of the wearable device 100, or by combining the results of both. These will be explained separately in the following disclosure.

[0127] In some implementations, considering that angle measuring devices such as UWB angle measuring modules and ultrasonic angle measuring modules have high detection accuracy when the rotation angle is small (e.g., within ±60° range), when the rotation angle of the wearable device 100 is small, for example, when the second deflection angle |β|≤60°, an angle measuring device can be used to detect the signal of the attitude change of the wearable device 100, thereby obtaining the second deflection angle based on the measurement signal.

[0128] In some implementations, considering that angle measurement devices such as UWB angle measurement modules and ultrasonic angle measurement modules have significantly reduced detection accuracy when the rotation angle is large (e.g., exceeding ±60°), when the rotation angle of the wearable device 100 is large and the second deflection angle |β| > 60°, the inertial measurement unit of the wearable device 100 can be used to detect the attitude change information of the wearable device 100, and the second deflection angle can be calculated based on the detection signal using an inertial navigation algorithm.

[0129] Furthermore, in some embodiments, considering that both the angle measuring device and the inertial measuring device can measure the angle when the rotation angle of the wearable device 100 is small (e.g., within ±60°), the detection results of the two devices can be fused based on the Kalman filter algorithm to improve the detection accuracy of the second deflection angle.

[0130] As can be seen from the above, in this embodiment of the disclosure, the second deflection angle of the wearable device can be determined in a variety of ways, which can meet the needs of a wide range of scenarios and improve the detection accuracy.

[0131] In some embodiments, the audio signal processing method of this disclosure can be implemented by setting a corresponding trigger switch on the mobile device 200 or wearable device 100, and the above process can be executed and turned off by the user manually turning the trigger switch on / off.

[0132] In one example, a corresponding trigger switch can be set on the video playback interface of the mobile device, and this trigger switch can only be displayed when the user wears the wearable device. Thus, when the user wears the wearable device 100 to watch the video playing on the mobile device 200, the trigger switch can be manually turned on. The mobile device 200 can detect the state of the trigger switch, and when the trigger switch is detected to be on, it can perform the aforementioned audio signal processing procedure.

[0133] In other embodiments, the trigger switch may also be located on the wearable device 100. The trigger switch may be a virtual touch switch located on the mobile device or wearable device, or it may be a physical button. This disclosure does not limit this.

[0134] In some embodiments, the wearable device 100 of this disclosure may include TWS earphones or headphones, and the mobile device may include a smartphone. The smartphone may send audio signals to the earphones, which will then play the corresponding audio.

[0135] In one example, such as Figure 5 As shown, the user wears the wearable device 100 and watches the video played on the mobile device 200. The user manually turns on the trigger switch on the mobile device 200. The position where the user turns on the trigger switch is the initial position. At this time, the mobile device 200 detects the initial distance d between the mobile device 200 and the wearable device 100 through the distance measurement unit.

[0136] When the mobile device moves from the initial position O to the current position O1, the mobile device 200 detects the first distance s1 between the initial position O and the current position O1, and the second distance d1 between the current position of the mobile device 200 and the wearable device 100 through the distance measurement unit. At the same time, the user's head causes the wearable device 100 to rotate 35° to the left from the initial posture, so that the angle measurement device of the wearable device 100 can detect that the second deflection angle β is 35°.

[0137] The mobile device 200 calculates a first deflection angle α of 65° using the aforementioned cosine theorem formula based on the first distance s1, the second distance d1, and the initial distance d. Based on the first deflection angle α, the second deflection angle β, and the second distance d1, the relative position information between the mobile device 200 and the wearable device 100 is determined to be "the mobile device is located 30° to the left and in front of the wearable device, and the distance between them is d1". Based on this relative position information, the corresponding head-related parameters are obtained. Then, the audio signal is remapped according to the head-related parameters and sent to the wearable device. The wearable device transmits the processed audio through the receiver, allowing the user to hear audio with a spatial surround sound effect.

[0138] As described above, in this embodiment, audio signal processing is based on the relative position information of the wearable device and the mobile device to achieve spatial surround audio and improve audio playback quality. Furthermore, compared to related technologies where the sound source position is fixed, this method is more suitable for mobile device scenarios, avoiding the inconsistency between the sound source position and the actual device position when the mobile device's position changes, thus improving user experience. Moreover, using the relative azimuth angle and distance between the wearable device and the mobile device to represent their relative position information improves detection accuracy.

[0139] In some implementations, considering that the computing power of the mobile device 200 is often stronger than that of the wearable device 100, the processing steps of the above-described implementation method can be performed by the processor of the mobile device 200. The following is in conjunction with... Figure 7 Please provide an explanation.

[0140] like Figure 7 As shown, in some embodiments, the audio signal processing method of this disclosure includes:

[0141] S701, The mobile device obtains a first distance between its current location and its initial location, a second distance between its current location and the wearable device, and an initial distance between the mobile device and the wearable device.

[0142] Specifically, in some embodiments, when the mobile device moves from its initial position to its current position, a first distance, a second distance, and the initial distance can be detected by a distance measuring device installed on the mobile device. The specific process is detailed in the foregoing embodiments and will not be repeated here.

[0143] S702, The mobile device determines the first deflection angle based on the first distance, the second distance, and the initial distance.

[0144] Specifically, the mobile device can determine the first deflection angle by referring to the process described in S220 above, which will not be repeated here.

[0145] S703, The second deflection angle of the wearable device to obtain its own posture change.

[0146] Specifically, in some embodiments, the second deflection angle of the wearable device's attitude change can be detected by an angle measuring device disposed within the wearable device 100. In other embodiments, the attitude change can be detected by an inertial measurement device disposed within the wearable device 100, and the second deflection angle of the wearable device 100's attitude change can be calculated based on the measurement signal from the inertial measurement device. Further details are omitted here.

[0147] S704, The mobile device receives the second deflection angle sent by the wearable device.

[0148] In one example, wearable device 100 and mobile device 200 can establish a wireless communication connection via Bluetooth module, so that wearable device 100 can send the second deflection angle to mobile device 200 via Bluetooth module.

[0149] S705 The mobile device determines the relative position information based on the first deflection angle, the second deflection angle, and the second distance.

[0150] Specifically, those skilled in the art can understand and fully implement it by referring to the aforementioned S240, and this disclosure will not elaborate further.

[0151] S706: The mobile device processes the audio signal based on the relative position information to obtain the audio to be played.

[0152] Specifically, after obtaining the relative position information, the mobile device 200 can remap the audio signal according to the processing procedure described in the foregoing embodiments to obtain the audio to be played. The specific process is detailed in the foregoing embodiments and will not be repeated here.

[0153] S707: The mobile device will play audio and send it to the wearable device.

[0154] In one example, wearable device 100 and mobile device 200 can establish a wireless communication connection via a Bluetooth module, allowing mobile device 200 to send audio to wearable device 100 via Bluetooth. Upon receiving the audio, wearable device 100 can play the audio through its receiver, allowing the user to hear audio with a spatial surround effect.

[0155] As can be seen from the above, the present invention utilizes mobile devices for data processing, which requires less computing power from wearable devices and is applicable to various types of wearable devices.

[0156] In some embodiments, the processing steps of the methods described above in this disclosure can be performed by the processor of the wearable device 100. The following is in conjunction with... Figure 8 Please provide an explanation.

[0157] like Figure 8 As shown, in some embodiments, the audio signal processing method of this disclosure includes:

[0158] S801, The mobile device obtains the first distance between the current location and the initial location.

[0159] Specifically, in some implementations, when the mobile device moves from its initial position to its current position, a first distance can be detected by a distance measuring device located on the mobile device.

[0160] S802, The wearable device receives the first distance sent by the mobile device.

[0161] In one example, wearable device 100 and mobile device 200 can establish a wireless communication connection via Bluetooth module, so that mobile device 200 can send a first distance to wearable device 100 via Bluetooth module.

[0162] S803, The wearable device obtains the second distance between the current location of the mobile device and the wearable device, as well as the initial distance between the mobile device and the wearable device.

[0163] Specifically, the second distance and the initial distance can be detected by a distance measuring device installed on the wearable device. The specific process is detailed in the aforementioned implementation method and will not be repeated here.

[0164] S804. The wearable device determines the first deflection angle of the mobile device based on the first distance, the second distance and the initial distance, and the wearable device obtains the second deflection angle of its own posture change.

[0165] Specifically, the wearable device can determine the first deflection angle by referring to the process described in S220 above, which will not be repeated here.

[0166] In some embodiments, the second deflection angle of the wearable device's attitude change can be detected by an angle measuring device disposed within the wearable device 100. In other embodiments, the attitude change can be detected by an inertial measurement device disposed within the wearable device 100, and the second deflection angle of the wearable device 100's attitude change can be calculated based on the measurement signal from the inertial measurement device. Details of this disclosure will not be elaborated further.

[0167] S805. The wearable device determines the relative position information between the mobile device and the wearable device based on the first deflection angle, the second deflection angle, and the second distance.

[0168] Specifically, those skilled in the art can understand and fully implement it by referring to the aforementioned S240, and this disclosure will not elaborate further.

[0169] S806, The wearable device receives audio signals sent by the mobile device.

[0170] In one example, wearable device 100 and mobile device 200 can establish a wireless communication connection via Bluetooth module, so that mobile device 200 can send audio signals to wearable device 100 via Bluetooth module.

[0171] S807: The wearable device processes the audio signal based on the relative position information to obtain the audio to be played.

[0172] Specifically, after obtaining the relative position information, the wearable device 100 can remap the received audio signal according to the processing procedure described in the foregoing embodiments to obtain the audio to be played. The specific process is detailed in the foregoing embodiments and will not be repeated here.

[0173] After the wearable device 100 processes and obtains the audio, it can play the audio through the receiver, so that the user can hear the audio with a spatial surround effect.

[0174] As can be seen from the above, the data processing using wearable devices in this embodiment of the present disclosure has lower requirements for the computing power of mobile devices and is applicable to various types of mobile devices.

[0175] This disclosure provides an audio signal processing apparatus, such as... Figure 9 As shown, in some embodiments, the audio signal processing apparatus of this disclosure includes:

[0176] The first acquisition module 710 is configured to acquire a first distance between the current position and the initial position of the mobile device, and a second distance between the current position of the mobile device and the wearable device; wherein the mobile device and the wearable device are communicatively connected.

[0177] The first determining module 720 is configured to determine a first deflection angle of the mobile device based on the first distance, the second distance, and the initial distance between the mobile device and the wearable device;

[0178] The second acquisition module 730 is configured to acquire a second deflection angle of the wearable device's posture change;

[0179] The second determining module 740 is configured to determine the relative position information between the mobile device and the wearable device based on the first deflection angle, the second deflection angle, and the second distance;

[0180] The processing module 750 is configured to process the audio signal according to the relative position information to obtain the playback audio played by the wearable device.

[0181] As described above, in this embodiment, audio signal processing is based on the relative position information of the wearable device and the mobile device to achieve spatial surround audio and improve audio playback quality. Furthermore, compared to related technologies where the sound source position is fixed, this method is more suitable for mobile device scenarios, avoiding the inconsistency between the sound source position and the actual device position when the mobile device's position changes, thus improving user experience. Moreover, using the relative azimuth angle and distance between the wearable device and the mobile device to represent their relative position information improves detection accuracy.

[0182] In some implementations, the first acquisition module 710 is specifically configured as follows:

[0183] The first distance is obtained by a distance measuring device installed on the mobile device;

[0184] And / or,

[0185] The second distance is obtained by detecting a distance measuring device located on the mobile device and / or the wearable device.

[0186] In some implementations, the first determining module 720 is specifically configured as follows:

[0187] Based on the first distance, the second distance, and the initial distance, the first deflection angle is determined using the law of cosines; the first deflection angle is the angle between the current position and the initial position of the mobile device and the line connecting the wearable device.

[0188] In some implementations, the second acquisition module 730 is specifically configured as follows:

[0189] The second deflection angle is detected by an angle measuring device installed on the wearable device;

[0190] And / or,

[0191] The second deflection angle is obtained by calculating the measurement signal from the inertial measurement device installed on the wearable device.

[0192] In some implementations, the second determining module 740 is specifically configured as follows:

[0193] The relative azimuth angle between the mobile device and the wearable device is obtained based on the first deflection angle and the second deflection angle;

[0194] The relative position information is determined based on the relative azimuth angle and the second distance.

[0195] In some embodiments, the processing module 750 is specifically configured as follows:

[0196] The head-related parameters of the wearable device and the mobile device are determined based on the relative position information;

[0197] The audio signal is remapped based on the head-related parameters to obtain the playback audio.

[0198] In some embodiments, the audio signal processing apparatus described herein further includes:

[0199] The detection module is configured to detect the state of the trigger switch on the mobile device, and in response to the trigger switch being turned on, to perform the steps of acquiring the first rotation information and the second rotation information.

[0200] As described above, in this embodiment, audio signal processing is based on the relative position information of the wearable device and the mobile device to achieve spatial surround audio and improve audio playback quality. Furthermore, compared to related technologies where the sound source position is fixed, this method is more suitable for mobile device scenarios, avoiding the inconsistency between the sound source position and the actual device position when the mobile device's position changes, thus improving user experience. Moreover, using the relative azimuth angle and distance between the wearable device and the mobile device to represent their relative position information improves detection accuracy.

[0201] This disclosure provides an electronic device, including:

[0202] processor; and

[0203] The memory stores computer instructions that can be read by the processor, and when the computer instructions are read, the processor executes the method according to any of the above embodiments.

[0204] This disclosure provides a storage medium for storing computer-readable instructions that cause a computer to perform the method according to any of the above embodiments.

[0205] Figure 10 The diagram shows a structural block diagram of an electronic device according to some embodiments of the present disclosure. The following is a description of the structure of the device in conjunction with the provided text. Figure 10 The principles related to the electronic devices and storage media of some embodiments of this disclosure will be explained.

[0206] Reference Figure 10 The electronic device 1800 may include one or more of the following components: a processing component 1802, a memory 1804, a power supply component 1806, a multimedia component 1808, an audio component 1810, an input / output (I / O) interface 1812, a sensor component 1816, and a communication component 1818.

[0207] Processing component 1802 typically controls the overall operation of electronic device 1800, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 1802 may include one or more processors 1820 to execute instructions. Furthermore, processing component 1802 may include one or more modules to facilitate interaction between processing component 1802 and other components. For example, processing component 1802 may include a multimedia module to facilitate interaction between multimedia component 1808 and processing component 1802. As another example, processing component 1802 may read executable instructions from memory to implement relevant functions of the electronic device.

[0208] Memory 1804 is configured to store various types of data to support the operation of electronic device 1800. Examples of this data include instructions for any application or method operating on electronic device 1800, contact data, phonebook data, messages, pictures, videos, etc. Memory 1804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0209] Power supply component 1806 provides power to various components of electronic device 1800. Power supply component 1806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 1800.

[0210] The multimedia component 1808 includes a display screen that provides an output interface between the electronic device 1800 and the user. In some embodiments, the multimedia component 1808 includes a front-facing camera and / or a rear-facing camera. When the electronic device 1800 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera can receive external multimedia data. Each front-facing camera and rear-facing camera can be a fixed optical lens system or have focal length and optical zoom capabilities.

[0211] Audio component 1810 is configured to output and / or input audio signals. For example, audio component 1810 includes a microphone (MIC) configured to receive external audio signals when electronic device 1800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 1804 or transmitted via communication component 1818. In some embodiments, audio component 1810 also includes a speaker for outputting audio signals.

[0212] I / O interface 1812 provides an interface between processing component 1802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0213] Sensor assembly 1816 includes one or more sensors for providing state assessments of various aspects of electronic device 1800. For example, sensor assembly 1816 may detect the on / off state of electronic device 1800, the relative positioning of components such as the display and keypad of electronic device 1800, changes in position of electronic device 1800 or a component of electronic device 1800, the presence or absence of user contact with electronic device 1800, the orientation or acceleration / deceleration of electronic device 1800, and temperature changes of electronic device 1800. Sensor assembly 1816 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 1816 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 1816 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0214] Communication component 1818 is configured to facilitate wired or wireless communication between electronic device 1800 and other devices. Electronic device 1800 can access wireless networks based on communication standards, such as Wi-Fi, 2G, 3G, 4G, 5G, or 6G, or combinations thereof. In one exemplary embodiment, communication component 1818 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 1818 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0215] In an exemplary embodiment, the electronic device 1800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components.

[0216] Obviously, the above embodiments are merely examples for clear illustration and are not intended to limit the embodiments. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively list all embodiments here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this disclosure.

Claims

1. An audio signal processing method, characterized in that, include: The first distance between the current location and the initial location of the mobile device, and the second distance between the current location of the mobile device and the wearable device are obtained; wherein the mobile device and the wearable device are communicatively connected. The first deflection angle of the mobile device is determined based on the first distance, the second distance, and the initial distance between the mobile device and the wearable device; Obtain the second deflection angle of the wearable device's posture change; The relative azimuth angle between the mobile device and the wearable device is obtained based on the first deflection angle and the second deflection angle; The relative position information is determined based on the relative azimuth angle and the second distance; The audio signal is processed based on the relative position information to obtain the playback audio played by the wearable device.

2. The method according to claim 1, characterized in that, The acquisition of the first distance between the current location and the initial location of the mobile device, and the second distance between the current location of the mobile device and the current location of the wearable device, includes: The first distance is obtained by a distance measuring device installed on the mobile device; And / or, The second distance is obtained by detecting a distance measuring device located on the mobile device and / or the wearable device.

3. The method according to claim 1, characterized in that, Determining the first deflection angle of the mobile device based on the first distance, the second distance, and the initial distance between the mobile device and the wearable device includes: Based on the first distance, the second distance, and the initial distance, the first deflection angle is determined using the law of cosines; the first deflection angle is the angle between the current position and the initial position of the mobile device and the line connecting the wearable device.

4. The method according to claim 1, characterized in that, The second deflection angle for obtaining the attitude change of the wearable device includes: The second deflection angle is detected by an angle measuring device installed on the wearable device; or, The second deflection angle is obtained by calculating the measurement signal from the inertial measurement device installed on the wearable device.

5. The method according to claim 1, characterized in that, The step of processing the audio signal based on the relative position information to obtain the playback audio played by the wearable device includes: The head-related parameters of the wearable device and the mobile device are determined based on the relative position information; The audio signal is remapped based on the head-related parameters to obtain the playback audio.

6. The method according to any one of claims 1 to 5, characterized in that, Before obtaining the first distance between the current location and the initial location of the mobile device, and the second distance between the current location of the mobile device and the wearable device, the method further includes: The state of the trigger switch on the mobile device is detected, and in response to the trigger switch being turned on, the steps of obtaining the first distance and the second distance are performed.

7. An audio signal processing device, characterized in that, include: The first acquisition module is configured to acquire a first distance between the current position and the initial position of the mobile device, and a second distance between the current position of the mobile device and the wearable device; wherein the mobile device and the wearable device are communicatively connected. The first determining module is configured to determine a first deflection angle of the mobile device based on the first distance, the second distance, and the initial distance between the mobile device and the wearable device; The second acquisition module is configured to acquire the second deflection angle of the wearable device's posture change; The second determining module is configured to obtain the relative azimuth angle between the mobile device and the wearable device based on the first deflection angle and the second deflection angle; and to determine the relative position information based on the relative azimuth angle and the second distance. The processing module is configured to process the audio signal based on the relative position information to obtain the playback audio to be played by the wearable device.

8. The apparatus according to claim 7, characterized in that, The first acquisition module is specifically configured as follows: The first distance is obtained by a distance measuring device installed on the mobile device; And / or, The second distance is obtained by detecting a distance measuring device located on the mobile device and / or the wearable device.

9. The apparatus according to claim 7, characterized in that, The first determining module is specifically configured as follows: Based on the first distance, the second distance, and the initial distance, the first deflection angle is determined using the law of cosines; the first deflection angle is the angle between the current position and the initial position of the mobile device and the line connecting the wearable device.

10. The apparatus according to claim 7, characterized in that, The second acquisition module is specifically configured as follows: The second deflection angle is detected by an angle measuring device installed on the wearable device; or, The second deflection angle is obtained by calculating the measurement signal from the inertial measurement device installed on the wearable device.

11. The apparatus according to claim 7, characterized in that, The processing module is specifically configured as follows: The head-related parameters of the wearable device and the mobile device are determined based on the relative position information; The audio signal is remapped based on the head-related parameters to obtain the playback audio.

12. The apparatus according to any one of claims 7 to 11, characterized in that, Also includes: The detection module is configured to detect the state of a trigger switch on the mobile device, and in response to the trigger switch being turned on, to perform the steps of obtaining the first distance and the second distance.

13. An electronic device, characterized in that, include: processor; and The memory stores computer instructions that can be read by the processor, and when the computer instructions are read, the processor executes the method according to any one of claims 1 to 6.

14. A storage medium, characterized in that, Used to store computer-readable instructions for causing a computer to perform the method according to any one of claims 1 to 6.

Images