Audio playing method and device, electronic equipment, medium and product

Abstract

Embodiments of the present application provide an audio playing method and device, electronic equipment, medium and product. The method is applied to a smart earphone, and a first sensor is arranged in the smart earphone. The method comprises: acquiring first sensor data collected by the first sensor; determining a water entry state of the smart earphone according to the first sensor data, wherein the water entry state comprises an above-water state or an underwater state; switching the smart earphone to an above-water playing mode when the smart earphone is currently in the above-water state; and switching the smart earphone to an underwater playing mode when the smart earphone is currently in the underwater state. The method of the present application is used to switch the audio playing mode according to the water entry state of the earphone, thereby effectively reducing the difference between the audio playing effects above water and underwater.

Description

Technical Field

[0001] This application relates to the field of headphone technology, and more particularly to an audio playback method, device, electronic device, medium, and product. Background Technology

[0002] Swimmers often wear headphones while swimming to play audio, which helps them focus, adjust their mood, and better enter a swimming state.

[0003] However, during swimming, the position of the headphones may constantly change above and below the water as the swimmer moves. Due to the significant difference in sound wave conduction characteristics between water and air, this drastic change in the sound transmission medium can lead to fluctuating volume and abrupt changes in sound quality, resulting in a poor user experience.

[0004] Therefore, there is an urgent need for a headphone audio playback solution that is suitable for swimming scenarios. Summary of the Invention

[0005] This application provides an audio playback method, apparatus, electronic device, medium, and product for switching audio playback modes according to the water immersion status of headphones, effectively reducing the difference in audio playback effects above and below water.

[0006] In a first aspect, embodiments of this application provide an audio playback method, including:

[0007] The method is applied to smart earphones, wherein a first sensor is provided within the smart earphones; the method includes:

[0008] Acquire the first sensor data collected by the first sensor;

[0009] Based on the data from the first sensor, the water immersion status of the smart earphone is determined, including the state above water or the state underwater.

[0010] When the smart earphones are currently in a water-based state, switch the smart earphones to water playback mode;

[0011] When the smart earphones are currently underwater, switch the smart earphones to underwater playback mode.

[0012] Optionally, the sensor includes an accelerometer and a gyroscope, and the first sensor data includes acceleration data and angular velocity data;

[0013] Determining the water immersion status of the smart earphones based on the data from the first sensor includes:

[0014] Based on the acceleration data and the angular velocity data, determine whether the user is in a state of breathing.

[0015] If the user is in a state of breathing, the smart earphone is determined to be in a state of being on water.

[0016] Otherwise, the smart earphones are determined to be currently underwater.

[0017] Optionally, determining whether the user is in a breathing state based on the acceleration data and the angular velocity data includes:

[0018] The user's swimming style is identified based on the acceleration and angular velocity data within a preset time period;

[0019] Based on the current acceleration and angular velocity data, the user's current head posture is identified; and based on the swimming stroke, it is determined whether the user is in a breathing state.

[0020] Optionally, the method further includes:

[0021] The duration of a user's air exchange cycle is determined based on the time taken for each of the user's historical air exchanges.

[0022] Based on the swimming stroke, determining whether the user is in a breathing state according to the user's current head posture includes:

[0023] If the user's current head posture matches the breathing characteristics corresponding to the swimming stroke, and the difference between the current time and the previous breath and the difference between the breathing cycle duration is not greater than a preset threshold, then it is determined whether the user is in a breathing state; otherwise, it is determined that the user is not in a breathing state.

[0024] Optionally, the first sensor includes a pressure sensor, and the data from the first sensor includes pressure values;

[0025] Determining the water immersion status of the smart earphones based on the data from the first sensor includes:

[0026] If the pressure value is greater than the preset pressure threshold, the smart earphone is determined to be in an underwater state.

[0027] Otherwise, the smart earphones are determined to be currently in a water-based state.

[0028] Optionally, the first sensor includes a microphone, and the first sensor data includes ambient noise data;

[0029] Determining the water immersion status of the smart earphones based on the data from the first sensor includes:

[0030] The environmental noise data is subjected to spectral transformation to calculate the high-low frequency energy ratio of the environmental noise data; the high-low frequency energy ratio is the ratio of the energy of the high-frequency component to the low-frequency component in the spectrum of the environmental noise data, wherein the frequency of the high-frequency component is higher than a first frequency and the frequency of the low-frequency component is lower than a second frequency.

[0031] If the high-low frequency energy ratio is greater than a preset ratio threshold, the smart earphone is determined to be in a water-based state.

[0032] Otherwise, the smart earphones are determined to be currently underwater.

[0033] Optionally, switching the smart headphones to water playback mode includes:

[0034] Within the time window, gradually decrease the sound gain of the underwater playback mode and gradually increase the sound gain of the surface playback mode;

[0035] The smart headphones are switched to underwater playback mode.

[0036] Within the time window, gradually decrease the sound gain of the above-water playback mode and gradually increase the sound gain of the underwater playback mode.

[0037] Optionally, the above-water playback mode is the regular playback mode, while the underwater playback mode is used to compensate for high-frequency attenuation and suppress low-frequency vibration.

[0038] Secondly, embodiments of this application provide an audio playback method applied to a waterproof audio device, the method comprising:

[0039] Obtain water immersion status information sent by the smart earphone; the water immersion status of the smart earphone is determined by the smart earphone based on the first sensor data collected by the built-in first sensor, and the water immersion status includes above-water status or underwater status;

[0040] Based on the water entry status information, the audio data to be transmitted is processed using an above-water / underwater algorithm, and the processed audio data is transmitted to the smart earphone; or, a control command is sent to the smart earphone to switch the smart earphone to above-water / underwater playback mode.

[0041] Optionally, the step of performing above-water / underwater algorithm processing on the audio data to be transmitted based on the water entry status information, or sending control commands to the smart earphones, includes:

[0042] Acquire the second sensor data collected by the second sensor built into the waterproof audio device;

[0043] Based on the data from the second sensor, the water immersion determination result of the smart earphone is obtained;

[0044] Based on the water entry determination result and the water entry status information, the audio data to be transmitted is processed using an above-water / underwater algorithm, or a control command is sent to the smart earphone.

[0045] Thirdly, embodiments of this application provide an audio playback device for use in smart headphones, wherein a first sensor is disposed within the smart headphones; the device includes:

[0046] The acquisition module is used to acquire the first sensor data collected by the first sensor;

[0047] The determining module is used to determine the water immersion state of the smart earphone based on the data from the first sensor, wherein the water immersion state includes an above-water state or an underwater state.

[0048] The switching module is used to switch the smart earphones to water playback mode when the smart earphones are currently in a water-based state;

[0049] When the smart earphones are currently underwater, switch the smart earphones to underwater playback mode.

[0050] Fourthly, embodiments of this application provide an electronic device, including: a memory and a processor;

[0051] The memory stores instructions that the computer executes;

[0052] The processor executes computer execution instructions stored in memory, causing the processor to perform the methods provided above.

[0053] Fifthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the methods provided above.

[0054] The audio playback method, apparatus, electronic device, medium, and product provided in this application identify the water immersion state through the first sensor data of the smart earphone and switch the smart earphone to the corresponding audio playback mode according to the water immersion state. This can effectively compensate for the changes in the same audio playback effect caused by the different sound propagation characteristics in air and water, so that the user's audio listening experience is similar underwater and above water, effectively improving the user's audio playback experience while swimming. Attached Figure Description

[0055] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0056] Figure 1 This is a flowchart illustrating the audio playback method provided in an embodiment of this application.

[0057] Figure 2 A schematic diagram of the structure of a portable electronic device provided in an embodiment of this application.

[0058] Figure 3 This is a flowchart illustrating another audio playback method provided in an embodiment of this application.

[0059] Figure 4 This is a schematic diagram of the structure of the audio playback device provided in the embodiments of this application.

[0060] Figure 5 This is a flowchart illustrating another audio playback method provided in an embodiment of this application.

[0061] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0062] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0063] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0064] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of the relevant data all comply with the relevant laws, regulations, and standards of the relevant countries and regions, have taken necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation access points for users to choose to authorize or refuse.

[0065] Furthermore, the technical solution involved in this application, which involves big data analysis of user information (including but not limited to personal biometrics, identity data, consumption data, asset data, electronic terminal operation data, etc.) and uses distributed technology for automated decision-making, and makes decisions that have a significant impact on personal rights based on the results of automated decision-making, provides users with corresponding operation entry points for users to choose to agree to or reject the results of automated decision-making; if the user chooses to reject, the process will proceed to the expert decision-making process.

[0066] It should be noted that the audio playback method, apparatus, electronic device, medium and product provided in this application can be used in the field of data processing, and can also be used in any field other than distributed systems, such as the field of financial technology. The application field of the audio playback method, apparatus, electronic device, medium and product in this application is not limited.

[0067] Swimmers often wear headphones while swimming to play audio, which helps them focus, adjust their mood, and better enter a swimming state.

[0068] However, during swimming, the position of the headphones may constantly change above and below the water as the swimmer moves. Due to the significant difference in sound wave conduction characteristics between water and air, this drastic change in the sound transmission medium can lead to fluctuating volume and abrupt changes in sound quality, resulting in a poor user experience.

[0069] Therefore, there is an urgent need for a headphone audio playback solution that is suitable for swimming scenarios.

[0070] The audio playback method, apparatus, electronic device, medium, and product provided in this application identify the water immersion state through the first sensor data of the smart earphone and switch the smart earphone to the corresponding audio playback mode according to the water immersion state. This can effectively compensate for the changes in the same audio playback effect caused by the different sound propagation characteristics in air and water, so that the user's audio listening experience is similar underwater and above water, effectively improving the user's audio playback experience while swimming.

[0071] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0072] Figure 1 This is a flowchart illustrating an audio playback method provided in an embodiment of this application. The method is applied to a head-mounted wearable device, such as a smart headset, smart swimming goggles, or smart swimming cap. The device contains a first sensor, such as an accelerometer or gyroscope, and is capable of interacting with the user. Figure 1 As shown, the method includes the following steps:

[0073] S101. Obtain the first sensor data collected by the first sensor.

[0074] S102. Based on the data from the first sensor, determine whether the smart earphone is currently in a state above water or underwater.

[0075] For example, the first sensor can be a pressure sensor. Based on the pressure data collected by the pressure sensor, the water immersion status of the headphones can be determined based on the different pressures above and below water.

[0076] For example, the first sensor can also be an accelerometer and a gyroscope. Based on the motion data collected by the accelerometer and gyroscope, the user's swimming posture can be analyzed, and then the state of the headphones in the water can be determined.

[0077] For example, the first sensor could also be the microphone of a smart headset, which collects ambient noise, identifies its spectral characteristics, and distinguishes the headset's water immersion status.

[0078] S103. When the smart earphones are currently in a water-based state, switch the smart earphones to water playback mode.

[0079] Understandably, when smart headphones are on water, that is, in the air, they can switch to the regular audio playback mode.

[0080] S104. When the smart earphones are currently underwater, switch the smart earphones to underwater playback mode.

[0081] Understandably, the density and speed of sound in water weaken the perception of low frequencies while making high frequencies relatively prominent. The same sound may sound more muffled or harsh underwater. Therefore, underwater playback mode can compensate for high-frequency attenuation using an equalizer (EQ), improve the penetration of high frequencies, prevent music from sounding muddy underwater, compensate for sound distortion caused by water pressure and medium changes, suppress excessive low-frequency resonance, avoid body resonance, and make the listening experience more balanced.

[0082] In this embodiment, the first sensor data of the smart earphone identifies the water entry state, and the smart earphone is switched to the corresponding audio playback mode according to the water entry state. This can effectively compensate for the changes in the same audio playback effect caused by the different sound propagation characteristics in air and water, so that the user's audio listening experience is similar underwater and above water, effectively improving the user's audio playback experience while swimming.

[0083] The following is a detailed explanation of how to identify whether headphones have been submerged in water.

[0084] In some embodiments, the first sensor includes a pressure sensor, and the first sensor data includes a pressure value;

[0085] Based on data from the first sensor, the water immersion status of the smart earphones is determined, including:

[0086] If the pressure value is greater than the preset pressure threshold, the smart earphone is determined to be in an underwater state.

[0087] Otherwise, the smart earphones are determined to be currently in a water-based state.

[0088] In swimming scenarios, the air pressure at the water surface is normal pressure. For every 10 meters of descent underwater, the pressure increases by about 1 atmosphere. Even a 10-centimeter descent will generate a pressure difference of about 100 Pa. Therefore, a high-precision pressure sensor can be used to collect pressure values. If the pressure value is greater than the preset pressure threshold, it is determined that the person is underwater; if it is less than the preset pressure threshold, it is determined that the person is above water.

[0089] For example, to avoid misjudgment, the headphone's water immersion status can be determined to have changed only if the pressure value is greater than or less than the preset pressure threshold within a predetermined time period, and the corresponding playback mode can be switched.

[0090] For example, assuming the headphones are currently in water playback mode, if the pressure values ​​sampled multiple times within the last 0.5 seconds are all greater than the preset pressure threshold, then it is determined that the headphones have changed from water to underwater, and the headphones are switched to underwater playback mode.

[0091] This embodiment identifies the underwater state of the headphones based on the different pressures above and below water. The identification logic is simple, the algorithm is low in complexity, and it is easy to implement.

[0092] In some embodiments, the first sensor includes a microphone, and the first sensor data includes ambient noise data;

[0093] Based on data from the first sensor, the water immersion status of the smart earphones is determined, including:

[0094] Perform spectral transformation on the environmental noise data and calculate the high-low frequency energy ratio of the environmental noise data; the high-low frequency energy ratio is the ratio of the energy of the high-frequency component to the low-frequency component in the spectrum of the environmental noise data, where the frequency of the high-frequency component is higher than the first frequency and the frequency of the low-frequency component is lower than the second frequency.

[0095] If the high-low frequency energy ratio is greater than the preset ratio threshold, the smart earphone is determined to be in a water state.

[0096] Otherwise, the smart earphones are determined to be currently underwater.

[0097] In practical implementation, the microphone, acting as an acoustic sensor, captures noise spectra fundamentally different underwater and above water. Underwater, where water is the sound propagation medium, high-frequency sound waves attenuate extremely quickly, and the microphone primarily captures low-frequency components such as water flow sounds, bubble sounds, and the muffled noise of the body rubbing against the water. Once above water, the sound propagation medium is air, high-frequency components are significantly enhanced, and sounds such as wind, human voices, and ambient pool noise can be heard. This solution can identify the underwater state of the headphones based on the high and low frequency characteristics of the noise.

[0098] For example, a Fast Fourier Transform (FFT) can be performed on the environmental noise data to extract spectral features. Based on a preset first and second frequency, the high-frequency and low-frequency components in the environmental noise data spectrum can be determined, and the energy ratio of the high-frequency to low-frequency components can be calculated (for example, the energy ratio of frequencies above 2kHz to frequencies below 500Hz). If the high-low frequency energy ratio is greater than a preset pressure threshold, the location is determined to be above water; if it is not greater than the preset pressure threshold, the location is determined to be underwater.

[0099] This embodiment utilizes existing components of the headphones, eliminating the need for additional hardware costs. Furthermore, the acoustic characteristics of the collected ambient noise are directly related to the user's actual listening experience, allowing for targeted adjustments to the audio playback effect based on the ambient noise.

[0100] It should be noted that, in practical applications, the first sensor in this embodiment can also be a multi-dimensional sensor, including at least two of an accelerometer and gyroscope, a pressure sensor, and a microphone. Determining the water immersion status of the smart earphones based on the data from the first sensor can include determining whether the smart earphones are currently above water or underwater based on whether the user is in a breathing state, and / or whether the pressure value is greater than a preset pressure threshold, and / or whether the high-low frequency energy ratio is greater than a preset ratio threshold.

[0101] Taking a first sensor including an accelerometer, gyroscope, pressure sensor, and microphone as an example, the conditions for determining that the smart earphone is currently underwater can include the user not being in a state of breathing, the pressure value being greater than a preset pressure threshold, and the high-low frequency energy ratio not being greater than a preset ratio threshold. Alternatively, at least two of the above conditions can be met, and specific determinations can be made based on multi-dimensional first sensor data according to actual production needs, without any restrictions here.

[0102] In some embodiments, the first sensor includes an accelerometer and a gyroscope, and the first sensor data includes acceleration data and angular velocity data;

[0103] Based on data from the first sensor, the water immersion status of the smart earphones is determined, including:

[0104] Based on acceleration and angular velocity data, determine whether the user is in a state of breathing.

[0105] If the user is breathing, the smart earphones are determined to be in a water-based state.

[0106] Otherwise, the smart earphones are determined to be currently underwater.

[0107] In practical implementation, based on acceleration and angular velocity data, the user's overall motion state can be identified. For example, whether the user is initially entering the water or exiting the water after swimming, or whether the user is surfacing to breathe while swimming. Based on the user's motion state, it can be indirectly inferred whether the user's head is currently above or below the water, thereby correspondingly switching the headphone audio playback effect.

[0108] For example, if a user's head is rapidly lifted from a horizontal position (with a significant change in pitch angle) to a near-vertical standing posture, it can be determined that the user has finished swimming and stood up, and the smart earphones are currently in a water-based state.

[0109] If the user's head moves periodically, it can be determined that the user is continuously swimming and breathing, and the smart headphones can be determined to be in an underwater state during breathing.

[0110] Figure 2 This is a schematic diagram of the structure of a portable electronic device provided in an embodiment of this application. Figure 2 As shown, this application embodiment provides a portable electronic device with an adaptive audio switching function for both above-water and underwater environments. Specifically, it can be a smart headset, etc. It uses a six-axis inertial measurement unit (three-axis accelerometer + three-axis gyroscope) to collect fluid dynamic characteristics and combines them with an adaptive threshold algorithm to achieve accurate identification of the immersion state.

[0111] Specifically, the device incorporates a six-axis inertial measurement unit (IMU) 10 (model LSM6DS3) as the primary sensor. Its accelerometer range is configured to ±8g, and its gyroscope range to ±1000 dps. The accelerometer output data rate (ODR) and gyroscope ODR are both set to 200Hz. The IMU 10 is electrically connected to the main microcontroller unit (MCU) 20 within the device via an I2C serial bus. To prevent cavity resonance interference caused by water flow impact, the IMU 10 is encapsulated within a solid potted cavity of the device housing, with its geometric center located near the device's center of mass. This ensures that the acquired acceleration vector a=(ax,ay,az) and angular velocity vector ω=(ωx,ωy,ωz) accurately reflect the device's motion and the external forces acting upon it.

[0112] For example, the MCU 20 has a built-in state determination program that performs the following steps:

[0113] Step S1: Signal preprocessing. The MCU 20 reads the raw data output by the six-axis inertial measurement unit 10 in real time, and uses a first-order low-pass digital filter with a cutoff frequency of 25Hz to denoise the raw acceleration signal, obtaining the filtered acceleration vector amplitude Amag. The calculation formula is as follows:

[0114]

[0115] Furthermore, a first-order high-pass digital filter with a cutoff frequency of 0.2Hz and a first-order low-pass digital filter with a cutoff frequency of 25Hz are used to denoise the original angular velocity signal.

[0116] Step S2: Dynamic baseline calibration. After the device is powered on, the system automatically collects the Amag data and triaxial angular velocity data for the first 5 seconds, calculates the average Amag value as the static gravitational acceleration reference value Gref (ideally 1g), and calculates the average triaxial angular velocity value as the zero-point reference. Simultaneously, the system calculates the standard deviation of Amag and the standard deviation of each axis angular velocity for this period as a reference for the ambient noise floor.

[0117] Step S3: Immersion Feature Extraction and Determination. After the device enters the working mode, the MCU 20 continuously monitors the changes in Amag and triaxial angular velocity. Within a 3-second swimming stroke recognition window, the current swimming stroke is identified according to the preset swimming stroke recognition model. Then, within a 0.5-second breathing recognition window, it identifies whether the user's current head posture matches the corresponding breathing posture of the current swimming stroke, thereby determining whether the user is in a breathing state. When the user is breathing, the device is determined to be in an above-water state.

[0118] Swimming stroke recognition models can identify a user's current swimming stroke based on preset classification rules. The most significant distinguishing features of different swimming strokes are: freestyle is the only swimming stroke in which the head turns left and right; backstroke is the only swimming stroke in which the face is always facing upwards; breaststroke and butterfly both involve lifting the head, but butterfly has two peaks per cycle, while breaststroke has only one. Therefore, swimming strokes can be identified based on these features.

[0119] The swimming stroke recognition rule can be: if the pitch angle data is always within the back position range and the average acceleration value is greater than 1, then the current swimming stroke is determined to be backstroke.

[0120] If the standard deviation of the yaw angle data is greater than the seventh threshold, and there are alternating peaks on both sides within each cycle, then the current swimming style is determined to be freestyle.

[0121] If the standard deviation of the pitch angle data is greater than the eighth threshold, and there is only one peak in each cycle, then the current swimming style is determined to be breaststroke.

[0122] If the standard deviation of the pitch angle data is greater than the eighth threshold, and there are two peaks within each cycle, then the current swimming style is determined to be butterfly.

[0123] The corresponding head posture for breathing in backstroke can be: pitch angle -10 to 10 degrees, yaw angle -5 to 5 degrees.

[0124] The head posture for breathing in breaststroke can be: pitch angle 45-60 degrees, yaw angle -5 to 5 degrees.

[0125] The corresponding head posture for breathing in butterfly stroke can be: pitch angle 15~45 degrees, yaw angle -5~5 degrees.

[0126] The corresponding head posture for breathing in freestyle swimming can be: pitch angle 5~15 degrees, yaw angle ±30~±60 degrees.

[0127] To avoid momentary misjudgments, the system introduces a hysteresis comparison mechanism. The state switching command is only formally triggered when N consecutive sampling points (N=50) all meet the above ventilation judgment conditions.

[0128] Step S4: Audio switching execution. After confirming the state switch, MCU 20 immediately sends a control command to audio codec 30 through a serial audio interface (such as I2S) to switch to surface / underwater mode.

[0129] Water Mode: The audio codec 30 loads the first set of preset digital filter parameters (EQ Profile A), which focuses on high-frequency gain to simulate the effect of an open sound field propagating through the air.

[0130] Underwater Mode: Audio codec 30 loads the preset second set of digital filter parameters (EQ Profile B). Due to the severe attenuation of high frequencies when sound waves propagate in water, Profile B automatically attenuates frequency components above 2kHz and significantly increases the fundamental frequency gain of human voices in the 300Hz-800Hz frequency band. At the same time, it enables the bone conduction compensation algorithm to enhance the clarity of underwater listening.

[0131] Cross-fade is used during switching, with a transition duration of 15ms to prevent popping sounds or interruptions during the switching process.

[0132] In one possible implementation, determining whether a user is in a breathing state based on acceleration and angular velocity data includes:

[0133] Based on acceleration and angular velocity data within a preset time period, the user's swimming style is identified;

[0134] Based on the current acceleration and angular velocity data, the system identifies the user's current head posture; and based on the swimming stroke, it determines whether the user is in a breathing state.

[0135] It is understandable that different swimming strokes have different breathing methods. For example, butterfly and breaststroke involve breathing face down, while freestyle involves breathing from side to side. The specific head angle for breathing also varies.

[0136] Therefore, the user's swimming style can be identified first based on acceleration and angular velocity data within a preset time period. Based on the swimming style, the corresponding breathing characteristics can be determined, and it can be judged whether the user's current head posture matches these breathing characteristics, thereby determining whether the user is in a breathing state.

[0137] For example, when breathing in the breaststroke, the swimmer's head will rise about 45 degrees and will not turn to the left or right. If the user's swimming style is breaststroke and the current angle between the user's head and the water surface is 10 degrees, then it is determined that the user is not currently breathing.

[0138] When breathing in freestyle, the swimmer's head will turn about 90 degrees to the left and right, without obviously rising or falling. If the user's swimming style is freestyle, and the current side angle of the user's head to the water surface is 85 degrees, and the pitch angle of the user's head to the water surface is 10 degrees, then the user is determined to be in a breathing state.

[0139] For example, the recognition of swimming strokes and breathing states can be achieved according to preset rule algorithms or through models such as neural networks, without any restrictions here.

[0140] In one possible implementation, the method also includes:

[0141] The duration of a user's air exchange cycle is determined based on the time taken for each of the user's historical air exchanges.

[0142] Based on swimming stroke and the user's current head position, determine whether the user is breathing, including:

[0143] If the user's current head posture matches the breathing characteristics corresponding to a swimming stroke, and the difference between the current time and the last breath and the difference between the breathing cycle duration is not greater than a preset threshold, then it is determined whether the user is in a breathing state; otherwise, it is determined that the user is not in a breathing state.

[0144] In practice, when the user's head posture matches the breathing characteristics corresponding to their current swimming stroke, an additional check is made to determine whether the user's current breathing action conforms to their historical breathing time patterns. Only if both are true, is the user considered to be in a breathing state. For example, assuming the user's average breathing cycle is 3 seconds, and the current head posture matches the breathing characteristics corresponding to their current swimming stroke, but the time difference since the last breath is 1 second, then the user's current action is determined not to be breathing.

[0145] It should be noted that the aforementioned preset threshold can be a fixed value or can be adaptively adjusted based on the user's historical ventilation time data. For example, the average duration of the user's last 50 ventilation cycles can be used as the user's ventilation cycle duration, and the preset threshold can be determined based on the standard deviation of the duration of these 50 ventilation cycles as a reasonable fluctuation range for the difference between two ventilation times. There are no restrictions on it here.

[0146] By using dual judgment criteria, the accuracy and reliability of recognizing the user's breathing posture can be effectively improved, avoiding misjudgments caused by factors such as occasional head movements of the user or misidentification by the sensor.

[0147] In this embodiment, the system determines whether the user's head has come out of the water due to breathing by combining the user's overall swimming state. For headphones with motion monitoring functions, the system can accurately identify the water entry state using its own hardware. The waterproof structure is simple, and the system can share data with the motion monitoring function, resulting in high recognition efficiency.

[0148] Figure 3 This is a flowchart illustrating another audio playback method provided in an embodiment of this application. Figure 3 As shown, in some embodiments, the method includes the following steps:

[0149] S201. Obtain the first sensor data collected by the first sensor.

[0150] S202. Based on the data from the first sensor, determine whether the smart earphone is currently in a state above water or underwater.

[0151] S203. When the smart headphones are currently in a water-based state, gradually decrease the sound effect gain of the underwater playback mode and gradually increase the sound effect gain of the surface playback mode within the time window.

[0152] S204. When the smart headphones are currently underwater, gradually decrease the sound effect gain of the surface playback mode and gradually increase the sound effect gain of the underwater playback mode within the time window.

[0153] It's understandable that abrupt switching between different audio playback modes can lead to sudden changes in audio playback effects. Therefore, a smooth transition can be used during the switch. After determining that the headphones have changed their water immersion status, the playback parameters are not switched instantly. Instead, a time window (e.g., 200-500ms) is initiated, allowing the current sound effect to gradually weaken while the target sound effect gradually strengthens. The two sound effects are superimposed and mixed during the transition period, so the user perceives the audio mode switch as smooth rather than abrupt, greatly improving the user's listening comfort.

[0154] For example, crossfade-in / fade-out or adaptive gain ramp can be used to switch audio playback modes within the target window.

[0155] Optionally, the duration of the time window can be dynamically adjusted according to the switching speed of the headphones entering the water. For example, a short time window (e.g., 100ms) can be used to ensure responsiveness during rapid breathing (e.g., <1 second), while a long time window (800ms) can be used to ensure a more natural transition during long periods of exiting the water (e.g., resting by the pool).

[0156] For example, in addition to compensating for high frequencies and suppressing low frequencies, the underwater playback mode can also increase the gain of the vocal frequency band and suppress the interference of continuous low-frequency background noise such as underwater current sounds and bubble sounds on vocals. Furthermore, the underwater environment is enclosed, and the sound waves reflected from the pool walls create significant reverberation, resulting in prolonged low-frequency tails; while above water, the open space has shorter reverberation and cleaner sound, thus the underwater playback mode can increase reverberation suppression.

[0157] Optionally, for bone conduction headphones, underwater load changes alter the diaphragm / oscillator damping characteristics, slowing down the transient response and blurring the edges of fast transient signals such as drum beats. Above water, the oscillator vibrates freely, resulting in sharper transients. The transient enhancement algorithm can be adjusted for different states. Furthermore, the human ear's sensitivity to 1-4kHz decreases underwater, and the subjective loudness is lower than above water at the same physical sound pressure. Therefore, the underwater playback mode can establish a separate underwater loudness curve for loudness compensation, rather than simply adjusting the gain.

[0158] The above is an audio playback method provided in the embodiments of this application. Figure 4 This is a schematic diagram of the audio playback device provided in an embodiment of this application. The device is applied to a head-mounted wearable device, which has a first sensor built-in. Figure 4 As shown, this application embodiment also provides an audio playback device 40, including:

[0159] Acquisition module 41 is used to acquire the first sensor data collected by the first sensor;

[0160] The determining module 42 is used to determine the water immersion status of the smart earphone based on the data from the first sensor. The water immersion status includes the state above water or the state below water.

[0161] The switching module 43 is used to switch the smart headphones to the water playback mode when the smart headphones are currently in a water state;

[0162] When the smart earbuds are currently underwater, switch them to underwater playback mode.

[0163] It should be noted that the audio playback device provided in this application embodiment is used to execute the audio playback method provided in the above embodiment, and its specific implementation is as described above, and will not be repeated here.

[0164] This application also provides a multi-device interconnection scheme that uses a waterproof audio device to switch between above-water and underwater playback modes.

[0165] Figure 5 This is a flowchart illustrating another audio playback method provided in this application. The method is executed by a waterproof audio device, such as a smartwatch. After the waterproof audio device and the wearable head device are successfully paired, the audio data is transmitted to the wearable head device for playback. Figure 5 As shown, the method includes:

[0166] S301. Obtain the water immersion status information sent by the smart earphone; the water immersion status of the smart earphone is determined by the first sensor data collected by the built-in first sensor, and the water immersion status includes the above-water status or the underwater status.

[0167] S302. Based on the water entry status information, perform above-water / underwater algorithm processing on the audio data to be transmitted, and transmit the processed audio data to the smart headphones; or, send a control command to the smart headphones to switch the smart headphones to above-water / underwater playback mode.

[0168] In its implementation, the smart earphones use their built-in first sensor to collect data and determine whether they are above or below water, generating corresponding above-water / underwater status information. The smart earphones then send this water immersion status information to a waterproof audio device. Based on the received water immersion status information, the waterproof audio device confirms the current operating mode and performs the corresponding audio effect switching operation.

[0169] For example, a waterproof audio device can process audio data based on its water immersion status before sending the processed audio data to smart headphones. This effectively utilizes the more powerful chip computing power, memory, and battery of the waterproof audio device. The headphones do not need to perform complex audio processing; they only need to receive and play the processed audio stream. Power consumption is significantly reduced, which can effectively extend the usage time of the headphones.

[0170] The digital signal processor (DSP) in the waterproof audio device performs frequency domain equalization (EQ) algorithm processing (underwater state compensation for high frequencies and suppression of low frequencies), dynamic range compression (DRC), and loudness compensation according to the water immersion state. It also uses the LC3 compression algorithm to compress the audio data and then transmits it to the smart headphones via Bluetooth Low Energy.

[0171] For example, waterproof audio devices can also send only raw audio data and control commands to smart headphones, allowing the smart headphones to adjust the playback mode themselves.

[0172] To further improve the accuracy of recognizing the water immersion status of headphones, in some embodiments, based on the water immersion status information, the audio data to be transmitted is processed using an above-water / underwater algorithm, or control commands are sent to the smart headphones, including:

[0173] Acquire data from the second sensor built into the waterproof audio device;

[0174] Based on the data from the second sensor, the water immersion determination result of the smart earphones is obtained;

[0175] Based on the water entry determination result and water entry status information, the audio data to be transmitted is processed by above-water / underwater algorithms, or control commands are sent to the smart headphones.

[0176] For example, the waterproof audio device may have a built-in second sensor, such as an accelerometer and a gyroscope, and / or a pressure sensor, etc. Its specific identification of the water immersion state can be seen in the aforementioned identification of the water immersion state by the smart headphones. For instance, the waterproof audio device uses a built-in six-axis inertial measurement unit to collect the user's arm strokes, uses a pre-trained swimming stroke recognition model to determine whether the user is swimming freestyle, breaststroke, butterfly, or backstroke, and combines this with water depth changes measured by a barometer to determine whether the wrist is in the exit or entry phase of the water. This allows it to deduce whether the headphones worn by the user are in an above-water or underwater environment, thus obtaining the smart headphones' water immersion determination result.

[0177] Understandably, waterproof audio devices use multi-source sensor data to combine their own water immersion determination results with the water immersion status information obtained by the smart earphones to collaboratively determine the water immersion status of the smart earphones. For example, if both the device's own water immersion determination results and the water immersion status information obtained by the smart earphones indicate that the device is in water, then it will finally be determined to be in an underwater state and execute underwater mode control. This effectively improves the confidence in the final water immersion status determination, effectively reduces the false judgment rate, and thus enhances the robustness of the control of the smart earphones.

[0178] Figure 6This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Specifically, this electronic device can be a wearable head-mounted device or a waterproof audio device. Figure 6 As shown, the electronic device includes:

[0179] The electronic device includes a processor 291 and a memory 292; it may also include a communication interface 293 and a bus 294. The processor 291, memory 292, and communication interface 293 can communicate with each other via the bus 294. The communication interface 293 can be used for information transmission. The processor 291 can invoke logical instructions stored in the memory 292 to execute the methods of the above embodiments.

[0180] Furthermore, the logic instructions in the aforementioned memory 292 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0181] The memory 292, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as program instructions / modules corresponding to the methods in the embodiments of this application. The processor 291 executes functional applications and data processing by running the software programs, instructions, and modules stored in the memory 292, thereby implementing the methods in the above-described method embodiments.

[0182] The memory 292 may include a program storage area and a data storage area. The program storage area may store the operating system and application programs required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 292 may include high-speed random access memory and may also include non-volatile memory.

[0183] This application provides a non-transitory computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the methods described in the foregoing embodiments.

[0184] This application provides a computer program product, including a computer program that, when executed by a processor, implements the methods provided in any of the embodiments described above.

[0185] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0186] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0187] It should be understood that the above-described device embodiments are merely illustrative, and the device of this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0188] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0189] When integrated units / modules are implemented in hardware, the hardware can be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the processor can be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, the storage unit can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc.

[0190] If the integrated unit / module is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0191] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0192] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0193] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. An audio playback method, characterized in that, The method is applied to smart earphones, wherein a first sensor is provided within the smart earphones; the method includes: Acquire the first sensor data collected by the first sensor; Based on the data from the first sensor, the water immersion status of the smart earphone is determined, including the state above water or the state underwater. When the smart earphones are currently in a water-based state, switch the smart earphones to water playback mode; When the smart earphones are currently underwater, switch the smart earphones to underwater playback mode.

2. The method according to claim 1, characterized in that, The first sensor includes an accelerometer and a gyroscope, and / or a pressure sensor, and / or a microphone; the first sensor data includes acceleration data and angular velocity data, and / or pressure values, and / or ambient noise data. Determining the water immersion status of the smart earphones based on the data from the first sensor includes: Based on the acceleration data and the angular velocity data, determine whether the user is in a breathing state; and / or, perform spectral transformation on the environmental noise data to calculate the high-low frequency energy ratio of the environmental noise data, wherein the high-low frequency energy ratio is the ratio of the energy of the high-frequency component to the low-frequency component in the spectrum of the environmental noise data, the frequency of the high-frequency component is higher than a first frequency, and the frequency of the low-frequency component is lower than a second frequency. Based on whether the user is in a breathing state, and / or whether the pressure value is greater than a preset pressure threshold, and / or whether the high-low frequency energy ratio is greater than a preset ratio threshold, the smart earphone is determined to be in an above-water or underwater state.

3. The method according to claim 2, characterized in that, The step of determining whether the user is in a breathing state based on the acceleration data and the angular velocity data includes: The user's swimming style is identified based on the acceleration and angular velocity data within a preset time period; Based on the current acceleration and angular velocity data, identify the user's current head posture; and based on the swimming stroke, determine whether the user is in a breathing state. And / or, the method further includes: The duration of a user's air exchange cycle is determined based on the time taken for each of the user's historical air exchanges. Based on the swimming stroke, determining whether the user is in a breathing state according to the user's current head posture includes: If the user's current head posture matches the breathing characteristics corresponding to the swimming stroke, and the difference between the current time and the previous breath and the difference between the breathing cycle duration is not greater than a preset threshold, then it is determined whether the user is in a breathing state; otherwise, it is determined that the user is not in a breathing state.

4. The method according to claim 1, characterized in that, Switching the smart headphones to water playback mode includes: Within the time window, gradually decrease the sound gain of the underwater playback mode and gradually increase the sound gain of the surface playback mode; The smart headphones are switched to underwater playback mode. Within a time window, the sound gain of the above-water playback mode is gradually decreased, while the sound gain of the underwater playback mode is gradually increased; wherein, the above-water playback mode is the normal playback mode, and the underwater playback mode is used to compensate for high-frequency attenuation and suppress low-frequency vibration.

5. An audio playback method, characterized in that, Applied to waterproof audio devices, the method includes: Obtain water immersion status information sent by the smart earphone; the water immersion status of the smart earphone is determined by the smart earphone based on the first sensor data collected by the built-in first sensor, and the water immersion status includes above-water status or underwater status; Based on the water entry status information, the audio data to be transmitted is processed using an above-water / underwater algorithm, and the processed audio data is transmitted to the smart earphone; or, a control command is sent to the smart earphone to switch the smart earphone to above-water / underwater playback mode.

6. The method according to claim 5, characterized in that, The step of performing above-water / underwater algorithm processing on the audio data to be transmitted based on the water entry status information, or sending control commands to the smart headphones, includes: Acquire the second sensor data collected by the second sensor built into the waterproof audio device; Based on the data from the second sensor, the water immersion determination result of the smart earphone is obtained; Based on the water entry determination result and the water entry status information, the audio data to be transmitted is processed using an above-water / underwater algorithm, or a control command is sent to the smart earphone.

7. An audio playback device, characterized in that, The device is applied to smart earphones, wherein a first sensor is provided within the smart earphones; the device includes: The acquisition module is used to acquire the first sensor data collected by the first sensor; The determining module is used to determine the water immersion state of the smart earphone based on the data from the first sensor, wherein the water immersion state includes an above-water state or an underwater state. The switching module is used to switch the smart earphones to water playback mode when the smart earphones are currently in a water-based state; When the smart earphones are currently underwater, switch the smart earphones to underwater playback mode.

8. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 6.

Images