Double-earphone sound effect adjusting method and system, storage medium and program product

Abstract

The invention discloses a double-earphone sound effect adjusting method and system, a storage medium and a program product, and relates to the field of loudspeakers, and the method comprises the steps: collecting an environment audio signal through a binaural microphone, and executing spectrum analysis to obtain a frequency distribution characteristic of the environment audio signal; based on a preset human voice frequency range, extracting an audio signal conforming to the human voice feature from the frequency distribution feature as a target audio; calculating the time difference and the intensity difference of the target audio between the left ear and the right ear to obtain the human sound source direction and the relative distance in the environment; acquiring head attitude sensor data, and calculating a matching included angle between the head orientation of the user and the human sound source direction based on the head attitude sensor data; when the matching included angle is smaller than a preset threshold value, increasing the audio signal gain of the head facing the direction; and when the conversation voice of the user is detected, the audio signal gain is kept unchanged. By implementing the application, the interaction efficiency when the double earphones are used can be improved.

Description

Technical Field

[0001] The present application relates to the field of loudspeakers, and in particular to a method, system, storage medium and program product for adjusting the sound effects of dual headphones. Background Art

[0002] With the widespread adoption of smart wearable devices, headphones have become an indispensable accessory in people's daily lives. Users often wear headphones for extended periods of time during work, study, and commuting, significantly changing how they interact with the outside world. This is especially true in noisy environments, such as construction sites and tunnel construction sites, where workers wearing noise-canceling headphones need to communicate effectively and efficiently with colleagues. This places higher demands on the headphones' intelligent interaction capabilities.

[0003] Smart earphones in related technologies primarily use touch or voice commands for interactive control. Some earphones use touch sensors to detect user touch operations, such as taps and swipes, to control music playback, answer calls, or switch noise cancellation modes. Other earphones use voice wake-up or preset gestures to trigger ambient sound transmission, allowing users to hear external sounds without removing the earphones.

[0004] However, related technologies generally only start sound source processing after the interaction signal is triggered. This passive signal processing mechanism limits the efficiency of real-time interaction. Summary of the Invention

[0005] The present application provides a dual-headphone sound effect adjustment method, system, storage medium and program product for improving the interaction efficiency when using dual headphones.

[0006] In the first aspect, the present application provides a dual-headphone sound effect adjustment method, which is applied to a dual-headphone system. The method includes: collecting ambient audio signals through binaural microphones, performing spectrum analysis to obtain frequency distribution characteristics of the ambient audio signals; based on a preset human voice frequency range, extracting an audio signal that meets the human voice characteristics from the frequency distribution characteristics as the target audio; calculating the time difference and intensity difference of the target audio between the left and right ears to obtain the direction and relative distance of the human voice source in the environment; obtaining head posture sensor data, and calculating the matching angle between the user's head orientation and the direction of the human voice source based on the head posture sensor data; when the matching angle is less than a preset threshold, increasing the audio signal gain in the direction of the head orientation; when the user's conversational voice is detected, maintaining the audio signal gain unchanged.

[0007] In the above embodiment, the dual-headphone system uses binaural microphones to collect ambient audio in real time and analyze human voice characteristics. Combined with head posture detection, the system can automatically identify the direction of the speaker the user is paying attention to and intelligently adjust the audio gain in that direction. When the user is talking to others, the system automatically maintains the audio gain stable to avoid feedback interference; this active sound source tracking and gain adjustment mechanism enables users to talk naturally and smoothly with people around them while wearing headphones, without having to manually switch modes or take off the headphones.

[0008] In combination with some embodiments of the first aspect, in some embodiments, the step of collecting ambient audio signals through binaural microphones and performing spectrum analysis to obtain frequency distribution characteristics of the ambient audio signals specifically includes: synchronously collecting ambient audio signals through left and right ear microphones and generating a digital audio data stream; the digital audio data stream includes binaural sound pressure signals and timestamp information; denoising the digital audio data stream to obtain a noise-reduced audio signal; the signal-to-noise ratio of the noise-reduced audio signal is higher than that of the ambient audio signal; performing fast Fourier transform on the noise-reduced audio signal to generate frequency distribution characteristics including frequency domain data; the frequency domain data includes frequency components and amplitude information.

[0009] In the above embodiment, the dual-headphone system synchronously collects binaural audio signals and adds timestamps, and combines noise reduction processing and fast Fourier transform. The system can accurately obtain the frequency domain characteristics of ambient sound, providing a reliable data basis for voice recognition and direction positioning, ensuring that the system can accurately distinguish between human voices and ambient noise.

[0010] In combination with some embodiments of the first aspect, in some embodiments, the step of calculating the time difference and intensity difference of the target audio between the left and right ears to obtain the direction and relative distance of the human voice source in the environment specifically includes: performing cross-correlation operation on the target audio between the left and right ears to obtain a cross-correlation time series; based on the peak position of the cross-correlation time series, extracting the time delay of the target audio between the left and right ears, and calculating the spatial transfer function in combination with the sound pressure level difference to generate spatial coordinate data of the target audio; constructing a sound source localization map according to the spatial coordinate data, and determining the direction and relative distance of the human voice source based on the sound source localization map.

[0011] In the above embodiment, the dual-headphone system can construct an accurate sound source localization map by performing cross-correlation analysis and spatial transfer function calculation on binaural signals, accurately tracking the speaker's position, and enabling the system to accurately identify and track the target speaker in a complex sound field environment.

[0012] In combination with some embodiments of the first aspect, in some embodiments, before the step of collecting ambient audio signals through binaural microphones and performing spectral analysis to obtain frequency distribution characteristics of the ambient audio signals, the method also includes: collecting sound pressure level data of the ambient noise and generating a noise characteristic spectrum; the noise characteristic spectrum contains energy distribution information of each frequency band; calculating the gain control parameters of the microphone based on the noise characteristic spectrum, and writing the gain control parameters into the microphone control register; adjusting the sensitivity of the signal acquisition channel based on the gain control parameters so that the collected audio signal range is within a preset optimal range.

[0013] In the above embodiment, before audio collection, the dual-headphone system first analyzes the ambient noise characteristics and adaptively adjusts the microphone gain parameters to ensure that the collected signal always remains within the optimal dynamic range. This preprocessing mechanism significantly improves the accuracy of subsequent sound source identification and positioning, enabling the system to maintain stable performance in various noise environments.

[0014] In combination with some embodiments of the first aspect, in some embodiments, the step of collecting sound pressure level data of ambient noise and generating a noise characteristic map specifically includes: receiving motion state data output by an acceleration sensor and generating a motion characteristic vector; performing motion noise compensation on the ambient audio data based on the motion characteristic vector to obtain corrected ambient noise data; storing the corrected ambient noise data in a noise characteristic database according to frequency bands, and generating a noise characteristic map including a noise characteristic index table.

[0015] In the above embodiment, the dual-headphone system performs motion compensation for ambient noise by combining acceleration sensor data. The system establishes a complete noise feature database, enabling the system to accurately distinguish and filter out interference caused by user movement, thereby improving the accuracy of human voice detection.

[0016] In combination with some embodiments of the first aspect, in some embodiments, after the step of increasing the audio signal gain in the direction in which the head is facing when the matching angle is less than a preset threshold, the method also includes: receiving the angular velocity data stream output by the gyroscope and calculating the head rotation parameters; generating an adaptive adjustment curve for the audio gain based on the head rotation parameters; applying the adaptive adjustment curve to the audio signal processing channel to adjust the gain coefficient of the audio signal.

[0017] In the above embodiment, the dual-headphone system generates a dynamic adjustment curve of the audio gain by real-time analysis of the head rotation parameters, thereby achieving a smooth transition of the audio gain and ensuring that the user can obtain a continuous and natural auditory experience during the head turning process.

[0018] In combination with some embodiments of the first aspect, in some embodiments, the step of receiving the angular velocity data stream output by the gyroscope and calculating the head rotation parameters specifically includes: reading the three-axis angular velocity data stream of the head gyroscope, and converting the three-axis angular velocity data stream into an Euler angle vector; performing Kalman filtering on the Euler angle vector to obtain head posture data; calculating the head rotation direction and angular velocity value based on the head posture data to generate head rotation parameters.

[0019] In the above embodiment, the dual-headphone system can obtain high-precision head posture information by performing Kalman filtering on the gyroscope data, providing a reliable reference for audio gain adjustment and ensuring that the system response is highly synchronized with the user behavior.

[0020] In a second aspect, an embodiment of the present application provides a dual-headphone system, comprising: one or more processors and a memory; the memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code comprising computer instructions, the one or more processors calling the computer instructions to cause the dual-headphone system to execute the method described in the first aspect and any possible implementation of the first aspect.

[0021] In a third aspect, an embodiment of the present application provides a computer program product comprising instructions. When the computer program product is run on a dual-headphone system, the dual-headphone system executes the method described in the first aspect and any possible implementation of the first aspect.

[0022] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium comprising instructions. When the instructions are executed on a dual-headphone system, the dual-headphone system executes the method described in the first aspect and any possible implementation of the first aspect.

[0023] It is understood that the dual-headphone system provided in the second aspect, the computer program product provided in the third aspect, and the computer storage medium provided in the fourth aspect are all used to execute the methods provided in the embodiments of this application. Therefore, the beneficial effects that can be achieved can be referenced to the beneficial effects of the corresponding methods and will not be repeated here.

[0024] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages: 1. Due to the use of binaural microphones to collect ambient audio in real time and the technical features of head posture detection, combined with spectrum analysis and voice feature extraction, the sound source location is determined by calculating the time difference and intensity difference of the binaural signals, and finally intelligent gain adjustment is performed based on the head orientation. Therefore, the system can automatically identify and enhance the sound signals in the direction of the user's current focus, effectively solving the problem in the existing technology that users need to manually trigger or voice control to hear the surrounding sounds clearly, thereby realizing a fully automatic, natural and smooth hearing enhancement experience, allowing users to conveniently interact with the surrounding environment while wearing headphones.

[0025] 2. Due to the technical feature of analyzing the environmental noise characteristics and generating a noise feature map before audio collection, and then dynamically calculating and adjusting the microphone gain control parameters based on this, the system can adaptively adjust the sensitivity of the signal acquisition channel in different environments to keep it in the optimal working range at all times. This effectively solves the problem of signal distortion or weak signal caused by fixed gain parameters in the existing technology, and thus achieves the goal of obtaining high-quality audio signals in various complex noise environments, significantly improving the accuracy of subsequent sound source identification and positioning.

[0026] 3. Due to the technical feature of calculating head rotation parameters in real time based on gyroscope data, generating an adaptive adjustment curve for audio gain based on this, and then applying it to the audio signal processing channel, the system can achieve smooth dynamic adjustment of audio gain, effectively solving the auditory discomfort problem caused by sudden changes in gain adjustment in the existing technology, and thus achieving an audio enhancement effect that naturally transitions with head movement, ensuring that users can obtain a continuous and comfortable listening experience even when turning their heads, thereby improving the user experience of the device. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a flow chart of a method for adjusting the sound effects of two headphones according to an embodiment of the present invention; Figure 2 This is another flowchart of the dual-headphone sound effect adjustment method according to an embodiment of the present application; Figure 3 This is a schematic diagram of the physical device structure of the dual-headphone system in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The terms used in the following examples of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification of the present application, the singular expressions "a", "an", "above", "the", and "this" are intended to include plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in the present application refers to any or all possible combinations of one or more of the listed items.

[0029] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to imply or suggest relative importance or implicitly indicate the number of the technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of this application, unless otherwise specified, "plurality" means two or more.

[0030] For ease of understanding, the application scenarios of the embodiments of the present application are introduced below.

[0031] Xiao Li is a freelancer who often works in cafes. He's used to wearing noise-canceling headphones to stay focused while listening to music. However, this often causes him to miss important social interactions. Sometimes he doesn't even notice when a colleague greets him, waiters need to tap him on the shoulder when delivering coffee, and he can't hear his friends calling him from several tables away. He's forced to constantly take off his headphones to look around or turn the music volume down. This situation is common in places like libraries and shared office spaces, highlighting the conflict between modern people's pursuit of concentration and maintaining social accessibility. While using noise-canceling headphones provides good sound insulation, it can also easily lead to excessive isolation from the surrounding environment, affecting normal social interactions and environmental perception.

[0032] Related technologies use touch sensors and voice wake-up mechanisms to enable interactive control of headphones and external sounds. These solutions rely on the user actively triggering or issuing voice commands to activate the ambient sound transmission function. The following describes a scenario using the dual-headphone sound effect adjustment method described in related technologies.

[0033] Xiao Wang bought a pair of Bluetooth headphones with a "transparency mode." This headset allows you to turn on the ambient sound transmission function by long-pressing the touchpad, allowing outside sounds to enter. However, in actual use, when he was watching a video on the subway and someone suddenly spoke to him, he had to pause the video first, then find the touch position and long-press to switch modes, and then switch back after hearing the other person clearly. The whole process was quite cumbersome. Another headset that supports voice control requires saying commands like "Hey, turn on transparency mode," but using voice commands in public places is very abrupt and is often interfered with by surrounding noise, resulting in recognition failure. These solutions require users to actively operate and cannot achieve a natural and smooth interactive experience.

[0034] The dual-headphone sound effect adjustment method in the embodiment of this application, by real-time acquisition and analysis of ambient audio signals, combined with head posture detection and adaptive gain control, achieves intelligent sound source tracking and automatic sound effect adjustment, eliminating the need for manual user operation and maintaining a natural and smooth interactive experience. The following describes scenarios in which the dual-headphone sound effect adjustment method in this application is used.

[0035] After adopting the dual-headphone system of this application, Xiao Zhang's experience in the office has been significantly improved. When he is concentrating on his work, the noise reduction function can effectively isolate the ambient noise such as printers and air conditioners. When a colleague walks up to him and says, "Hey, do you want to go to lunch together?", the headset automatically identifies the direction of the voice and appropriately enhances the sound in that direction. When he turns his head to look at his colleague, the system further optimizes the volume of the conversation so that he can hear the content of the conversation clearly. The whole process is natural and smooth, without any manual operation, and it will not interrupt his work status or miss important social interactions. When he responds to the conversation, the system will also automatically maintain an appropriate volume level to prevent him from unconsciously raising his speaking volume.

[0036] It can be seen that the dual-earphone sound effect adjustment method in the embodiment of the present application can not only realize the intelligent transmission of ambient sound, but also effectively solve the problems of manual triggering and response delay in traditional solutions, thereby realizing a fully automatic, real-time and humanized auditory enhancement experience.

[0037] For ease of understanding, the following describes the process of the method provided by this implementation in combination with the above scenario. Figure 1 , which is a flow chart of the dual-headphone sound effect adjustment method in an embodiment of the present application.

[0038] S101 : collecting ambient audio signals through binaural microphones, and performing spectrum analysis to obtain frequency distribution characteristics of the ambient audio signals.

[0039] Among them, binaural microphones refer to a pair of directional acoustic sensors installed on the left and right earphones, which are used to convert sound wave signals into electrical signals; environmental audio signals refer to acoustic signals generated by various sound waves in the surrounding environment propagating in the air, including human voices, background noise, etc.; spectrum analysis refers to the mathematical processing method of converting time domain audio signals into frequency domain for analysis; frequency distribution characteristics represent the energy distribution of audio signals in different frequency components, including the amplitude, phase and other information of each frequency component.

[0040] When the user wears headphones and turns on the audio processing function, the dual-headphone system will continue to perform the environmental sound collection and analysis process. Specifically, the dual-headphone system first uses the left and right ear microphones to synchronously collect environmental sounds at a preset sampling rate (such as 48kHz), and converts the analog sound signal into a digital audio data stream. The collected audio data is then preprocessed, including signal amplification, noise suppression, and digital filtering to improve signal quality. Next, the dual-headphone system performs a fast Fourier transform (FFT) on the preprocessed audio data to convert the time domain signal to the frequency domain. Finally, by calculating the power spectral density of the frequency domain signal, a frequency distribution diagram that characterizes the characteristics of the environmental sound is obtained, which contains the energy distribution information of each frequency band.

[0041] In some embodiments, spectrum analysis and processing of ambient audio signals can be achieved in a variety of ways: Optionally, the dual-headphone system uses a short-time Fourier transform (STFT) method to first divide the audio signal into several overlapping time windows (e.g., 20ms window length, 10ms overlap length), adds a Hanning window to the data of each window, performs an FFT operation, and finally combines the spectrum results of each window to obtain a time-frequency diagram, thereby achieving analysis of the frequency characteristics of ambient sound over time; Optionally, the dual-headphone system uses a wavelet transform method to decompose the audio signal through wavelet functions at different scales and time positions to obtain multi-resolution time-frequency analysis results, thereby extracting more detailed frequency feature information. It is understandable that other digital signal processing methods can also be used to achieve spectrum analysis of ambient audio, such as Mel-frequency cepstral coefficient (MFCC) analysis, etc., which are not limited here.

[0042] S102 : Based on a preset human voice frequency range, extract an audio signal that meets the human voice characteristics from the frequency distribution characteristics as a target audio.

[0043] Among them, the preset human voice frequency range refers to the frequency band range covered by the fundamental frequency and main harmonic components of human voice, usually 85-255Hz (male voice fundamental frequency) and 165-255Hz (female voice fundamental frequency) and their harmonic frequency bands; human voice characteristics represent the typical characteristics of human voice signals in terms of spectrum, energy distribution, time domain envelope, etc.; target audio refers to the audio clip suspected of containing human voice separated from the ambient audio signal.

[0044] After obtaining the frequency distribution characteristics of the ambient audio, the dual-headphone system immediately performs vocal extraction. Specifically, based on a preset vocal frequency template, the dual-headphone system first locates frequency components within the spectrum that match the fundamental frequency range of vocal sound. It then analyzes the harmonic structure of these frequency components to check for any harmonic frequency relationships unique to vocal sound. It then calculates the energy distribution and time-domain modulation characteristics of these frequency bands to further verify their conformity to the acoustic characteristics of vocal sound. Finally, it applies a bandpass filter to the qualified frequency bands to extract the target audio signal containing vocal information.

[0045] In some embodiments, human voice feature extraction can be achieved through a variety of methods: Optionally, the dual-headphone system uses a resonance peak analysis method to identify the resonance peaks in the spectrum (near 500Hz, 1500Hz, and 2500Hz) and their bandwidth and energy ratio characteristics, combined with the vocal tract model to determine whether it is a human voice; Optionally, the dual-headphone system uses a deep neural network model to analyze the spectrum features through a pre-trained sound classification network to achieve accurate separation of human voice and non-human voice. It is understandable that other acoustic feature analysis methods can also be used to achieve human voice recognition and extraction, such as methods based on modulation spectrum analysis or acoustic event detection, which are not limited here.

[0046] It's important to note that neural network models for sound classification first perform a short-time Fourier transform on the input audio signal, obtaining a time-frequency representation S(t, f). For each time window t and frequency component f, features are extracted through multiple layers of convolution. Each convolution operation can be expressed as F_l(x) = σ(W_l * x + b_l), where W_l is the convolution kernel weight matrix, b_l is the bias term, and σ is the Reluctant Unified Unit (ReLU) activation function. After L layers of convolution, the feature map undergoes global average pooling to obtain a fixed-dimensional feature vector v. Finally, a fully connected layer and a softmax function are used to calculate the output probability p(c|v), representing the probability that the input audio belongs to each category c. During training, optimization is performed using the cross-entropy loss function L = -Σy_c log(p(c|v)), where y_c is the ground-truth label. For example, when the system captures a mixed audio segment containing construction site noise and human voice, the model can accurately identify the human voice component, providing a foundation for subsequent audio enhancement.

[0047] S103: Calculate the time difference and intensity difference of the target audio between the left and right ears to obtain the direction and relative distance of the human voice source in the environment.

[0048] Among them, the time difference refers to the time delay between the sound signal emitted by the same sound source reaching the microphones of the left and right ears, with a maximum value of about 0.6-0.8 milliseconds; the intensity difference indicates the difference in sound pressure levels of the sound signal between the left and right ears; the direction of the human voice source refers to the spatial azimuth angle of the speaker relative to the listener's head; the relative distance refers to the approximate distance estimate from the sound source to the listener.

[0049] After acquiring the target audio, the dual-headphone system calculates the sound source location using the fundamental principles of human auditory localization. Specifically, the dual-headphone system first synchronizes the time and amplitude of the target audio signals for the left and right ears. It then calculates the time delay between the two signals through cross-correlation analysis, while also measuring the effective sound pressure level difference between the left and right ear signals. Next, based on the head-related transfer function (HRTF) model, the time and intensity differences are mapped to spatial angular coordinates. Finally, the relative distance to the sound source is estimated using an acoustic attenuation model.

[0050] In some embodiments, the sound source location can be calculated in a variety of ways: Optionally, the dual-headphone system uses a generalized cross-correlation algorithm (GCC-PHAT) to improve the accuracy of delay estimation through frequency domain weighting and phase transformation, and constructs a three-dimensional sound source localization model based on sound pressure level differences; Optionally, the dual-headphone system uses an adaptive filter bank method to improve positioning accuracy through time delay analysis of multiple sub-band signals, and achieves stable direction tracking by combining multi-frame timing information. It is understood that other sound source localization algorithms can also be used to estimate direction and distance, such as end-to-end localization methods based on deep learning, which are not limited here.

[0051] It should be noted that the system uses a spatial sound field analysis method based on spherical harmonic expansion to calculate the sound source position. First, the three-dimensional sound field is described as p(r, θ, φ) = Σ(n, m)A_nm·Y_nm(θ, φ)·h_n(kr), where Y_nm represents the spherical harmonics, h_n represents the first-kind spherical Hankel function, and k represents the wave number. The sound pressure signals collected by the left and right ear microphones are analyzed using the MUSIC algorithm to construct the spatial covariance matrix R = E[X(t)X^H(t)], where X(t) represents the observation vector. After performing eigenvalue decomposition on R, the orthogonality between the signal and noise subspaces is exploited to calculate the spatial spectrum function P(θ, φ) = 1 / |a^H(θ, φ)·V_N·V_N^H·a(θ, φ)|, where a(θ, φ) represents the steering vector and V_N represents the noise subspace matrix. The sound source direction is determined by searching for the peak of P(θ, φ), and the distance is estimated using a sound pressure attenuation model. For example, in a conference room scenario, the system can accurately locate the main speaker even when multiple people are speaking at the same time and background noise is interfering.

[0052] S104: Acquire head posture sensor data, and calculate a matching angle between the user's head orientation and the direction of a human voice source based on the head posture sensor data.

[0053] Among them, head posture sensor data refers to the user's head spatial posture information collected by sensors such as gyroscopes and accelerometers; head orientation indicates the pointing direction of the user's head in three-dimensional space; matching angle refers to the angle between the head orientation and the direction of the sound source.

[0054] The dual-headphone system continuously monitors the user's head movement while tracking the sound source in real time. Specifically, the dual-headphone system first reads the raw data from the gyroscope and accelerometer and calculates the accurate head posture angle using a sensor fusion algorithm. The head posture angle is then converted to the same spatial reference system as the sound source coordinate system. Next, the angle between the head orientation vector and the sound source direction vector is calculated. Finally, the calculated result is smoothed in the time domain to eliminate the effects of instantaneous jitter.

[0055] In some embodiments, head orientation detection and angle calculation can be achieved through a variety of methods: Optionally, the dual-headphone system uses an extended Kalman filter algorithm to fuse multi-sensor data to estimate head posture and achieve stable spatial vector calculations through quaternion operations; Optionally, the dual-headphone system uses a complementary filtering method, combining the advantages of gyroscopes and accelerometers to achieve fast response and long-term stable posture tracking. It is understood that other posture estimation algorithms can also be used to achieve head orientation detection, such as methods based on particle filtering or deep learning, which are not limited here.

[0056] S105: When the matching angle is smaller than a preset threshold, increase the audio signal gain in the direction in which the head is facing.

[0057] Among them, the preset threshold refers to the angular limit for judging whether the user is paying attention to the sound source, which is usually set within the range of 15-30 degrees; the audio signal gain indicates the amplification factor of the audio signal; the audio signal in the direction of the head refers to the sound signal from the direction the user is currently paying attention to.

[0058] The dual-headphone system activates directional gain control when it detects that the user's head orientation is roughly aligned with the sound source. Specifically, the dual-headphone system first compares the matching angle with a preset threshold. If the angle is less than the threshold, the system calculates a smoothed gain curve based on the angle. The audio signal in the target direction is then dynamically amplified based on the gain curve. Finally, a sound field reconstruction algorithm is used to combine the enhanced signals into a stereo output.

[0059] In some embodiments, directional gain control can be implemented in a variety of ways: Optionally, a dual-headphone system can employ adaptive beamforming technology to selectively enhance sounds from a target direction by adjusting the spatial filter coefficients of the left and right ear microphones. Optionally, a dual-headphone system can employ frequency-domain selective gain control, employing different gain strategies for different frequency bands to maintain speech clarity while suppressing ambient noise. It is understood that other audio processing methods, such as neural network-based adaptive gain algorithms, can also be employed to implement directional gain control, which is not limited here.

[0060] It's important to note that the sound field reconstruction algorithm is based on wave field synthesis theory and mathematically models the spatial propagation characteristics of sound waves. First, the algorithm uses the head-related transfer function (HRTF) to create a mathematical description of the three-dimensional sound field. This function captures the complex interactions between sound waves, the ear, and the head at different angles of incidence. Then, using spherical harmonic expansion, the three-dimensional sound field is decomposed into spatial modes of varying orders, each corresponding to a specific spatial feature in the sound field. Next, the algorithm calculates the speaker drive signals required for reconstruction based on a least-squares optimization criterion, ensuring that the reconstructed sound field, at the listener's ear, closely resembles the original sound field characteristics. In practice, the algorithm also applies compensation filters in the frequency domain to improve the reconstruction, taking into account the nonlinearities of the headphone's acoustic characteristics. For example, when the user's head is facing the speaker, the algorithm accurately calculates the time and intensity differences between the left and right ears, creating a natural sense of sound source localization.

[0061] S106: When the user's conversational voice is detected, maintain the audio signal gain unchanged.

[0062] The user's conversational voice refers to the voice signal emitted by the user himself; maintaining the audio signal gain unchanged means temporarily freezing the gain adjustment mechanism to maintain the current gain setting.

[0063] When the user begins speaking, the dual-headphone system temporarily enters a gain hold state. Specifically, the dual-headphone system first identifies the user's speaking state through characteristic analysis of bone conduction signals or voiceprints. It then saves the current gain parameters as temporary values. Next, the system suspends the dynamic gain adjustment algorithm, maintaining the gain settings of the signal processing channels. Finally, when the user stops speaking, the system resumes normal gain adjustment.

[0064] In some embodiments, user voice detection and gain maintenance can be achieved through a variety of methods: Optionally, the dual-headphone system uses bone conduction sensors and voice activity detection algorithms to accurately identify the user's speaking state and control the pause and resume of gain adjustment through a state machine; Optionally, the dual-headphone system uses voiceprint recognition technology to determine whether it is the user's voice through real-time analysis of the acoustic characteristics of the microphone signal and control the operating state of the gain processing module accordingly. It is understood that other voice detection methods can also be used to achieve speaking state recognition and gain control, such as end-to-end voice detection methods based on deep learning, which are not limited here.

[0065] It's important to note that the acquisition and processing of bone conduction signals involves specialized acoustic sensing technology. The dual-earphone system incorporates a piezoelectric bone conduction sensor at the point where the earphone housing contacts the body. This sensor captures vibration signals transmitted through the skull. When the user speaks, the mechanical vibrations generated by the vocal cords through skull conduction are converted by the sensor into electrical signals. These raw signals are first conditioned by a preamplifier and then passed through a bandpass filter to remove low-frequency ambient vibrations and high-frequency electromagnetic interference. The system then uses an adaptive filtering algorithm to isolate the characteristic frequency bands associated with speech and an envelope detection algorithm to identify the start and end of speech. During signal processing, the system also incorporates acoustic models to compensate for the frequency response of the bone conduction signal to extract more accurate speech features. For example, by analyzing the energy curve of the bone conduction signal, the system can accurately determine whether the user is engaged in conversation, providing a reliable trigger for gain control.

[0066] In addition, after completing the basic sound enhancement and maintenance functions, the dual-earphone system also needs to perform adaptive sound optimization processing. Specifically, the dual-earphone system first builds a personalized acoustic compensation curve based on the user's ear canal characteristics and auditory model. This curve optimizes the spatial perception of sound by applying different gain and delay parameters in different frequency bands. Then, the system monitors the changes in ambient noise in real time and dynamically adjusts the parameter settings of the noise suppression algorithm to ensure that the enhanced voice signal has good clarity. At the same time, the system will also analyze the user's usage habits and preferences, and gradually establish a personalized sound configuration database for optimizing subsequent audio processing strategies. Finally, the system outputs the processed audio signal to the headphone speaker through a digital-to-analog converter, and monitors the volume level in real time during the output process to ensure that there is no damage to the user's hearing.

[0067] This dual-earphone system implements an adaptive voice activity detection mechanism based on the ratio of long-term and short-term energy and the zero-crossing rate. Short-term energy is calculated using an exponential weighting scheme: E_s(n) = λ·E_s(n-1) + (1-λ)·|x(n)|^2, while long-term energy is smoothed using a larger factor: E_l(n) = γ·E_l(n-1) + (1-γ)·|x(n)|^2, where λ and γ are smoothing coefficients. By analyzing the energy ratio R(n) = E_s(n) / E_l(n) and combining it with the zero-crossing rate Z(n) = Σ|sign(x(n))-sign(x(n-1))| / 2, a dual-threshold decision rule is constructed: voice activity is detected when R(n) > η_1 and Z(n) > η_2. The threshold parameters η_1 and η_2 are adaptively updated based on noise statistics. From the detected speech fragments, voiceprint features based on linear prediction coefficients are further extracted and speaker recognition is performed using a Gaussian mixture model to distinguish the user's own voice from that of others. This method can accurately detect the user's conversation status and maintain a high detection accuracy even in noisy environments.

[0068] The following is a more detailed description of the process of the method provided by this implementation. Figure 2 , is another flow chart of the dual-headphone sound effect adjustment method in an embodiment of the present application.

[0069] S201: Collect sound pressure level data of ambient noise and generate a noise characteristic spectrum.

[0070] Among them, environmental noise refers to all background sounds except the target human voice, including mechanical noise, natural noise and other interfering sounds; sound pressure level data represents the pressure change caused by sound waves propagating in the air, usually in decibels (dB); noise characteristic map refers to a two-dimensional data matrix that describes the distribution characteristics of environmental noise in the time-frequency domain, which contains information such as the noise's spectral characteristics, time-varying characteristics and energy distribution; data acquisition refers to the process of converting analog sound signals into digital signals.

[0071] The dual-headphone system performs environmental noise analysis periodically during startup and initialization. Specifically, the dual-headphone system first activates the high-precision sampling mode of the left and right ear microphones, continuously acquiring ambient sound pressure signals at a sampling rate of at least 44.1kHz. The collected raw signals are then preprocessed, including DC offset removal, anti-aliasing filtering, and signal normalization. Next, the system divides the preprocessed signals into multiple time windows (typical window length is 20ms), performs a short-time Fourier transform on each window, and obtains a time-varying spectrum. The system then calculates the energy value of each time-frequency point and converts it to a decibel scale through logarithmic operations. Finally, the system organizes these time-frequency energy data into a two-dimensional matrix to form a complete noise signature map.

[0072] In some embodiments, the generation of noise feature maps can be achieved in a variety of ways: Optionally, the dual-headphone system uses a sub-band decomposition method, first decomposing the audio signal into multiple frequency bands through a polyphase filter group, calculating the short-time energy envelope and statistical characteristics of each frequency band respectively, and then combining these features into a multi-dimensional feature vector, and finally reducing the dimension through principal component analysis to obtain a compact noise feature representation; Optionally, the dual-headphone system uses an acoustic scene classification method, extracting deep features of ambient sound through a pre-trained deep neural network, combining Mel frequency features and modulation spectrum features, and constructing a noise feature description containing time-frequency dynamic information. It is understandable that other signal processing methods can also be used to extract and represent noise features, such as wavelet packet decomposition or auditory perceptual coding, which are not limited here.

[0073] In some embodiments, the dual-headphone system will store the ambient noise to facilitate subsequent feature identification, that is, the dual-headphone system will receive the motion state data output by the acceleration sensor and generate a motion feature vector; based on the motion feature vector, the ambient audio data will be compensated for motion noise to obtain corrected ambient noise data; the corrected ambient noise data will be stored in a noise feature database according to frequency bands to generate a noise feature map containing a noise feature index table.

[0074] Among them, accelerometer refers to an electronic device that measures the acceleration of an object; motion state data refers to a multidimensional data sequence that describes motion characteristics; motion feature vector refers to a mathematical description that characterizes the motion pattern; motion noise compensation refers to the process of eliminating interference signals caused by motion; corrected ambient noise data represents a pure noise signal that has been processed with motion compensation; frequency band division refers to the process of classifying sound signals according to frequency range; the noise feature database is used to store and manage noise feature templates; and the feature index table refers to a data structure used to quickly retrieve noise features.

[0075] When the dual-headphone system detects a change in motion state, it starts the motion noise processing process. Specifically, the dual-headphone system first reads the three-axis acceleration sensor data at a sampling rate of 200Hz and removes the high-frequency vibration component through low-pass filtering. The motion statistical characteristics of the filtered acceleration data are then calculated, including time domain characteristics and spectral characteristics such as mean, variance, and kurtosis. Next, the system establishes a motion noise model and estimates the impact of motion on the audio signal through an adaptive algorithm. Subsequently, the system compensates the ambient audio data to eliminate the interference introduced by motion. Finally, the system divides the processed noise data into frequency bands according to the critical band theory to construct a hierarchical noise feature database.

[0076] In some embodiments, motion noise processing and feature extraction can be achieved through a variety of methods: Optionally, the dual-headphone system uses a motion compensation method based on Kalman filtering, predicts the dynamic characteristics of motion noise by establishing a motion state space model, and combines multi-sensor data fusion technology to achieve accurate noise estimation and elimination; Optionally, the dual-headphone system uses a deep learning framework to model the temporal dependencies of motion sequences through recurrent neural networks to achieve adaptive separation of motion noise. It is understood that other signal processing methods can also be used to achieve motion noise compensation, such as techniques based on empirical mode decomposition or wavelet packet transform, which are not limited here.

[0077] S202: Calculate a gain control parameter of the microphone according to the noise characteristic spectrum, and write the gain control parameter into a microphone control register.

[0078] Among them, the gain control parameter refers to the digital quantization value used to adjust the microphone sensitivity, including preamplifier gain, digital gain, etc.; the microphone control register represents the digital storage unit used to store and control the working status of the microphone; the write operation refers to the process of updating the parameter value to the hardware register through the digital interface; parameter calculation refers to the mathematical process of determining the optimal gain value based on the noise characteristics.

[0079] After obtaining the complete noise characteristic map, the dual-headphone system immediately performs gain parameter optimization calculations. Specifically, the dual-headphone system first analyzes the energy distribution of each frequency band in the noise characteristic map and calculates the overall noise level and peak noise. Then, based on the preset signal dynamic range requirements and combined with the electroacoustic characteristic curve of the microphone, a gain optimization objective function is established. Next, the system uses a numerical optimization algorithm to solve the optimal gain parameter combination to maximize the signal-to-noise ratio of signal acquisition in the current noise environment. Subsequently, the system converts the calculated gain parameters into a hardware register format and writes the parameter values into the control register of the microphone chip through the I2C or SPI interface. Finally, the system verifies whether the parameter write is successful and confirms the parameter effectiveness.

[0080] In some embodiments, the optimization calculation and update of gain parameters can be achieved through a variety of methods: Optionally, the dual-headphone system uses an adaptive control algorithm to dynamically adjust the gain parameters based on the time-varying characteristics of the noise characteristics, and optimizes the signal acquisition quality in real time through a feedback loop while incorporating system latency and stability constraints; Optionally, the dual-headphone system uses a machine learning-based parameter prediction method to establish a mapping relationship from noise characteristics to optimal gain parameters through training data, achieving fast and accurate parameter calculation and update. It is understood that other parameter optimization methods can also be used to achieve gain control, such as fuzzy control or neural network adaptive control technologies, which are not limited here.

[0081] S203: Adjust the sensitivity of the signal acquisition channel based on the gain control parameter so that the range of the acquired audio signal is within a preset optimal range.

[0082] Among them, the signal acquisition channel refers to the complete signal processing link from the microphone to the analog-to-digital converter; sensitivity refers to the conversion ratio between the input sound pressure signal and the output electrical signal; the preset optimal interval refers to the ideal amplitude range to ensure signal quality, usually 60%-80% of the full scale of the analog-to-digital converter; the signal range refers to the peak variation range of the audio signal.

[0083] After completing the gain parameter writing, the dual-headphone system immediately enters the signal channel adjustment stage. Specifically, the dual-headphone system first verifies whether the gain control parameters are effective and detects the DC bias and AC components of the microphone output signal. It then performs end-to-end gain calibration on the signal acquisition channel, including joint optimization of the analog pre-stage gain, digital gain, and compensation filter. Next, the system collects test audio data and analyzes whether the dynamic range of the signal falls within the preset optimal range. If it is found that the signal amplitude exceeds or does not reach the target range, the system will automatically fine-tune the gain parameters until the requirements are met. Finally, the system saves the optimized parameter configuration as the baseline value for the current operating mode.

[0084] In some embodiments, the sensitivity adjustment of the signal channel can be achieved in a variety of ways: optionally, the dual-headphone system uses a piecewise linear mapping method to dynamically adjust the slope and intercept of the gain curve according to the intensity characteristics of the input signal, thereby achieving adaptive compression and expansion of the signal dynamic range while maintaining the relative relationship of the signals unchanged; optionally, the dual-headphone system uses an automatic gain control algorithm to achieve stable regulation of the signal amplitude through envelope detection and feedback control, and introduces soft threshold smoothing to avoid sudden gain changes. It is understandable that other signal conditioning methods can also be used to achieve sensitivity control, such as nonlinear compression or adaptive quantization techniques, which are not limited here.

[0085] S204: Collect ambient audio signals through binaural microphones, and perform spectrum analysis to obtain frequency distribution characteristics of the ambient audio signals.

[0086] Referring to step S101 , the dual-headphone system determines the frequency distribution characteristics of the ambient sound.

[0087] In some embodiments, the dual-headphone system will perform noise reduction processing on the ambient sound, that is, the dual-headphone system will synchronously collect the ambient audio signal through the left and right ear microphones and generate a digital audio data stream; the digital audio data stream includes binaural sound pressure signals and timestamp information; the digital audio data stream is denoised to obtain a noise-reduced audio signal; the signal-to-noise ratio of the noise-reduced audio signal is higher than that of the ambient audio signal; the noise-reduced audio signal is fast Fourier transformed to generate a frequency distribution feature containing frequency domain data; the frequency domain data contains frequency components and amplitude information.

[0088] Among them, synchronous acquisition refers to the process of simultaneous sound sampling by the left and right ear microphones according to the same clock signal; the digital audio data stream represents the discrete digital signal sequence after analog-to-digital conversion; the binaural sound pressure signal refers to the quantized value of the sound wave pressure change collected by the left and right ear microphones; the timestamp information is used to mark the precise sampling moment of each sampling point; denoising refers to the technical means of reducing background noise through digital signal processing algorithms; the signal-to-noise ratio represents the ratio of useful signal power to noise power; the fast Fourier transform refers to the mathematical operation method of converting time domain signals to frequency domain; the frequency domain data represents the component characteristics of the audio signal at different frequencies; the frequency component refers to the various frequency components contained in the signal; the amplitude information is used to indicate the intensity of each frequency component.

[0089] After completing power-on initialization, the dual-headphone system enters a continuous audio acquisition state. Specifically, the dual-headphone system first configures the sampling parameters of the left and right microphones, including the sampling rate (typically 48kHz), quantization resolution (typically 24 bits), and gain settings. It then starts a synchronous sampling clock, controlling the two microphones to capture ambient sound in strict synchronization and adding precise timestamp information to each sampling point. Next, the dual-headphone system applies an adaptive noise cancellation algorithm to the collected raw data stream, estimating the ambient noise characteristics and constructing an anti-phase noise filter to offset background noise. The system then frames the noise-reduced audio data (typically 20ms long), windowing each frame and performing a 2048-point Fast Fourier Transform. Finally, the system calculates the magnitude and phase spectra of the transformed results to generate a frequency distribution diagram describing the sound characteristics.

[0090] In some embodiments, the collection and processing of audio signals can be achieved in a variety of ways: Optionally, the dual-headphone system adopts a multi-stage adaptive filtering scheme, first using a Wiener filter to pre-process the original signal, then dividing the signal into multiple frequency bands through sub-band decomposition, and independently applying the adaptive filtering algorithm of the minimum mean square error criterion to each frequency band, and finally reconstructing the processed signals of each frequency band to obtain high-quality noise reduction results; Optionally, the dual-headphone system uses a deep neural network noise reduction method, extracting audio features through a pre-trained multi-layer convolutional neural network, combining long short-term memory networks to model temporal dependencies, and achieving end-to-end noise suppression. It is understandable that other signal processing methods can also be used to achieve audio collection and noise reduction, such as algorithms based on independent component analysis or sparse representation, which are not limited here.

[0091] It should be noted that the system employs an adaptive noise cancellation strategy based on frequency domain decomposition. The input signal is first decomposed into M subbands x_m(n), where m = 1, ..., M, by a polyphase filter bank. Each subband signal is processed by an independent adaptive filter, and the filter coefficients w_m(n) are updated using the Normalized Least Mean Squared Error (NLMS) algorithm: w_m(n+1) = w_m(n) + μ_m·e_m(n)·x_m(n) / (||x_m(n)||^2+δ), where e_m(n) is the error signal, μ_m is the adaptive step size, and δ is the regularization parameter. The step size parameter is dynamically adjusted based on the local statistical characteristics of the subband signal: μ_m(n) = β·(1-exp(-α·|x_m(n)|^2 / σ_x^2)), where σ_x^2 is the input signal variance, and α and β are control parameters. The processed signals from each subband are combined through a perfect reconstruction filter and finally post-processed using a psychoacoustic masking model to ensure the natural and clear sound of the de-noised speech. This approach is effective for various types of ambient noise, such as the low-frequency noise in subway cars and the crowd noise in shopping malls.

[0092] S205 : Based on a preset human voice frequency range, extract an audio signal that meets the human voice characteristics from the frequency distribution characteristics as the target audio.

[0093] Referring to step S102 , the dual-headphone system extracts target audio.

[0094] S206: Calculate the time difference and intensity difference of the target audio between the left and right ears to obtain the direction and relative distance of the human voice source in the environment.

[0095] Referring to step S103 , the dual-headphone system calculates the direction and relative distance of the human voice source in the environment.

[0096] In some embodiments, the dual-headphone system will perform map construction and positioning, that is, the dual-headphone system will perform cross-correlation operations on the target audio between the left and right ears to obtain a cross-correlation time series; based on the peak position of the cross-correlation time series, the time delay of the target audio between the left and right ears is extracted, and the spatial transfer function is calculated in combination with the sound pressure level difference to generate spatial coordinate data of the target audio; a sound source localization map is constructed according to the spatial coordinate data, and the direction and relative distance of the human voice source are determined based on the sound source localization map.

[0097] Among them, the cross-correlation operation represents a mathematical operation that calculates the change in the similarity of two signals over time offset; the cross-correlation time series refers to the time function that describes the degree of correlation between the left and right ear signals; the peak position is used to indicate the time offset when the two signals are most similar; the time delay represents the time difference between the sound source and the left and right ears; the sound pressure level difference refers to the logarithmic difference in the sound intensity received by the left and right ears; the spatial transfer function represents the description of the acoustic characteristics of sound during spatial propagation; the spatial coordinate data is used to represent the position information of the sound source in three-dimensional space; the sound source localization map refers to the probability density function that describes the spatial distribution of the sound source; and the relative distance represents the spatial distance from the sound source to the center of the listener's head.

[0098] After acquiring the target audio signals for the left and right ears, the dual-headphone system immediately performs sound source localization calculations. Specifically, the dual-headphone system first performs time domain alignment and amplitude normalization on the left and right ear signals to ensure a consistent time base. The processed signals are then segmented into short time frames (typically 32ms in length and 16ms in frame shift), and a normalized cross-correlation function is calculated for each frame. Next, the system uses interpolation and peak detection algorithms to precisely locate the maximum value of the cross-correlation function, obtaining a sub-sampling-accurate time delay estimate. Subsequently, the system incorporates a head-related transfer function model to convert the time delay and sound pressure level difference into azimuth and elevation angles. Finally, the system calculates the spatial position and relative distance of the sound source using triangulation principles and an acoustic energy attenuation model, mapping the results to a probability distribution diagram in a spherical coordinate system.

[0099] In some embodiments, sound source localization and spatial parameter estimation can be achieved in a variety of ways: Optionally, the dual-headphone system adopts a generalized cross-correlation method, by applying a phase transformation weighting function in the frequency domain to enhance the cross-correlation peak, combined with an adaptive pre-filter to improve the robustness of the delay estimation, and then uses the maximum likelihood criterion to estimate the sound source position parameters; Optionally, the dual-headphone system uses an end-to-end positioning method based on deep learning, using a convolutional neural network to directly learn the mapping relationship of the sound source spatial information from the binaural audio features, to achieve real-time and accurate three-dimensional positioning. It is understandable that other sound source localization methods can also be used to achieve spatial parameter estimation, such as algorithms based on subspace decomposition or particle filtering, which are not limited here.

[0100] It should be noted that the key to processing binaural audio signals in a sound source localization neural network model lies in establishing a mapping from acoustic feature space to physical space. First, the cross-correlation function R(τ) = Σx_l(t)x_r(t+τ) between the left and right ear signals x_l(t) and x_r(t) is calculated to extract time delay features. Simultaneously, the power spectral densities P_l(f) and P_r(f) of the two channels are calculated to obtain frequency domain features. These features are processed by a multi-layer convolutional neural network. Each convolution operation includes modeling the spatial receptive field, namely, F(p) = σ(ΣW(q)·S(p+q)), where p represents the feature map position and q is the convolution kernel offset. The final output layer predicts the spherical coordinate parameters (r, φ, θ) of the sound source. Training uses a composite loss function L = α·L_angle + β·L_distance, where L_angle measures angular error and L_distance measures distance error. For example, in a noisy construction environment, when multiple workers speak at the same time, the model can accurately locate the position of each speaker, effectively supporting spatially selective audio enhancement.

[0101] S207: Acquire head posture sensor data, and calculate a matching angle between the user's head orientation and the direction of the human voice source based on the head posture sensor data.

[0102] Referring to step S104 , the dual-headphone system calculates the matching angle between the head orientation and the sound source direction.

[0103] S208: When the matching angle is smaller than a preset threshold, increase the audio signal gain in the direction in which the head is facing.

[0104] Referring to step S105 , the dual-headphone system increases the audio signal gain.

[0105] S209: Receive the angular velocity data stream output by the gyroscope and calculate the head rotation parameters.

[0106] Among them, the gyroscope refers to an inertial sensor that measures angular velocity; the angular velocity data stream represents a real-time data sequence describing the rotational motion of the head; the head rotation parameters include kinematic characteristic quantities such as rotation angle, angular velocity, and angular acceleration; and data reception refers to the process of reading sensor output data through a digital interface.

[0107] The dual-headphone system continuously monitors the user's head movement during operation. Specifically, it first configures the gyroscope's sampling rate (typically 100Hz) and measurement range (usually ±2000° / s), reading three-axis angular velocity data in real time via a digital interface. The raw data is then calibrated and filtered to eliminate the effects of zero bias and high-frequency noise. Next, the system calculates the head rotation angle through numerical integration and fuses accelerometer data using a Kalman filter algorithm for posture correction. Finally, the system calculates the acceleration characteristics of head rotation based on a kinematic model for subsequent audio gain adjustment.

[0108] In some embodiments, the calculation of head motion parameters can be achieved in a variety of ways: optionally, the dual-headphone system uses a quaternion integration method to integrate the angular velocity by constructing a quaternion motion equation to achieve accurate three-dimensional rotation posture calculation, and eliminates integral drift through gravity vector calibration; optionally, the dual-headphone system uses an adaptive multi-sensor fusion algorithm to combine data from the gyroscope, accelerometer, and magnetometer to build a complete head motion state estimator. It is understandable that other motion parameter estimation methods can also be used, such as algorithms based on complementary filtering or particle filtering, which are not limited here.

[0109] It should be noted that the dual-headphone system uses a multi-sensor adaptive fusion algorithm based on Kalman filtering to process head posture data. The state vector x = [θ, ω, a]^T contains the posture angle, angular velocity, and angular acceleration. The observation vector z comes from the gyroscope and accelerometer. The system dynamics equation is x(k+1) = F·x(k) + w(k), and the observation equation is z(k) = H·x(k) + v(k), where F is the state transition matrix, H is the observation matrix, and w and v are process noise and observation noise, respectively. To address the nonlinear characteristics of sensors, an iterative extended Kalman filter (IER) algorithm is employed. The prediction steps are x'(k+1) = f(x(k), u(k)) and P'(k+1) = F·P(k)·F^T+Q. The update step is K(k) = P'(k)·H^T·(H·P'(k)·H^T+R)^(-1), with x(k) = x'(k) + K(k)·(z(k) - h(x'(k))). P is the state covariance matrix, and Q and R are the process noise and observation noise covariance matrices, respectively. The filter parameters are adjusted sequentially and adaptively to ensure stable posture estimation even during strenuous user movements.

[0110] In some embodiments, the dual-headphone system will perform data conversion and filtering to ensure data accuracy, that is, the dual-headphone system will read the three-axis angular velocity data stream of the head gyroscope and convert the three-axis angular velocity data stream into an Euler angle vector; perform Kalman filtering on the Euler angle vector to obtain head posture data; calculate the head rotation direction and angular velocity value based on the head posture data to generate head rotation parameters.

[0111] Among them, the three-axis angular velocity data stream represents a real-time data sequence describing the rotation of the head around three spatial axes; the Euler angle vector refers to a mathematical representation method that uses three consecutive rotation angles to represent three-dimensional rotation; the Kalman filter refers to an optimal state estimation algorithm; the head posture data represents the position and orientation information of the head in space; the rotation direction is used to indicate the spatial direction of the head rotation; the angular velocity value refers to the instantaneous rate of head rotation; and the head rotation parameters contain a complete feature set that describes the head motion state.

[0112] The dual-headphone system continuously executes the attitude estimation process when head movement is detected. Specifically, the dual-headphone system first configures the gyroscope's operating parameters, including the sampling rate (typically 200Hz), range (±2000° / s), and digital filter settings. The collected angular velocity data is then temperature compensated and zero-bias calibrated to eliminate inherent sensor errors. Next, the system converts angular velocity into azimuth angle changes through numerical integration and uses quaternion representation to avoid gimbal lock. Subsequently, the system constructs an extended Kalman filter to perform sensor fusion between the gyroscope data and the accelerometer data to correct for integration drift. Finally, the system converts the filtered attitude data into an intuitive Euler angle representation and calculates the instantaneous characteristic parameters of the head movement.

[0113] In some embodiments, head posture estimation and parameter extraction can be achieved in a variety of ways: Optionally, the dual-headphone system uses a complementary filtering algorithm, which combines the high-frequency response characteristics of the gyroscope and the low-frequency stability of the accelerometer to construct an adaptively weighted complementary filter to achieve fast and accurate posture estimation, while introducing a dynamic calibration mechanism to compensate for sensor error accumulation; Optionally, the dual-headphone system uses a nonlinear state estimation method based on particle filtering, which simulates the probability distribution of head motion through Monte Carlo sampling to achieve robust tracking of complex motion states. It is understandable that other posture estimation methods can also be used to calculate head motion parameters, such as algorithms based on unscented Kalman filtering or adaptive fuzzy control, which are not limited here.

[0114] It should be noted that head motion can be simulated and calculated based on a prediction model. This model uses an LSTM network structure to process time series data. The input is the sensor sample sequence X_t = [ω_t, a_t, θ_t], which includes angular velocity ω_t, acceleration a_t, and attitude angle θ_t. The LSTM unit controls the flow of information through the input gate i_t, the forget gate f_t, and the output gate o_t. Its state update can be expressed as h_t = o_t⊙tanh(c_t), where c_t is the unit state and ⊙ represents element-wise multiplication. The model is expanded in the time dimension. The hidden state h_t at each moment is mapped through a fully connected layer to obtain the predicted gain g_t = W_h·h_t + b_h. During training, the mean squared error loss function L = (g_t - g_t^*)² is used to minimize the difference between the predicted value and the true optimal gain g_t^*. In practical applications, for example, when a user quickly turns their head to look at the speaker, the model can predict the head movement trajectory and adjust the audio gain parameters in advance to ensure the continuity of the auditory experience.

[0115] S210: Generate an adaptive adjustment curve for audio gain according to the head rotation parameter.

[0116] Among them, the adaptation adjustment curve refers to the mathematical function that describes the change of audio gain with head movement; audio gain represents the amplification or attenuation multiple of the sound signal; adjustment curve generation refers to the process of calculating the corresponding gain value based on motion parameters; adaptability refers to the dynamic response characteristics of gain adjustment to head movement.

[0117] After obtaining the head rotation parameters, the dual-headphone system immediately executes the gain curve calculation process. Specifically, the dual-headphone system first establishes a head motion state space model, mapping the rotation angle, angular velocity, and angular acceleration to normalized state variables. Then, based on the physiological characteristics of the human ear and the principle of auditory compensation, a benchmark function for gain adjustment is constructed. This function must be combined with the dynamic adaptive characteristics of the auditory system to sound localization. Next, the system combines the current sound source position information to calculate the optimal gain value under different rotation states, and generates a smooth adjustment curve through spline interpolation. Finally, the system imposes time domain constraints on the generated curve to ensure the continuity and stability of the gain change.

[0118] In some embodiments, the generation of an adaptive adjustment curve can be achieved through a variety of methods: Optionally, the dual-headphone system uses a fuzzy control method to establish a mapping relationship between head motion characteristics and gain adjustment rules, thereby achieving dynamic gain adjustment based on empirical knowledge and introducing an adaptive factor to optimize the control effect; Optionally, the dual-headphone system uses a neural network prediction model to establish a nonlinear mapping function by learning the motion-gain correspondence in historical data, thereby achieving real-time gain prediction and adjustment. It is understood that other curve generation methods can also be used to achieve gain adjustment, such as algorithms based on optimal control or adaptive filtering, which are not limited here.

[0119] It should be noted that the dual-headphone system uses a gain control curve optimization method based on dynamic programming. First, the objective function for gain adjustment is established: J = Σ(w_1·|g(n)-g(n-1)|^2 + w_2·|g(n)-g_target(n)|^2), where g(n) is the actual gain value, g_target(n) is the target gain value, and w_1 and w_2 are weight coefficients. Constraints include the gain change rate limit: |g(n)-g(n-1)| ≤ Δg_max and the gain range limit: g_min ≤ g(n) ≤ g_max. The optimal gain sequence is recursively solved using the Bellman equation: V(n, g) = min{c(n, g, u) + V(n+1, f(g, u))}, where c is the current cost function, f is the state transition function, and u is the control input. Finally, a smooth gain control curve is generated using cubic spline interpolation, and the interpolation node density is dynamically adjusted based on head movement speed. This method can achieve smooth gain transition while ensuring audio quality and avoid auditory discomfort caused by sudden changes.

[0120] S211 : Apply the adaptive adjustment curve to the audio signal processing channel to adjust the gain coefficient of the audio signal.

[0121] Among them, the signal processing channel refers to the digital processing flow of audio data; the gain coefficient refers to the real-time amplification factor applied to the audio signal; the curve application refers to the conversion of the theoretically calculated gain value into the actual signal processing operation; the adjustment process refers to the dynamic update and smooth transition of the gain value.

[0122] After generating the adaptation curve, the dual-headphone system performs gain control in real time. Specifically, the system first discretizes the adaptation curve into gain control points over a time series. A digital signal processor then calculates the instantaneous gain value corresponding to each audio sampling point. The system then smoothes the gain value to avoid audio distortion caused by sudden gain changes. The system then applies the processed gain coefficient to the audio data stream, dynamically modulating the signal amplitude. Finally, the system monitors the quality of the processed audio signal to ensure that gain adjustments do not introduce noticeable audible distortion or noise.

[0123] In some embodiments, dynamic adjustment of the gain coefficient can be achieved through a variety of methods: Optionally, the dual-headphone system can employ a frequency-band processing approach, using independent gain control parameters for different frequency components, achieving fine-grained spectral adjustment through multi-band decomposition and reconstruction while maintaining the naturalness of the audio signal; Optionally, the dual-headphone system can employ an adaptive filter bank, dynamically adjusting the filter coefficients based on the time-frequency characteristics of the signal to achieve continuous and smooth gain changes. It is understood that other signal processing methods, such as those based on wavelet transforms or adaptive equalization, can also be employed to achieve gain adjustment, and these are not limited here.

[0124] S212: When the user's conversational voice is detected, maintain the audio signal gain unchanged.

[0125] Referring to step S106 , the dual-headphone system maintains the audio signal gain unchanged when the users are talking.

[0126] In the embodiment of the present application, due to the use of binaural microphones to collect ambient audio in real time and combine it with the head posture sensor data for intelligent analysis, and cooperate with the adaptive gain control algorithm, it is possible to automatically identify and enhance the sound signal in the direction of the user's current focus, effectively solving the problem that traditional headphones require manual triggering or voice control to hear external sounds. Through multi-sensor fusion and spatial audio processing technology, the system can accurately locate and track multiple sound sources, while maintaining the noise reduction effect while achieving a natural and smooth interactive experience. Combined with motion compensation and dynamic noise processing, the system can adapt to various complex environments, ensuring that the audio effect is always stable during the user's movement, thereby realizing a fully automatic and intelligent hearing enhancement solution.

[0127] The following describes the dual-headphone system in the embodiment of the present invention from the perspective of hardware processing. Figure 3 , is a schematic diagram of a physical device structure of a dual-earphone system in an embodiment of the present application.

[0128] It should be noted that Figure 3 The structure of the dual-earphone system shown is only an example and should not limit the functions and scope of use of the embodiments of the present invention.

[0129] like Figure 3 As shown, the dual-headphone system includes a CPU 301, which can perform various appropriate actions and processes based on programs stored in a ROM 302 or programs loaded from a storage unit 308 into a RAM 303, such as executing the methods described in the above embodiments. The RAM 303 also stores various programs and data required for system operation. The CPU 301, ROM 302, and RAM 303 are interconnected via a bus 304. An I / O interface 305 is also connected to the bus 304.

[0130] The following components are connected to the I / O interface 305: an input section 306 including an audio input device, push button switches, and the like; an output section 307 including a liquid crystal display (LCD), an audio output device, indicator lights, and the like; a storage section 308 including a hard disk and the like; and a communication section 309 including a network interface card such as a LAN (Local Area Network) card or a modem. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to the I / O interface 305 as needed. Removable media 311, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 310 as needed, so that computer programs read from the removable media can be installed in the storage section 308 as needed.

[0131] In particular, according to an embodiment of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, an embodiment of the present invention includes a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 309 and / or installed from the removable medium 311. When the computer program is executed by the CPU 301, the various functions defined in the present invention are performed.

[0132] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. Each box in the flowchart or block diagram can represent a module, program segment, or part of the code, and the above-mentioned module, program segment, or part of the code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings.

[0133] Specifically, the dual-headphone system of this embodiment includes a processor and a memory. The memory stores a computer program. When the computer program is executed by the processor, the dual-headphone sound effect adjustment method provided in the above embodiment is implemented.

[0134] As another aspect, the present invention further provides a computer-readable storage medium, which may be included in the dual-headphone system described in the above embodiments, or may exist independently and not be incorporated into the dual-headphone system. The storage medium carries one or more computer programs. When executed by a processor of the dual-headphone system, the one or more computer programs enable the dual-headphone system to implement the dual-headphone sound effect adjustment method provided in the above embodiments.

[0135] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0136] As used in the above embodiments, the term “when” may be interpreted to mean “if” or “after” or “in response to determining that” or “in response to detecting that”, depending on the context. Similarly, the phrases “upon determining that” or “if (stated condition or event) is detected” may be interpreted to mean “if determining that” or “in response to determining that” or “upon detecting (stated condition or event)” or “in response to detecting (stated condition or event)”, depending on the context.

Claims

1. A method for adjusting the sound effects of two headphones, characterized in that: Applied to a dual-headphone system, the method includes: Collecting ambient audio signals through binaural microphones and performing spectrum analysis to obtain frequency distribution characteristics of the ambient audio signals; Based on a preset human voice frequency range, extracting an audio signal that meets the human voice characteristics from the frequency distribution characteristics as the target audio; Calculating the time difference and intensity difference of the target audio between the left and right ears to obtain the direction and relative distance of the human voice source in the environment; Acquiring head posture sensor data, and calculating a matching angle between the user's head orientation and the direction of the human voice source based on the head posture sensor data; When the matching angle is smaller than a preset threshold, increasing the audio signal gain in the direction in which the head is facing; When the conversational voice of the user is detected, the gain of the audio signal is maintained unchanged.

2. The method according to claim 1, characterized in that The step of collecting the ambient audio signal by using binaural microphones and performing spectrum analysis to obtain the frequency distribution characteristics of the ambient audio signal specifically includes: Synchronously collecting ambient audio signals through left and right ear microphones and generating a digital audio data stream; the digital audio data stream includes binaural sound pressure signals and timestamp information; Performing denoising on the digital audio data stream to obtain a noise-reduced audio signal; the noise-reduced audio signal has a higher signal-to-noise ratio than the ambient audio signal; Performing a fast Fourier transform on the noise-reduced frequency signal to generate a frequency distribution feature including frequency domain data; the frequency domain data includes frequency components and amplitude information.

3. The method according to claim 1, characterized in that The step of calculating the time difference and intensity difference of the target audio between the left and right ears to obtain the direction and relative distance of the human voice source in the environment specifically includes: Performing a cross-correlation operation on the target audio between the left and right ears to obtain a cross-correlation time series; Extracting the time delay of the target audio between the left and right ears based on the peak position of the cross-correlation time series, and calculating the spatial transfer function in combination with the sound pressure level difference to generate spatial coordinate data of the target audio; A sound source localization map is constructed according to the spatial coordinate data, and the direction and relative distance of the human voice source are determined based on the sound source localization map.

4. The method according to claim 1, wherein Before the step of collecting the ambient audio signal by using binaural microphones and performing spectrum analysis to obtain the frequency distribution characteristics of the ambient audio signal, the method further includes: Collecting sound pressure level data of ambient noise and generating a noise characteristic spectrum; the noise characteristic spectrum includes energy distribution information of each frequency band; Calculating a gain control parameter of a microphone according to the noise characteristic spectrum, and writing the gain control parameter into a microphone control register; The sensitivity of the signal acquisition channel is adjusted based on the gain control parameter so that the range of the acquired audio signal is within a preset optimal range.

5. The method according to claim 4, characterized in that The step of collecting the sound pressure level data of the ambient noise and generating the noise characteristic spectrum specifically includes: receiving motion state data output by the acceleration sensor and generating a motion feature vector; Performing motion noise compensation on the ambient audio data based on the motion feature vector to obtain corrected ambient noise data; The corrected environmental noise data is divided into frequency bands and stored in a noise feature database to generate a noise feature map including a noise feature index table.

6. The method according to claim 1, characterized in that After the step of increasing the audio signal gain in the direction in which the head is facing when the matching angle is less than a preset threshold, the method further includes: Receive the angular velocity data stream output by the gyroscope and calculate the head rotation parameters; generating an adaptive adjustment curve of audio gain according to the head rotation parameter; The adaptive adjustment curve is applied to the audio signal processing channel to adjust the gain coefficient of the audio signal.

7. The method according to claim 6, characterized in that The step of receiving the angular velocity data stream output by the gyroscope and calculating the head rotation parameter specifically includes: Reading a three-axis angular velocity data stream from a head gyroscope and converting the three-axis angular velocity data stream into an Euler angle vector; Performing Kalman filtering on the Euler angle vector to obtain head posture data; The rotation direction and angular velocity value of the head are calculated based on the head posture data to generate head rotation parameters.

8. A dual-headphone system, characterized in that: The dual-earphone system includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the dual-earphone system to execute the method according to any one of claims 1 to 7.

9. A computer-readable storage medium comprising instructions, characterized in that: When the instruction is executed on a dual-headphone system, the dual-headphone system is caused to execute the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that When the computer program product is run on a dual-headphone system, the dual-headphone system is enabled to perform the method according to any one of claims 1 to 7.

Images