Earphone sound signal processing method and system, earphone and computer

Abstract

The invention provides an earphone sound signal processing method and system, an earphone and a computer. The sound signal processing method comprises the following steps: receiving input multichannel stereo playing audio data; performing frequency equalization processing on the multi-channel stereo playing audio data based on an inverse filtering algorithm to reduce noise in the multi-channel stereo playing audio data; performing spatial filtering on the multi-channel stereo playing audio data to obtain a transmission function of sound source filtering, calculating the multi-channel stereo playing audio data according to the transmission function, and converting the multi-channel stereo playing audio data into a dual-channel stereo signal of the earphone; mel segmentation is carried out on the two-channel stereo information, feature vectors of Mel segments of the stereo signal are extracted, and time-dependent analysis is carried out on the feature vectors to evaluate the sound quality of the two-channel stereo signal. The earphone sound signal processing method provided by the invention is good in sound three-dimensional effect.

Description

Technical Field

[0001] The present invention relates to the technical field of sound signal processing, and in particular to a method and system for processing earphone sound signals, earphones and a computer. Background Art

[0002] With the continuous development of computer communication technology, the widespread application of terminals such as smart phones, tablet computers and laptops, terminals are developing in a diversified and personalized direction, and are becoming an indispensable terminal in people's life and work. Earphones are a pair of conversion units that receive electrical signals from media players or receivers and convert them into audible sound waves using speakers close to the ears. Earphones can be separated from the media player terminal, so that you can listen to the sound alone without disturbing others; they can also isolate the sounds of the surrounding environment.

[0003] With the continuous development of acoustic theory and technology, sound reproduction pays more attention to realism and presence. Traditional two-channel stereo reproduction can no longer meet higher listening requirements. The sound reproduction effect can be improved by increasing the number of stereo playback channels. It mainly includes 5.1 channel, 7.1 channel, 9.1 channel stereo and panoramic sound playback system. 5.1 channel is the most common in surround sound music programs. However, the existing stereo playback is mainly played and reproduced through large audio equipment. The number of devices is large and the size is large. It is inconvenient to carry, and it is impossible to eliminate the influence of the sound field environment on the listening effect during playback, so the scope of application is not high. Summary of the invention

[0004] Based on this, the purpose of the present invention is to provide a headphone sound signal processing method, system, headphone and computer to solve the technical problems existing in the prior art.

[0005] The present invention provides a method for processing earphone sound signals, comprising:

[0006] Receive input multi-channel stereo playback audio data;

[0007] Performing frequency equalization processing on the multi-channel stereo playback audio data based on an inverse filtering algorithm to reduce noise in the multi-channel stereo playback audio data;

[0008] Performing spatial filtering on the multi-channel stereo playback audio data to obtain a transfer function of sound source filtering, and converting the multi-channel stereo playback audio data into a dual-channel stereo signal of headphones according to the transfer function;

[0009] Mel segmentation is performed on the dual-channel stereo information to extract feature vectors of the Mel segments of the stereo signal, and time-dependent analysis is performed on the feature vectors to evaluate the sound quality of the dual-channel stereo signal.

[0010] Optionally, the step of performing frequency equalization processing on the multi-channel stereo playback audio data based on an inverse filtering algorithm to reduce noise in the multi-channel stereo playback audio data includes:

[0011] Performing system decomposition on the transfer function of the secondary channel in the earphone, and obtaining that the transfer function of the secondary channel includes a cascade form of a minimum phase system and an all-pass system;

[0012] A minimum phase system inverse filtering system is cascaded before the decomposed transfer function of the secondary channel to obtain an all-pass system of the secondary channel, so as to convert the secondary channel model into a pure delay model;

[0013] Acquire a reference noise input signal and an error noise input signal in the multi-channel stereo playback audio data, and iteratively calculate the sound input signal and the error sound input signal based on a normalized least mean square model;

[0014] Based on the iterative calculation results and the pure delay model, the primary channel in the headphone active noise reduction process is identified online, and the noise in the multi-channel stereo playback audio data is reduced through the primary channel.

[0015] Optionally, the cascade form expression of the secondary channel is:

[0016]

[0017] Where J(z) is the transfer function of the secondary channel, J min (z) is the minimum phase system of the transfer function of the secondary channel, J ap (z) is the transfer function of the secondary channel for the all-pass system;

[0018] The expression for iterative calculation of the normalized least mean square model is:

[0019]

[0020] Where d(i+1) is the updated weight for the i+1th iteration, d(i) is the updated weight for the kth iteration, x(i) is the reference noise input signal, and e(i) is the error noise input signal. is the average amplitude of the reference noise;

[0021] The identification expression of the primary channel is:

[0022] C(z)=J ap (z)d * (z)

[0023] Where C(z) is the transfer function of the primary channel, d *(z) is to satisfy J ap Iteration results with only phase delay between (z) and C(z).

[0024] Optionally, the step of converting the multi-channel stereo playback audio data into a dual-channel stereo signal for headphones according to the transfer function comprises:

[0025] Respectively obtain the sound signals of the left sound source and the right sound source, the sound signal of the center sound source, the sound signals of the left surround sound source and the sound signals of the right surround sound source in the multi-channel stereo playback audio data;

[0026] Determine the position of each sound source according to the multi-channel stereo playback audio data, and determine the head-related data corresponding to each sound source based on the transfer function and the position of each sound source;

[0027] The sound signal conversion is performed on the head-related data corresponding to each sound source and the sound signal of each channel in the multi-channel stereo playback audio data to obtain a dual-channel stereo signal of the earphone.

[0028] Optionally, the expression of the transfer function is:

[0029]

[0030] In the formula, H L , H R are the transfer functions from the sound source to the left and right ears of the listener, P L , P R are the complex sound pressures generated by the sound source at the left and right ears, respectively; P0 is the complex sound pressure at the center of the left and right ears excluding the listener's head; α is the horizontal azimuth; β is the elevation angle; h is the distance from the sound source to the center of the left and right ears; ω is the sound wave frequency; and k is the personalized parameter corresponding to different listeners;

[0031] The expression of the sound signal conversion is:

[0032]

[0033] Where, L ′ , R ′ They are the left and right channel stereo signals of the headphones, H LL , H RL , H LR , H RR H is the head-related data of the left and right channels of the headphones from the left and right sound sources, respectively. LLs , H RLs , H LRs , H rRsThey are the head-related data from the left surround sound source and the right surround sound source to the left and right channels of the headphones. L, R, C, LS, and RS are the sound signals of the left sound source, right sound source, center sound source, left surround sound source, and right surround sound source respectively.

[0034] Optionally, the step of performing Mel segmentation on the dual-channel stereo information, extracting feature vectors of Mel segments of the stereo signal, and performing time-dependent analysis on the feature vectors to evaluate the sound quality of the dual-channel stereo signal includes:

[0035] Setting a maximum Mel frequency and a Mel segmentation time window, and using a Mel filter to filter and perform logarithmic operations on the dual-channel stereo signal to capture short-time frequency information in the dual-channel stereo signal;

[0036] Dividing the segmented short-time sound signal into Mel segments based on a preset segmentation time length, and modeling the dual-channel stereo signal through the Mel segments;

[0037] The obtained Mel segment is input into the CNN network model, and the relationship between the input and output of the CNN network model is learned based on the residual block in the CNN network model to extract the feature vector of the Mel segment of the stereo signal;

[0038] Input the feature vector into the TFA attention mechanism for average pooling to obtain a time domain feature vector and a frequency domain feature vector, and perform vector product of the time domain feature vector and the frequency domain feature vector to obtain a two-dimensional attention weight;

[0039] The two-dimensional attention weight is weightedly fused with the feature vector by dot product to capture the time dependence and energy distribution of the stereo signal quality to evaluate the sound quality of the two-channel stereo signal.

[0040] Optionally, the expressions of the time domain feature vector and the frequency domain feature vector are:

[0041] T A =σ(f TA (δ(Z T )))

[0042] F A =σ(f FA ((δ(Z F )))

[0043] Where, T A 、F A are the time domain feature vector and frequency domain feature vector respectively, σ and δ are ReLU and Sigmoid activation functions respectively, and f TA 、f FA are time domain and frequency domain convolution respectively, Z T, Z F are the sequence positions of the time domain feature vector and the frequency domain feature vector respectively.

[0044] The present invention also provides an earphone sound signal processing system, comprising:

[0045] A receiving module, used for receiving input multi-channel stereo playback audio data;

[0046] An equalization module, used for performing frequency equalization processing on the multi-channel stereo playback audio data based on an inverse filtering algorithm to reduce noise in the multi-channel stereo playback audio data;

[0047] A conversion module, used to construct the multi-channel stereo playback audio data for spatial filtering, obtain a transfer function of sound source filtering, and convert the multi-channel stereo playback audio data into a dual-channel stereo signal of headphones according to the transfer function;

[0048] The evaluation module is used to perform Mel segmentation on the dual-channel stereo information, extract feature vectors of the Mel segments of the stereo signal, and perform time-dependent analysis on the feature vectors to evaluate the sound quality of the dual-channel stereo signal.

[0049] The present invention also provides an earphone having a computer program stored thereon, and the program implements the above-mentioned earphone sound signal processing method when executed by a processor.

[0050] The present invention also provides a computer, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the headphone sound signal processing method when executing the computer program.

[0051] Compared with the prior art, the present invention has the following beneficial effects: the headphone sound signal processing method provided by the present application first receives input multi-channel stereo playback audio data, performs frequency equalization processing on the multi-channel stereo playback audio data based on an inverse filtering algorithm to reduce noise in the multi-channel stereo playback audio data; the inverse filtering algorithm provides inverse noise with a phase opposite to the original noise in real time to actively balance the original noise, constructs multi-channel stereo playback audio data for spatial filtering, obtains a transfer function of sound source filtering, calculates the multi-channel stereo playback audio data according to the transfer function and converts it into a dual-channel stereo signal of the headphone, improves the realism and surround of the stereo in the headphone, performs Mel segmentation on the dual-channel stereo information, extracts the feature vector of the Mel segment of the stereo signal, performs time-dependent analysis on the feature vector to evaluate the sound quality of the dual-channel stereo signal, and finally performs quality evaluation on the stereo in the headphone to improve the evaluation quality of the stereo.

[0052] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 is a flow chart of a method for processing earphone sound signals in Embodiment 1 of the present invention;

[0054] Figure 2 This is a structural block diagram of a computer in Embodiment 4 of the present invention.

[0055] The following specific implementation manner will further illustrate the present invention in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0056] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Several embodiments of the present invention are given in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

[0057] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0058] Embodiment 1

[0059] See also Figure 1 , which shows a method for processing earphone sound signals in Embodiment 1 of the present invention. The method for processing earphone sound signals specifically includes steps S10 to S40:

[0060] S10, receiving input multi-channel stereo playback audio data;

[0061] In this embodiment, 5.1 channels are used as the object, including left channel, right channel, center channel, left surround sound, right surround sound and bass channel LFE. 5.1 surround sound can provide a relatively accurate front sound image positioning effect and a full auditory experience. The left and right surround channels supplement the spatial environment information that the left channel, right channel and center channel do not have, and the listening effect is real and full; multi-channel stereo playback audio data can be stored in terminal devices such as smart phones, tablets and laptops, which is convenient for listeners to replay.

[0062] S20, performing frequency equalization processing on the multi-channel stereo playback audio data based on an inverse filtering algorithm to reduce noise in the multi-channel stereo playback audio data;

[0063] In the specific implementation, the received multi-channel stereo playback audio data may contain noise from the external environment. Noise reduction can be divided into passive noise reduction and active noise reduction. Passive noise reduction mainly absorbs noise through sound-absorbing materials. Due to reasons such as materials and size, its scope of application is limited. Active noise reduction is to emit anti-phase noise that is opposite to the phase of the noise, and achieve the purpose of canceling noise through mutual interference of sound waves. Active noise reduction has good suppression and strong applicability. Active noise reduction technology applied to headphones can significantly improve the noise of people's work, study, and rest, and improve the quality of life.

[0064] Optionally, in this embodiment, the step of performing frequency equalization processing on the multi-channel stereo playback audio data based on an inverse filtering algorithm to reduce noise in the multi-channel stereo playback audio data includes:

[0065] Performing system decomposition on the transfer function of the secondary channel in the earphone, and obtaining that the transfer function of the secondary channel includes a cascade form of a minimum phase system and an all-pass system;

[0066] A minimum phase system inverse filtering system is cascaded before the decomposed transfer function of the secondary channel to obtain an all-pass system of the secondary channel, so as to convert the secondary channel model into a pure delay model;

[0067] Acquire a reference noise input signal and an error noise input signal in the multi-channel stereo playback audio data, and iteratively calculate the sound input signal and the error sound input signal based on a normalized least mean square model;

[0068] Based on the iterative calculation results and the pure delay model, the primary channel in the headphone active noise reduction process is identified online, and the noise in the multi-channel stereo playback audio data is reduced through the primary channel.

[0069] The cascade form of the secondary channel is expressed as:

[0070]

[0071] Where J(z) is the transfer function of the secondary channel, J min (z) is the minimum phase system of the transfer function of the secondary channel, J ap (z) is the transfer function of the secondary channel for the all-pass system;

[0072] The expression for iterative calculation of the normalized least mean square model is:

[0073]

[0074] Where d(i+1) is the updated weight for the i+1th iteration, d(i) is the updated weight for the kth iteration, x(i) is the reference noise input signal, and e(i) is the error noise input signal. is the average amplitude of the reference noise;

[0075] The identification expression of the primary channel is:

[0076] C(z)=J ap (z)d * (z)

[0077] Where C(z) is the transfer function of the primary channel, d * (z) is to satisfy J ap Iteration results with only phase delay between (z) and C(z).

[0078] In this embodiment, an inverse filtering algorithm is used to emit an inverse noise of the original noise in real time, and the frequency response curve of the speaker in the active noise reduction is corrected by the frequency equalization technology. In the process of active noise reduction, the introduction of an inverse speaker will bring about the problem of phase delay. Due to problems such as nonlinear vibration in the propagation process, the speaker system is a non-minimum phase system, and the secondary channel is also a non-minimum phase system during the headphone playback process. Therefore, by cascading a minimum phase inverse system in the non-minimum phase system, the secondary channel can be turned into an all-pass system. The all-pass system has only phase delay but no frequency change. The phase delay of the cascaded all-pass system is less than the phase delay of the primary channel. Based on the normalized least mean square model, it is simple to identify the system that satisfies J. ap (z) and C(z) have only a phase delay of d * (z), and then online identifying the primary channel to reduce the noise in the multi-channel stereo playback audio data.

[0079] S30, constructing the multi-channel stereo playback audio data for spatial filtering, obtaining a transfer function of sound source filtering, and converting the multi-channel stereo playback audio data into a dual-channel stereo signal of headphones according to the transfer function;

[0080] In a specific implementation, after the multi-channel stereo playback audio data is subjected to noise processing, it is necessary to further virtually convert the 5.1-channel stereo playback audio data into binaural sound signals to improve the sound effect of stereo playback through headphones;

[0081] Optionally, in this embodiment, the step of converting the multi-channel stereo playback audio data into a dual-channel stereo signal for headphones according to the transfer function includes:

[0082] Respectively obtain the sound signals of the left sound source and the right sound source, the sound signal of the center sound source, the sound signals of the left surround sound source and the sound signals of the right surround sound source in the multi-channel stereo playback audio data;

[0083] Determine the position of each sound source according to the multi-channel stereo playback audio data, and determine the head-related data corresponding to each sound source based on the transfer function and the position of each sound source;

[0084] The sound signal conversion is performed on the head-related data corresponding to each sound source and the sound signal of each channel in the multi-channel stereo playback audio data to obtain a dual-channel stereo signal of the earphone.

[0085] The expression of the transfer function is:

[0086]

[0087] In the formula, H L , H R are the transfer functions from the sound source to the left and right ears of the listener, P L , P R are the complex sound pressures generated by the sound source at the left and right ears, respectively; P0 is the complex sound pressure at the center of the left and right ears excluding the listener's head; α is the horizontal azimuth; β is the elevation angle; h is the distance from the sound source to the center of the left and right ears; ω is the sound wave frequency; and k is the personalized parameter corresponding to different listeners;

[0088] The expression of the sound signal conversion is:

[0089]

[0090] Where, L ′ , R ′ They are the left and right channel stereo signals of the headphones, H LL , H RL , H LR , H RR H is the head-related data corresponding to the left and right channels of the headphones from the left and right sound sources, respectively. LLs , H RLS , H LRs , H RRs They are the head-related data from the left surround sound source and the right surround sound source to the left and right channels of the headphones. L, R, C, LS, and RS are the sound signals of the left sound source, right sound source, center sound source, left surround sound source, and right surround sound source respectively.

[0091] Optionally, in this embodiment, it is assumed that the head-related data of the center sound source sound signal to the left and right channels of the earphone is Head-related data is the result of spatial filtering of sound waves by physiological structures such as the head, auricle, and shoulders. It describes the transmission process of sound waves from the sound source to the human ear. Head-related data contains information about the location of the sound source and can simulate the auditory effects of sound sources in different spatial locations. Through the above sound signal conversion, headphones can be used to achieve a more realistic surround sound experience.

[0092] S40, performing Mel segmentation on the dual-channel stereo information, extracting feature vectors of Mel segments of the stereo signal, and performing time-dependent analysis on the feature vectors to evaluate the sound quality of the dual-channel stereo signal.

[0093] Optionally, the step of performing Mel segmentation on the dual-channel stereo information, extracting feature vectors of Mel segments of the stereo signal, and performing time-dependent analysis on the feature vectors to evaluate the sound quality of the dual-channel stereo signal includes:

[0094] Setting a maximum Mel frequency and a Mel segmentation time window, and using a Mel filter to filter and perform logarithmic operations on the dual-channel stereo signal to capture short-time frequency information in the dual-channel stereo signal;

[0095] Dividing the segmented short-time sound signal into Mel segments based on a preset segmentation time length, and modeling the dual-channel stereo signal through the Mel segments;

[0096] The obtained Mel segment is input into the CNN network model, and the relationship between the input and output of the CNN network model is learned based on the residual block in the CNN network model to extract the feature vector of the Mel segment of the stereo signal;

[0097] Input the feature vector into the TFA attention mechanism for average pooling to obtain a time domain feature vector and a frequency domain feature vector, and perform vector product of the time domain feature vector and the frequency domain feature vector to obtain a two-dimensional attention weight;

[0098] The two-dimensional attention weight is weightedly fused with the feature vector by dot product to capture the time dependence and energy distribution of the stereo signal quality to evaluate the sound quality of the two-channel stereo signal.

[0099] The expressions of the time domain feature vector and the frequency domain feature vector are:

[0100] T A =d(f TA (δ(Z T )))

[0101] F A =σ(f FA (δ(Z F )))

[0102] Where, T A 、F A are the time domain feature vector and frequency domain feature vector respectively, σ and δ are ReLU and Sigmoid activation functions respectively, and f TA 、f FA are time domain and frequency domain convolution respectively, Z T , Z F are the sequence positions of the time domain feature vector and the frequency domain feature vector respectively.

[0103] Optionally, in this implementation, after obtaining the dual-channel stereo signal of the headset, it is necessary to further objectively evaluate the stereo sound signal to measure the sound quality; in this embodiment, using the Mel spectrum as a feature extractor can improve the accuracy and robustness of the sound quality evaluation, the maximum Mel frequency can be the maximum frequency that the human ear can reach, 20kHz, and the segmentation time can be as long as 15m s , the frame shift of the mel segment can be 80m s , so as to model the sound signal in time; the feature vector is input into the TFA (Transformer attention) self-attention mechanism for average pooling, which can not only capture the dependency between time steps, but also extract the energy distribution in the sound signal, thereby improving the performance of the stereo signal quality assessment task. The stereo evaluation indicators can include the overall sound quality, noise, loudness, etc., to judge the sound quality level.

[0104] In summary, the headphone sound signal processing method provided by the present application first receives input multi-channel stereo playback audio data, performs frequency equalization processing on the multi-channel stereo playback audio data based on an inverse filtering algorithm to reduce the noise in the multi-channel stereo playback audio data; the inverse filtering algorithm provides an anti-phase noise with a phase opposite to the original noise in real time to actively balance the original noise, constructs the multi-channel stereo playback audio data for spatial filtering, obtains the transfer function of the sound source filtering, calculates the multi-channel stereo playback audio data according to the transfer function and converts it into a dual-channel stereo signal of the headphone, improves the realism and surround of the stereo in the headphone, performs Mel segmentation on the dual-channel stereo information, extracts the feature vector of the Mel segment of the stereo signal, performs time-dependent analysis on the feature vector to evaluate the sound quality of the dual-channel stereo signal, and finally performs quality evaluation on the stereo in the headphone to improve the evaluation quality of the stereo.

[0105] Embodiment 2

[0106] This embodiment provides an earphone sound signal processing system, including:

[0107] A receiving module, used for receiving input multi-channel stereo playback audio data;

[0108] An equalization module, used for performing frequency equalization processing on the multi-channel stereo playback audio data based on an inverse filtering algorithm to reduce noise in the multi-channel stereo playback audio data;

[0109] A conversion module, used to construct the multi-channel stereo playback audio data for spatial filtering, obtain a transfer function of sound source filtering, and convert the multi-channel stereo playback audio data into a dual-channel stereo signal of headphones according to the transfer function;

[0110] The evaluation module is used to perform Mel segmentation on the dual-channel stereo information, extract feature vectors of the Mel segments of the stereo signal, and perform time-dependent analysis on the feature vectors to evaluate the sound quality of the dual-channel stereo signal.

[0111] The step of performing frequency equalization processing on the multi-channel stereo playback audio data based on an inverse filtering algorithm to reduce noise in the multi-channel stereo playback audio data includes:

[0112] Performing system decomposition on the transfer function of the secondary channel in the earphone, and obtaining that the transfer function of the secondary channel includes a cascade form of a minimum phase system and an all-pass system;

[0113] A minimum phase system inverse filtering system is cascaded before the decomposed transfer function of the secondary channel to obtain an all-pass system of the secondary channel, so as to convert the secondary channel model into a pure delay model;

[0114] Acquire a reference noise input signal and an error noise input signal in the multi-channel stereo playback audio data, and iteratively calculate the sound input signal and the error sound input signal based on a normalized least mean square model;

[0115] Based on the iterative calculation results and the pure delay model, the primary channel in the headphone active noise reduction process is identified online, and the noise in the multi-channel stereo playback audio data is reduced through the primary channel.

[0116] The cascade form of the secondary channel is expressed as:

[0117]

[0118] Where J(z) is the transfer function of the secondary channel, J min (z) is the minimum phase system of the transfer function of the secondary channel, J ap (z) is the transfer function of the secondary channel for the all-pass system;

[0119] The expression for iterative calculation of the normalized least mean square model is:

[0120]

[0121] Where d(i+1) is the updated weight for the i+1th iteration, d(i) is the updated weight for the kth iteration, x(i) is the reference noise input signal, and e(i) is the error noise input signal. is the average amplitude of the reference noise;

[0122] The identification expression of the primary channel is:

[0123] C(z)=J ap (z)d * (z)

[0124] Where C(z) is the transfer function of the primary channel, d * (z) is to satisfy J ap Iteration results with only phase delay between (z) and C(z).

[0125] The step of converting the multi-channel stereo playback audio data into a dual-channel stereo signal for headphones according to the transfer function comprises:

[0126] Respectively obtain the sound signals of the left sound source and the right sound source, the sound signal of the center sound source, the sound signals of the left surround sound source and the sound signals of the right surround sound source in the multi-channel stereo playback audio data;

[0127] Determine the position of each sound source according to the multi-channel stereo playback audio data, and determine the head-related data corresponding to each sound source based on the transfer function and the position of each sound source;

[0128] The sound signal conversion is performed on the head-related data corresponding to each sound source and the sound signal of each channel in the multi-channel stereo playback audio data to obtain a dual-channel stereo signal of the earphone.

[0129] The expression of the transfer function is:

[0130]

[0131] In the formula, H L , H R are the transfer functions from the sound source to the left and right ears of the listener, P L , P R are the complex sound pressures generated by the sound source at the left and right ears, respectively; P0 is the complex sound pressure at the center of the left and right ears excluding the listener's head; α is the horizontal azimuth; β is the elevation angle; h is the distance from the sound source to the center of the left and right ears; ω is the sound wave frequency; and k is the personalized parameter corresponding to different listeners;

[0132] The expression of the sound signal conversion is:

[0133]

[0134] Where, L ′ , R ′ They are the left and right channel stereo signals of the headphones, H LL , H RL , H LE , H RR H is the head-related data of the left and right channels of the headphones from the left and right sound sources, respectively. LLs , H RLs , H LRs , H RRs They are the head-related data from the left surround sound source and the right surround sound source to the left and right channels of the headphones. L, R, C, LS, and RS are the sound signals of the left sound source, right sound source, center sound source, left surround sound source, and right surround sound source respectively.

[0135] The steps of performing Mel segmentation on the dual-channel stereo information, extracting feature vectors of Mel segments of the stereo signal, and performing time-dependent analysis on the feature vectors to evaluate the sound quality of the dual-channel stereo signal include:

[0136] Setting a maximum Mel frequency and a Mel segmentation time window, and using a Mel filter to filter and perform logarithmic operations on the dual-channel stereo signal to capture short-time frequency information in the dual-channel stereo signal;

[0137] Dividing the segmented short-time sound signal into Mel segments based on a preset segmentation time length, and modeling the dual-channel stereo signal through the Mel segments;

[0138] The obtained Mel segment is input into the CNN network model, and the relationship between the input and output of the CNN network model is learned based on the residual block in the CNN network model to extract the feature vector of the Mel segment of the stereo signal;

[0139] Input the feature vector into the TFA attention mechanism for average pooling to obtain a time domain feature vector and a frequency domain feature vector, and perform vector product of the time domain feature vector and the frequency domain feature vector to obtain a two-dimensional attention weight;

[0140] The two-dimensional attention weight is weightedly fused with the feature vector by dot product to capture the time dependence and energy distribution of the stereo signal quality to evaluate the sound quality of the two-channel stereo signal.

[0141] The expressions of the time domain feature vector and the frequency domain feature vector are:

[0142] T A =σ(f TA (δ(Z T )))

[0143] F A =σ(f FA ((δ(Z F )))

[0144] Where, T A 、F A are the time domain feature vector and frequency domain feature vector respectively, σ and δ are ReLU and Sigmoid activation functions respectively, and f TA 、f FA are time domain and frequency domain convolution respectively, Z T , Z F are the sequence positions of the time domain feature vector and the frequency domain feature vector respectively.

[0145] Embodiment 3

[0146] This embodiment provides an earphone having a computer program stored thereon, and when the program is executed by a processor, the above-mentioned earphone sound signal processing method is implemented.

[0147] Embodiment 4

[0148] The present invention also provides a computer, see Figure 2 , shown is a computer in an embodiment of the present invention, including a memory 10, a processor 20, and a computer program 30 stored in the memory 10 and executable on the processor 20. When the processor 20 executes the computer program 30, the above-mentioned headphone sound signal processing method is implemented.

[0149] The memory 10 includes at least one type of storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 10 may be an internal storage unit of a computer, such as a hard disk of the computer. In other embodiments, the memory 10 may also be an external storage device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card, etc. Further, the memory 10 may also include both an internal storage unit of a computer and an external storage device. The memory 10 may be used not only to store application software and various types of data installed in the computer, but also to temporarily store data that has been output or is to be output.

[0150] Among them, in some embodiments, the processor 20 can be an electronic control unit (Electronic Control Unit, abbreviated as ECU, also known as a vehicle computer), a central processing unit (Central Processing Unit, CPU), a controller, a microcontroller, a microprocessor or other data processing chip, used to run the program code stored in the memory 10 or process data, such as executing access restriction programs, etc.

[0151] It should be pointed out that Figure 2 The structure shown does not constitute a limitation on the computer. In other embodiments, the computer may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0152] Those skilled in the art will appreciate that the logic and / or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable instructions for implementing logical functions, and may be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For purposes of this specification, a "computer-readable medium" may be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0153] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0154] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or a combination thereof: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0155] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0156] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for processing earphone sound signals, characterized in that: include: Receive input multi-channel stereo playback audio data; Performing frequency equalization processing on the multi-channel stereo playback audio data based on an inverse filtering algorithm to reduce noise in the multi-channel stereo playback audio data; Performing spatial filtering on the multi-channel stereo playback audio data to obtain a transfer function of sound source filtering, and converting the multi-channel stereo playback audio data into a dual-channel stereo signal of headphones according to the transfer function; Mel segmentation is performed on the dual-channel stereo information to extract feature vectors of the Mel segments of the stereo signal, and time-dependent analysis is performed on the feature vectors to evaluate the sound quality of the dual-channel stereo signal.

2. The headphone sound signal processing method according to claim 1, characterized in that: The step of performing frequency equalization processing on the multi-channel stereo playback audio data based on an inverse filtering algorithm to reduce noise in the multi-channel stereo playback audio data comprises: Performing system decomposition on the transfer function of the secondary channel in the earphone, and obtaining that the transfer function of the secondary channel includes a cascade form of a minimum phase system and an all-pass system; A minimum phase system inverse filtering system is cascaded before the decomposed transfer function of the secondary channel to obtain an all-pass system of the secondary channel, so as to convert the secondary channel model into a pure delay model; Acquire a reference noise input signal and an error noise input signal in the multi-channel stereo playback audio data, and iteratively calculate the sound input signal and the error sound input signal based on a normalized least mean square model; Based on the iterative calculation results and the pure delay model, the primary channel in the headphone active noise reduction process is identified online, and the noise in the multi-channel stereo playback audio data is reduced through the primary channel.

3. The headphone sound signal processing method according to claim 2, characterized in that: The cascade form expression of the secondary channel is: Where J(z) is the transfer function of the secondary channel, J min (z) is the minimum phase system of the transfer function of the secondary channel, J ap (z) is the transfer function of the secondary channel for the all-pass system; The expression for iterative calculation of the normalized least mean square model is: Where d(i+1) is the updated weight for the i+1th iteration, d(i) is the updated weight for the kth iteration, x(i) is the reference noise input signal, and e(i) is the error noise input signal. is the average amplitude of the reference noise; The identification expression of the primary channel is: C(z)=J ap (z)d * (z) Where C(z) is the transfer function of the primary channel, d * (z) is to satisfy J ap Iteration results with only phase delay between (z) and C(z).

4. The headphone sound signal processing method according to claim 1, characterized in that: The step of converting the multi-channel stereo playback audio data into a dual-channel stereo signal for headphones according to the transfer function comprises: Respectively obtain the sound signals of the left sound source and the right sound source, the sound signal of the center sound source, the sound signals of the left surround sound source and the sound signals of the right surround sound source in the multi-channel stereo playback audio data; Determine the position of each sound source according to the multi-channel stereo playback audio data, and determine the head-related data corresponding to each sound source based on the transfer function and the position of each sound source; The sound signal conversion is performed on the head-related data corresponding to each sound source and the sound signal of each channel in the multi-channel stereo playback audio data to obtain a dual-channel stereo signal of the earphone.

5. The headphone sound signal processing method according to claim 4, characterized in that: The expression of the transfer function is: In the formula, H L , H R are the transfer functions from the sound source to the left and right ears of the listener, P L , P R are the complex sound pressures generated by the sound source at the left and right ears, respectively; P0 is the complex sound pressure at the center of the left and right ears excluding the listener's head; α is the horizontal azimuth; β is the elevation angle; h is the distance from the sound source to the center of the left and right ears; ω is the sound wave frequency; and k is the personalized parameter corresponding to different listeners; The expression of the sound signal conversion is: Where L' and R' are the left and right channel stereo signals of the headphone respectively, and H LL , H RL , H LR , H RR H is the head-related data of the left and right channels of the headphones from the left and right sound sources, respectively. LLs , H RLs , H LRs , H RRs They are the head-related data from the left surround sound source and the right surround sound source to the left and right channels of the headphones. L, R, C, LS, and RS are the sound signals of the left sound source, right sound source, center sound source, left surround sound source, and right surround sound source respectively.

6. The headphone sound signal processing method according to claim 1, characterized in that: The steps of performing Mel segmentation on the dual-channel stereo information, extracting feature vectors of Mel segments of the stereo signal, and performing time-dependent analysis on the feature vectors to evaluate the sound quality of the dual-channel stereo signal include: Setting a maximum Mel frequency and a Mel segmentation time window, and using a Mel filter to filter and perform logarithmic operations on the dual-channel stereo signal to capture short-time frequency information in the dual-channel stereo signal; Dividing the segmented short-time sound signal into Mel segments based on a preset segmentation time length, and modeling the dual-channel stereo signal through the Mel segments; The obtained Mel segment is input into the CNN network model, and the relationship between the input and output of the CNN network model is learned based on the residual block in the CNN network model to extract the feature vector of the Mel segment of the stereo signal; Input the feature vector into the TFA attention mechanism for average pooling to obtain a time domain feature vector and a frequency domain feature vector, and perform vector product of the time domain feature vector and the frequency domain feature vector to obtain a two-dimensional attention weight; The two-dimensional attention weight is weightedly fused with the feature vector by dot product to capture the time dependence and energy distribution of the stereo signal quality to evaluate the sound quality of the two-channel stereo signal.

7. The headphone sound signal processing method according to claim 6, characterized in that: The expressions of the time domain feature vector and the frequency domain feature vector are: T A =σ(f TA (δ(Z T ))) F A =σ(f FA (δ(Z F ))) Where, T A 、F A are the time domain feature vector and frequency domain feature vector respectively, σ and δ are ReLU and Sigmoid activation functions respectively, and f TA 、f FA are time domain and frequency domain convolution respectively, Z T , Z F are the sequence positions of the time domain feature vector and the frequency domain feature vector respectively.

8. A headphone sound signal processing system, characterized in that: include; A receiving module, used for receiving input multi-channel stereo playback audio data; An equalization module, used for performing frequency equalization processing on the multi-channel stereo playback audio data based on an inverse filtering algorithm to reduce noise in the multi-channel stereo playback audio data; A conversion module, used to construct the multi-channel stereo playback audio data for spatial filtering, obtain a transfer function of sound source filtering, and convert the multi-channel stereo playback audio data into a dual-channel stereo signal of headphones according to the transfer function; The evaluation module is used to perform Mel segmentation on the dual-channel stereo information, extract feature vectors of the Mel segments of the stereo signal, and perform time-dependent analysis on the feature vectors to evaluate the sound quality of the dual-channel stereo signal.

9. A headset having a computer program stored thereon, characterized in that: When the program is executed by a processor, the headphone sound signal processing method according to any one of claims 1 to 7 is implemented.

10. A computer comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the headphone sound signal processing method according to any one of claims 1 to 7 is implemented.

Images