Earphone noise reduction method based on improved Transform model, earphone and storage medium

Abstract

The invention discloses an earphone noise reduction method based on an improved Transform model, an earphone and a storage medium. The earphone noise reduction method comprises the following steps: acquiring a current environment audio signal, position information and attitude information of the earphone; extracting an audio feature sequence from the environment audio signal; extracting an auxiliary feature sequence from the position information and the attitude information; inputting the audio feature sequence and the auxiliary feature sequence into an improved Transform model configured with a proxy attention mechanism to obtain a matching value of each scene type; and obtaining a target scene type according to the size of the matching value, and obtaining a noise reduction parameter corresponding to the target scene type. By improving the Transform model and the multi-source feature information, the recognition and classification functions of the complex audio scene where the earphone is located are achieved, the efficiency and accuracy of recognizing the complex audio scene by the earphone are improved while low calculation complexity is kept, then the matching degree of adaptive noise reduction of the earphone is improved, the adaptive noise reduction effect of the earphone is improved, and the user experience is improved. And the user experience is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of headphone noise reduction, and in particular to a headphone noise reduction method based on an improved Transformer model, a headphone and a storage medium. Background Art

[0002] With the rapid development of electronic technology, wireless headphones have become an indispensable item for people's daily travel. When facing various relatively noisy environments, people usually choose to wear wireless headphones to reduce the impact of environmental noise.

[0003] Different noise environments have different requirements for noise reduction levels. To better cope with the changing noise environment, some existing wireless headphones use adaptive noise reduction to reduce the impact of environmental noise. However, the existing adaptive noise reduction method cannot cope with complex sound scenes well. Single recognition feature information has certain limitations in the recognition and classification of sound scenes, which leads to relatively poor accuracy and efficiency of scene recognition, and in turn leads to low matching of the noise reduction level required for headphones, affecting the user experience.

[0004] In view of this, it is necessary to provide a headphone noise reduction method, headphone and storage medium based on an improved Transformer model to solve the above problems. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention provides a headphone noise reduction method, headphone and storage medium based on an improved Transformer model, which aims to solve the technical problem that the accuracy and efficiency of scene recognition of existing headphones are low when dealing with complex sound scenes, which leads to poor adaptive noise reduction effect of the headphones.

[0006] To achieve the above-mentioned purpose, a first aspect of the present invention provides a headphone noise reduction method based on an improved Transformer model, which comprises:

[0007] Get the current ambient audio signal, location information and posture information of the headset;

[0008] Extracting an audio feature sequence from an ambient audio signal;

[0009] Extracting auxiliary feature sequences from position information and posture information;

[0010] The audio feature sequence and the auxiliary feature sequence are input into the improved Transformer model configured with the proxy attention mechanism to obtain the matching value of each scene type;

[0011] According to the size of the matching value, the target scene type is obtained, and the noise reduction parameters corresponding to the target scene type are obtained.

[0012] In a preferred embodiment, the step of inputting the audio feature sequence and the auxiliary feature sequence into an improved Transformer model configured with a proxy attention mechanism to obtain the matching value of each scene type includes:

[0013] Mapping and encoding the audio feature sequence and the auxiliary feature sequence to obtain an audio enhancement sequence and an auxiliary enhancement sequence respectively;

[0014] Based on the audio enhancement sequence, the auxiliary enhancement sequence and the preset proxy attention mechanism, an enhanced feature matrix is ​​obtained;

[0015] Based on the preset fully connected layer and enhanced feature matrix, the matching value of each scene type is obtained.

[0016] In a preferred embodiment, the step of obtaining an enhanced feature matrix based on the audio enhancement sequence, the auxiliary enhancement sequence and the preset proxy attention mechanism includes:

[0017] According to the audio enhancement sequence and the preset conversion weight matrix, a first matrix, a second matrix and a third matrix are obtained respectively;

[0018] According to the auxiliary enhancement sequence and the preset proxy conversion matrix, a proxy matrix is ​​obtained;

[0019] Interactively calculate the proxy matrix, the second matrix and the third matrix to obtain a fusion matrix;

[0020] Interactively calculate the first matrix, the proxy matrix and the fusion matrix to obtain the attention weight matrix;

[0021] The attention weight matrix is ​​weighted summed with the third matrix to obtain the enhanced feature matrix.

[0022] In a preferred embodiment, the step of obtaining the matching value of each scene type based on the preset fully connected layer and the enhanced feature matrix includes:

[0023] The enhanced feature matrix passes through the fully connected layer to obtain the prediction score for each scene type;

[0024] Based on the prediction score and the preset softmax activation function, the matching value of the corresponding scene type is obtained.

[0025] In a preferred embodiment, the step of extracting an audio feature sequence from an ambient audio signal comprises:

[0026] Preprocessing the ambient audio signal to obtain a noise-removed audio signal;

[0027] Convert the noise-free frequency signal into the ambient audio spectrum;

[0028] Divide the ambient audio spectrum into a number of target spectra;

[0029] The audio features are extracted from the target spectrum in sequence to obtain an audio feature sequence.

[0030] In a preferred embodiment, the step of extracting the auxiliary feature sequence from the position information and the posture information includes:

[0031] Preprocess the position information and attitude information respectively to obtain denoised position information and denoised attitude information;

[0032] Extract position features from the denoised position information to obtain a position feature sequence;

[0033] Extract posture features from the denoised posture information to obtain a posture feature sequence;

[0034] The position feature sequence and the posture feature sequence are fused to obtain an auxiliary feature sequence.

[0035] In a preferred embodiment, the step of extracting position features from the denoised position information to obtain a position feature sequence includes:

[0036] Extracting the current geographical location information of the headset from the denoising location information;

[0037] Determine the initial information of the scene type based on the geographic location information and the preset geographic information database;

[0038] The initial information of the scene type is encoded and transformed to obtain a position feature sequence.

[0039] In a preferred embodiment, the step of extracting posture features from the denoised posture information to obtain a posture feature sequence includes:

[0040] Extract the current acceleration features of the headset from the denoised posture information;

[0041] The acceleration features are encoded and transformed to obtain the posture feature sequence.

[0042] A second aspect of the present invention provides a headset, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of any one of the above-mentioned methods for headphone noise reduction based on the improved Transformer model are implemented.

[0043] A third aspect of the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the computer program implements the steps of any of the above-mentioned headphone noise reduction methods based on the improved Transformer model.

[0044] The beneficial effects of the present invention are as follows: by improving the Transformer model and multi-source feature information, the recognition and classification functions of the complex audio scenes in which the headphones are located are realized; by utilizing the proxy attention mechanism of the improved Transformer model, the integration and recognition of multi-source feature information are realized while maintaining low computational complexity; the recognition attention weight can be adjusted in combination with various information features, thereby improving the accuracy of the dynamic adjustment of the recognition attention weight, thereby improving the efficiency and accuracy of the headphones in recognizing complex audio scenes, thereby improving the matching degree of the headphones' adaptive noise reduction, improving the effect of the headphones' adaptive noise reduction, and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A first flow chart of a headphone noise reduction method based on an improved Transformer model disclosed in an embodiment of the present invention;

[0046] Figure 2 A second flow chart of the headphone noise reduction method based on the improved Transformer model disclosed in an embodiment of the present invention;

[0047] Figure 3 A third flow chart of the headphone noise reduction method based on the improved Transformer model disclosed in an embodiment of the present invention;

[0048] Figure 4 A fourth flow chart of the headphone noise reduction method based on the improved Transformer model disclosed in an embodiment of the present invention;

[0049] Figure 5 A fifth flow chart of the headphone noise reduction method based on the improved Transformer model disclosed in an embodiment of the present invention;

[0050] Figure 6 This is a schematic diagram of the module structure of the earphone disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In the present invention, the terms "disposed", "provided with" and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral structure; it can be a mechanical connection, or an electrical connection; it can be a direct connection, or an indirect connection through an intermediate medium, or it can be an internal connection between two devices, elements or components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0052] The terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "plurality" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0053] In addition, some of the above terms may be used to express other meanings in addition to indicating orientation or positional relationship. For example, the term "on" may also be used to express a certain dependency or connection relationship in some cases. For those skilled in the art, the specific meanings of these terms in the present invention can be understood according to specific circumstances.

[0054] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0055] The following is the content of the first aspect of the present invention:

[0056] Please refer to Figure 1 In this embodiment, the headphone noise reduction method based on the improved Transformer model includes the following steps:

[0057] S1. Obtain the current ambient audio signal, position information and posture information of the headset.

[0058] Among them, the earphones mentioned in this embodiment are earphones with active noise reduction function, which include a processor and an audio output unit. The processor can use the acquired noise reduction parameters to enable the audio output unit to generate an audio signal with a phase opposite to the ambient noise, thereby offsetting the external ambient noise.

[0059] The location information can be location information with the headset as the positioning center, or it can be location information with the terminal device connected to the headset as the positioning center. Accordingly, the location information of the headset can be obtained directly or indirectly. Specifically, a positioning module can be set inside the headset, and the location information of the current headset can be directly obtained through the positioning module; it can also be obtained indirectly through a terminal device (such as a mobile phone, smart watch, etc.) with a positioning module set inside, and the location information of the terminal device is approximately equivalent to the location information of the headset. In the daily use of the headset, the headset is mostly connected to a terminal device for use. Preferably, the location information of the terminal device can be used to indirectly obtain the location information of the headset.

[0060] The posture information of the headset can be detected by setting a posture sensor inside the headset, and the posture sensor includes but is not limited to an acceleration sensor, a gyroscope, a magnetic sensor, etc. The ambient audio signal can be obtained by a collection microphone array set on the headset. Specifically, the collection microphone array on the headset can obtain the sound signal of the environment where the headset is located in real time, and can also collect the sound signal of the environment where the headset is located within a preset period of time.

[0061] S2. Extracting an audio feature sequence from the ambient audio signal.

[0062] The audio feature sequence is extracted and transformed from the ambient audio signal, which can represent the sound characteristics of the current scene of the headset and can be input into the improved Transformer model.

[0063] When the microphone array set in the headset collects the ambient audio signal of the current scene of the headset, the ambient audio signal needs to be preprocessed to remove redundant or irrelevant interference information. After obtaining the preprocessed ambient audio signal, the obtained audio signal can be converted into a required type of spectrum diagram according to the actual design requirements, and then the obtained spectrum diagram can be divided into several sub-spectra according to the preset requirements, and then audio features are extracted from several sub-spectra. After obtaining the audio features, the extracted audio features are combined into an audio feature sequence.

[0064] S3. Extract auxiliary feature sequences from the position information and posture information.

[0065] Among them, the auxiliary feature sequence is data that can be input into the improved Transformer model, which is obtained by extracting feature information from the position information and the posture information respectively and transforming and fusing them to represent the current state characteristics of the headset.

[0066] After obtaining the location information of the current scene of the headset and the current posture information of the headset, the location information and posture information need to be preprocessed to remove redundant or irrelevant interference information. After completing the preprocessing of the location information and posture information, the required location features and posture features are extracted from the location information and posture information respectively, and the obtained location features and posture features are combined to obtain the location feature sequence and posture feature sequence, and then the two feature sequences are feature fused to obtain the required auxiliary feature sequence.

[0067] It is understandable that by fusing the position feature sequence with the posture feature sequence, the characteristic relationship between the current position of the headset and the headset posture can be better reflected from the overall level, preparing for the subsequent use of multimodal feature information to comprehensively adjust the agent's attention weight, and also reducing the complexity of subsequent calculations to a certain extent.

[0068] S4, input the audio feature sequence and the auxiliary feature sequence into the improved Transformer model configured with the proxy attention mechanism to obtain the matching value of each scene type;

[0069] S5. According to the size of the matching value, the target scene type is obtained, and the noise reduction parameters corresponding to the target scene type are obtained.

[0070] Among them, the improved Transformer model uses the encoding module of the Transformer model and quotes the proxy attention mechanism to replace the original self-attention mechanism in the Transformer model, which can realize more accurate and efficient recognition and classification of complex sound scenes by utilizing multi-source information features.

[0071] The proxy attention mechanism consists of two softmax attention operations. A set of additional proxy vectors A with fewer parameters is introduced into the original attention mechanism's ternary attention paradigm (Q, K, V), forming a new quaternary proxy attention paradigm (Q, A, K, V). Specifically, the proxy attention mechanism is characterized in that it first uses A with fewer parameters to replace Q with more parameters to query and collect information in K and V, and then fuses the feature information contained in A, K and V, and then broadcasts the information obtained from the above operations to Q, establishing a cross-modal information fusion mechanism, realizing the interactive fusion of multi-source feature information, and avoiding direct calculations between Q and K, completing the modeling of global information with lower computational complexity.

[0072] After the audio feature sequence and the auxiliary feature sequence are input into the improved Transformer model, the audio feature sequence and the auxiliary feature sequence are further processed to convert them into high-dimensional audio enhancement sequences and auxiliary enhancement sequences with sorting information. The obtained audio enhancement sequence and auxiliary enhancement sequence are then passed through the proxy attention mechanism of the Transformer model to obtain an enhanced feature matrix containing weight information. The enhanced feature matrix is ​​then transformed and calculated through the fully connected layer to obtain the matching value of each scene type.

[0073] Specifically, in the process of calculating the proxy attention weight, the audio enhancement sequence will convert the Q, K and V matrices with different weights and functions, and the auxiliary enhancement sequence will convert the proxy matrix A, that is, generate the corresponding four-element proxy attention paradigm (Q, A, K, V), and then use A, K and V to perform the first softmax attention operation to obtain the fusion matrix V' that integrates the feature information of both the audio enhancement sequence and the auxiliary enhancement sequence, and then use Q, A and V' to perform the second softmax attention operation to obtain the required attention weight matrix, and then use the weighted sum of the attention weight matrix and V to obtain the enhanced feature matrix containing weight information.

[0074] Based on the obtained enhanced feature matrix and the fully connected layer, the matching value of each scene type can be determined, and then the maximum value among the matching values ​​can be screened out. The scene type corresponding to the maximum matching value is the target scene type, and then the noise reduction parameters of the target scene type can be obtained to realize the adaptive noise reduction function of the headphones.

[0075] It is understandable that by improving the Transformer model and multi-source feature information, the recognition and classification function of the complex audio scene in which the headphones are located is realized. By utilizing the proxy attention mechanism of the improved Transformer model, while maintaining low computational complexity, the integration and recognition of multi-source feature information is realized, the accuracy of the dynamic adjustment of the recognition attention weight is improved, and the efficiency and accuracy of the headphones in recognizing complex audio scenes are improved, thereby improving the matching degree of the headphones' adaptive noise reduction, improving the effect of the headphones' adaptive noise reduction, and improving the user experience.

[0076] For further information, please refer to Figure 2 In one embodiment, step S2 of extracting an audio feature sequence from an ambient audio signal comprises:

[0077] S21, preprocessing the ambient audio signal to obtain a noise-removed audio signal;

[0078] S22, converting the noise-removed frequency signal into an ambient audio spectrum;

[0079] S23, dividing the ambient audio spectrum into a plurality of target spectra;

[0080] S24, extracting audio features from the target spectrum in sequence, and then obtaining an audio feature sequence.

[0081] The target spectrum is a sub-spectrum obtained by dividing the ambient audio spectrum.

[0082] Specifically, after collecting the ambient audio signal of the scene in which the headset is currently located, since the ambient audio signal may be mixed with interference signals such as background noise, it is necessary to perform preprocessing such as filtering on the ambient audio signal to obtain a denoised audio signal.

[0083] After obtaining the denoised audio signal, in order to further extract the audio features, the denoised audio signal needs to be converted into an ambient audio spectrum that can extract the required audio features. It is easy to understand that the type of ambient audio spectrum can be selected according to the design requirements, and the ambient audio spectrum includes but is not limited to the ambient audio spectrum obtained by short-time Fourier transform, Mel spectrum, etc. Preferably, since the sensitivity of the human ear to different frequencies of sound is different, in order to extract audio features that are more in line with the auditory characteristics of the human ear, the denoised audio signal can be converted into a Mel spectrum.

[0084] After obtaining the ambient audio spectrum, it is necessary to divide the ambient audio spectrum into several target spectra to facilitate the subsequent audio feature extraction. Specifically, the ambient audio spectrum can be divided in a linear or nonlinear way. Preferably, the ambient audio spectrum is divided in a nonlinear way, and the frequency bands to which the human ear is relatively highly sensitive are intercepted with a relatively denser width, and the frequency bands to which the human ear is relatively less sensitive are intercepted with a relatively sparser width, so that the overall distribution of the obtained target spectrum is more consistent with the auditory characteristics of the human ear, so as to improve the accuracy of subsequent feature extraction.

[0085] After obtaining several target spectra, audio features are extracted from the target spectra in turn, and the obtained audio features are sorted and combined to obtain an audio feature sequence. Specifically, feature encoders such as convolutional neural networks or recurrent neural networks can be used to extract high-order features, or spectral features, statistical features and other features of sub-spectra can be directly extracted to obtain audio features. Preferably, feature encoders such as convolutional neural networks or recurrent neural networks can be used to extract high-order features as audio features.

[0086] For further information, please refer to Figure 3 In one embodiment, step S3 of extracting the auxiliary feature sequence from the position information and the posture information includes:

[0087] S31, pre-processing the position information and the posture information respectively to obtain denoised position information and denoised posture information;

[0088] S32, extracting position features from the denoised position information to obtain a position feature sequence;

[0089] S33, extracting posture features from the denoised posture information to obtain a posture feature sequence;

[0090] S34, performing feature fusion on the position feature sequence and the posture feature sequence to obtain an auxiliary feature sequence.

[0091] The position feature is key information in the position information that can be used to represent the position characteristics of the current scene of the headset to a certain extent. The posture feature is key information in the posture feature that can be used to represent the current state of the headset to a certain extent.

[0092] Specifically, after obtaining the current position information and posture information of the headset, since both the position information and the posture information may be mixed with interference information or repeated meaningless information, it is necessary to denoise the position information and the posture information to obtain denoised position information and denoised posture information.

[0093] After obtaining the denoised position information and denoised posture information, feature extraction can be performed on the denoised position information and denoised posture information according to preset requirements. Specifically, the position features include but are not limited to geographic location information, scene type information, and point of interest information; posture feature information includes but is not limited to headset wearing posture information and headset acceleration information. It can be understood that the position features extracted from the denoised position information can be only a single type of position features or several types of position features; similarly, the posture features can be only a single type of posture features or several types of posture features, which can be selected according to actual design requirements.

[0094] After obtaining the position features and the posture features, the position features can be combined to obtain the position feature sequence, and the posture features can be combined to obtain the posture feature sequence. After the obtained position feature sequence and posture feature sequence are subjected to feature fusion, an auxiliary feature sequence can be obtained. Specifically, the method of feature fusion of the position feature sequence and the posture feature sequence includes but is not limited to direct splicing and weighted splicing between sequences, which can be selected according to the design requirements. Preferably, the weighted splicing method is adopted to better divide the weight between the position information and the posture information according to the actual situation, thereby improving the adaptability of the subsequent adjustment of the attention weight.

[0095] It can be understood that by obtaining an auxiliary feature sequence obtained by feature fusion of the position feature sequence and the posture feature sequence, the characteristic relationship between the current position of the headset and the headset posture can be better reflected from the overall level, enriching the amount of recognition information for subsequent sound scene classification, assisting in adjusting the proxy attention weight, improving the accuracy of the proxy attention weight, and thereby improving the accuracy of the sound scene classification.

[0096] Furthermore, in a preferred embodiment, the step S32 of extracting position features from the denoised position information to obtain a position feature sequence includes:

[0097] S321, extracting the current geographical location information of the headset from the denoising location information;

[0098] S322, determining the scene type initial information according to the geographical location information and a preset geographical information database;

[0099] S323: Encode and transform the initial scene type information to obtain a position feature sequence.

[0100] The geographic location information includes the latitude and longitude information and altitude information of the current location of the headset. The scene type initial information is information about the scene type in which the headset may be currently located, which is preliminarily determined based on the current geographic location information of the headset.

[0101] After extracting the current latitude and longitude information and altitude information of the headset from the denoised location information, the current geographic location information of the headset can be used to preliminarily determine the scene type that the headset is currently in through the geographic information database. It can be understood that after the geographic location information interacts with the geographic information database, the geographic information database can map the initial scene type that the current geographic location information of the headset may conform to. Due to the complexity of the actual scene, the initial scene type information preliminarily determined by the geographic location information may be a single scene type or multiple scene types. In order to ensure the comprehensiveness of the location feature sequence, all the scene types that are preliminarily determined to meet the requirements should be obtained as the initial scene type.

[0102] Specifically, the interaction between the geographic location information and the geographic information library can be achieved through a terminal connected to the headset. The interaction can be achieved through online interaction in the cloud, that is, the initial information of the scene type is determined through real-time data exchange between the terminal device and the cloud, or offline interaction, that is, the geographic information library can be pre-downloaded to the APP of the headset of the terminal. Preferably, the interaction method is offline interaction, which can better adapt to actual usage and improve the efficiency of obtaining the initial information of the scene type to a certain extent.

[0103] In a preferred embodiment, the user can choose to download the geographic information library of the required area in advance in the headset APP of the terminal according to their own usage needs. When the terminal receives a signal that requires the determination of the initial information of the scene type, the initial information of the scene type can be directly determined from the downloaded geographic information library, and then the initial information of the scene type corresponding to the current geographic location information can be determined relatively quickly. At the same time, after the headset is connected to the terminal, it will automatically trigger the detection of whether the geographic information library needs to be updated. If necessary, it will automatically update to ensure the accuracy of the initial scene type determined subsequently.

[0104] After determining the initial scene type information, the obtained initial scene type needs to be encoded and transformed to obtain a position feature sequence to ensure that the position feature sequence can be input into the improved Transformer model so as to be recognized and analyzed by the improved Transformer model.

[0105] It can be understood that the position feature sequence determined by the initial information of the scene type can be used to assist in adjusting the recognition weights of the proxy attention mechanism of the improved Transformer model according to the initial information of the scene type, thereby improving the accuracy of subsequent calculations of the proxy attention weights, thereby improving the accuracy and efficiency of the classification of complex sound scenes.

[0106] Furthermore, in a preferred embodiment, step S33 of extracting posture features from the denoised posture information to obtain a posture feature sequence includes:

[0107] S331, extracting the current acceleration feature of the headset from the denoised posture information;

[0108] S332. Encode and transform the acceleration features to obtain a posture feature sequence.

[0109] Among them, the acceleration feature can be used to reflect the current motion state of the headset, that is, the acceleration feature can be used to determine whether the headset is currently in a dynamic or static state as a whole, and then determine the current motion state of the user.

[0110] Specifically, acceleration data including acceleration components of the headset in the three directions of X-axis, Y-axis and Z-axis are obtained from the denoised posture information, and then the acceleration data are feature extracted to obtain acceleration features. It can be understood that acceleration features include but are not limited to peak acceleration, average acceleration, acceleration variance, etc., which can be selected and extracted according to actual design requirements.

[0111] After obtaining the required acceleration features, the obtained acceleration features are encoded and transformed to obtain a posture feature sequence, so as to ensure that the posture feature sequence can be input into the improved Transformer model so as to be recognized and analyzed by the improved Transformer model.

[0112] It can be understood that the posture feature sequence determined by the acceleration feature can determine the user's current motion state, and can assist in adjusting the recognition weights of the proxy attention mechanism of the improved Transformer model according to the user's current motion state, thereby improving the accuracy of subsequent calculations of the proxy attention weights, and thereby improving the accuracy and efficiency of the classification of complex sound scenes.

[0113] For further information, please refer to Figure 4In one embodiment, the audio feature sequence and the auxiliary feature sequence are input into the improved Transformer model configured with the proxy attention mechanism, and the step S4 of obtaining the matching value of each scene type includes:

[0114] S41, mapping and encoding the audio feature sequence and the auxiliary feature sequence to obtain an audio enhancement sequence and an auxiliary enhancement sequence respectively;

[0115] S42, obtaining an enhanced feature matrix based on the audio enhancement sequence, the auxiliary enhancement sequence and the preset proxy attention mechanism;

[0116] S43. Based on the preset fully connected layer and enhanced feature matrix, a matching value of each scene type is obtained.

[0117] Among them, the audio enhancement sequence and the auxiliary enhancement sequence are high-dimensional data with sorting information obtained after the conversion of the audio feature sequence and the auxiliary feature sequence respectively. The enhanced feature matrix is ​​the data with weight information obtained after calculation through the proxy attention mechanism. The matching value is the matching degree between the preset scene type and the sound scene currently in which the headset is located. The larger the matching value, the higher the matching degree, and vice versa.

[0118] After the audio feature sequence and the auxiliary feature sequence are input into the improved Transformer model, the audio feature sequence and the auxiliary feature sequence need to be linearly mapped through the embedding layer respectively, and each feature vector in the audio feature sequence and the auxiliary feature sequence is mapped to a higher-dimensional embedding space, thereby obtaining an audio embedding vector sequence and an auxiliary embedding vector sequence respectively. After obtaining the audio embedding vector sequence and the auxiliary embedding vector sequence, since the improved Transformer model does not have built-in position information, it is also necessary to add position encoding to indicate the order of the feature vectors in the feature sequence. After completing the position encoding, the audio enhancement sequence and the auxiliary enhancement sequence can be obtained.

[0119] After the audio enhancement sequence and the auxiliary enhancement sequence are input into the proxy attention mechanism in the improved Transformer model, the proxy attention mechanism uses the audio enhancement sequence and the auxiliary enhancement sequence to perform two softmax attention operations respectively. Based on the overall level, from the perspective of multi-source feature information fusion, the initial attention weight matrix is ​​obtained, and then the initial attention matrix is ​​normalized to obtain the attention weight matrix. The attention weight matrix is ​​further used for weighted summation to obtain the enhanced feature matrix containing the weight information of each feature vector.

[0120] For details, please refer to Figure 5In a preferred embodiment, step S42 of obtaining an enhanced feature matrix based on the audio enhancement sequence, the auxiliary enhancement sequence and the preset proxy attention mechanism includes:

[0121] S421, obtaining a first matrix, a second matrix, and a third matrix respectively according to the audio enhancement sequence and a preset conversion weight matrix;

[0122] S422, obtaining a proxy matrix according to the auxiliary enhancement sequence and a preset proxy conversion matrix;

[0123] S423, interactively calculating the proxy matrix, the second matrix, and the third matrix to obtain a fusion matrix;

[0124] S424, interactively calculating the first matrix, the proxy matrix, and the fusion matrix to obtain an attention weight matrix;

[0125] S425. Perform a weighted summation on the attention weight matrix and the third matrix to obtain an enhanced feature matrix.

[0126] The conversion weight matrix is ​​obtained in advance through training and is used to convert the audio enhancement sequence into data that can be used by the agent attention mechanism. The conversion weight matrix includes a first conversion matrix W corresponding to the first matrix, the second matrix and the third matrix respectively. A , the second transformation matrix W K and the third transformation matrix W V The proxy conversion matrix is ​​data obtained in advance through training and used to convert the auxiliary enhancement sequence into a proxy matrix applied to the proxy attention mechanism.

[0127] Specifically, after the audio enhancement sequence is input into the proxy attention mechanism, the first transformation matrix W A , the second transformation matrix W K and the third transformation matrix W V The first matrix, the second matrix and the third matrix are obtained by converting and calculating with the audio enhancement sequence respectively. After the auxiliary enhancement sequence is input into the proxy attention mechanism, the auxiliary enhancement sequence is calculated with the proxy conversion matrix to obtain the proxy matrix. The number of parameters of the proxy matrix is ​​much less than that of the first matrix.

[0128] After completing the conversion between the audio enhancement sequence and the auxiliary enhancement sequence, the first softmax attention operation is performed first, with the proxy matrix replacing the first matrix as the query matrix, the second matrix as the key matrix, and the third matrix as the value matrix, to achieve the interaction between the proxy matrix, the second matrix, and the third matrix, so as to aggregate the information in the audio enhancement sequence, and then fuse the aggregated information with the information in the auxiliary enhancement sequence. Specifically, each proxy vector in the proxy matrix is ​​dot-multiplied with each key vector in the second matrix to obtain a first attention score that reflects the strength of the correlation between different positions in the sequence, and then the first attention score is normalized to obtain a first attention weight, and then the first attention weight is weighted and summed with the value vector in the third matrix to obtain a fusion matrix.

[0129] It can be understood that by replacing the first matrix as the query matrix with a proxy matrix with fewer parameters, the direct interactive calculation between the first matrix and the second matrix can be avoided to reduce the complexity of the calculation. At the same time, a multi-source feature information fusion mechanism is established to integrate audio feature information, position feature information and posture feature information, so as to prepare for the subsequent recognition and classification of complex sound scenes.

[0130] After obtaining the information output by the first attention operation, the proxy matrix broadcasts the obtained information back to the first matrix. The second softmax attention operation is performed with the first matrix as the query matrix, the proxy matrix as the key matrix, and the fusion matrix as the value matrix to realize the interaction between the first matrix, the proxy matrix and the fusion matrix. Similar to the first softmax attention operation, the second attention score is first calculated using the first matrix and the proxy matrix, and then the second attention score is normalized to obtain the second attention weight, and then the second attention weight is weighted and summed with the fusion matrix to obtain the attention weight matrix. After obtaining the attention weight matrix, the attention weight matrix is ​​weighted and summed with the third matrix to obtain an enhanced feature matrix containing the weight relationship of each eigenvector.

[0131] It is understandable that the proxy attention mechanism can effectively process and integrate multi-source data information in complex acoustic environments, achieving a good balance between computational complexity and classification accuracy, and significantly improving the accuracy and efficiency in identifying complex audio scenes.

[0132] Furthermore, in a preferred embodiment, step S43 of obtaining the matching value of each scene type based on the preset fully connected layer and the enhanced feature matrix includes:

[0133] S431, the enhanced feature matrix is ​​passed through a fully connected layer to obtain a prediction score for each scene type;

[0134] S432: Based on the prediction score and the preset softmax activation function, a matching value corresponding to the scene type is obtained.

[0135] Among them, the fully connected layer is the classifier structure of the improved Transformer model, which can be used to obtain the prediction scores of each scene type.

[0136] Specifically, after obtaining the enhanced feature matrix output by the proxy attention mechanism, the enhanced feature matrix is ​​input into the fully connected layer, which then performs feature mapping on the enhanced feature matrix. That is, the fully connected layer linearly transforms each feature vector in the enhanced feature matrix and outputs a vector with the same number of elements as the number of scene types, where each element of the vector corresponds to a prediction score for a scene type.

[0137] After completing the feature mapping and obtaining the prediction score, the softmax activation function is used to convert the obtained prediction score into a probability distribution between 0 and 1, and then the matching value of each scene type is obtained.

[0138] It is easy to understand that by converting the enhanced feature matrix containing weight information of multi-source features to obtain matching values ​​corresponding to each scene type, the efficiency of determining the target scene type is improved.

[0139] In summary, this application realizes the recognition and classification function of the complex audio scene in which the headphones are located by improving the Transformer model and multi-source feature information. By utilizing the proxy attention mechanism of the improved Transformer model, while maintaining low computational complexity, the integration and recognition of multi-source feature information is realized, the accuracy of the dynamic adjustment of the recognition attention weight is improved, and the efficiency and accuracy of the headphones in recognizing complex audio scenes are improved, thereby improving the matching degree of the headphones' adaptive noise reduction, improving the effect of the headphones' adaptive noise reduction, and improving the user experience.

[0140] The following is the content of the second aspect of the present invention:

[0141] The present invention provides an earphone, such as Figure 6 As shown, the headset includes a memory 10, a processor 20, and a program instruction 30 for a headphone noise reduction method based on an improved Transformer model stored in the memory 10 and executable on the processor 20. When the program instruction 30 for the headphone noise reduction method based on the improved Transformer model is executed by the processor 20, the aforementioned headphone noise reduction method based on the improved Transformer model is implemented.

[0142] In some embodiments, the processor may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor is generally used to control the overall operation of the headset. In this embodiment, the processor is used to run program codes stored in a readable storage medium or process data.

[0143] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a readable storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods of various embodiments of the present invention.

[0144] The following is the content of the third aspect of the present invention:

[0145] The present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the headphone noise reduction method based on the improved Transformer model are implemented.

[0146] The above is only a specific implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A headphone noise reduction method based on an improved Transformer model, characterized in that: include: Get the current ambient audio signal, location information and posture information of the headset; Extracting an audio feature sequence from the ambient audio signal; Extracting an auxiliary feature sequence from the position information and the posture information; Inputting the audio feature sequence and the auxiliary feature sequence into an improved Transformer model configured with a proxy attention mechanism to obtain matching values ​​for each scene type; According to the size of the matching value, the target scene type is obtained, and the noise reduction parameters corresponding to the target scene type are obtained.

2. The headphone noise reduction method based on the improved Transformer model according to claim 1, characterized in that: The steps of inputting the audio feature sequence and the auxiliary feature sequence into the improved Transformer model configured with the proxy attention mechanism to obtain the matching value of each scene type include: Mapping and encoding the audio feature sequence and the auxiliary feature sequence to obtain an audio enhancement sequence and an auxiliary enhancement sequence respectively; Based on the audio enhancement sequence, the auxiliary enhancement sequence and the preset proxy attention mechanism, an enhancement feature matrix is ​​obtained; Based on the preset fully connected layer and the enhanced feature matrix, a matching value for each scene type is obtained.

3. The headphone noise reduction method based on the improved Transformer model according to claim 2, characterized in that: Based on the audio enhancement sequence, the auxiliary enhancement sequence and the preset proxy attention mechanism, the step of obtaining an enhanced feature matrix includes: According to the audio enhancement sequence and a preset conversion weight matrix, a first matrix, a second matrix and a third matrix are obtained respectively; Obtaining a proxy matrix according to the auxiliary enhancement sequence and a preset proxy conversion matrix; Interactively calculating the proxy matrix, the second matrix and the third matrix to obtain a fusion matrix; Interactively calculate the first matrix, the proxy matrix and the fusion matrix to obtain an attention weight matrix; The attention weight matrix is ​​weighted and summed with the third matrix to obtain an enhanced feature matrix.

4. The headphone noise reduction method based on the improved Transformer model according to claim 2, characterized in that: The step of obtaining the matching value of each scene type based on the preset fully connected layer and the enhanced feature matrix includes: The enhanced feature matrix passes through the fully connected layer to obtain a prediction score for each scene type; Based on the prediction score and the preset softmax activation function, a matching value corresponding to the scene type is obtained.

5. The headphone noise reduction method based on the improved Transformer model according to claim 1, characterized in that: The step of extracting an audio feature sequence from the ambient audio signal comprises: Preprocessing the ambient audio signal to obtain a noise-removed audio signal; Converting the de-noised audio signal into an ambient audio spectrum; Splitting the ambient audio spectrum into a number of target spectra; The audio features are extracted from the target spectrum in sequence to obtain an audio feature sequence.

6. The headphone noise reduction method based on the improved Transformer model according to claim 1, characterized in that: The step of extracting an auxiliary feature sequence from the position information and the posture information comprises: Preprocessing the position information and the attitude information respectively to obtain denoised position information and denoised attitude information; Extracting position features from the denoised position information to obtain a position feature sequence; Extracting posture features from the denoised posture information to obtain a posture feature sequence; The position feature sequence and the posture feature sequence are subjected to feature fusion to obtain an auxiliary feature sequence.

7. The headphone noise reduction method based on the improved Transformer model according to claim 6, characterized in that: The step of extracting position features from the denoised position information to obtain a position feature sequence comprises: Extracting the current geographical location information of the headset from the denoising location information; Determining scene type initial information according to the geographic location information and a preset geographic information database; The initial scene type information is encoded and transformed to obtain a position feature sequence.

8. The headphone noise reduction method based on the improved Transformer model according to claim 6, characterized in that: The step of extracting posture features from the denoised posture information to obtain a posture feature sequence comprises: Extracting the current acceleration feature of the headset from the denoised posture information; The acceleration features are coded and transformed to obtain a posture feature sequence.

9. A headset comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the headphone noise reduction method based on the improved Transformer model as described in any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the headphone noise reduction method based on the improved Transformer model as described in any one of claims 1 to 8 are implemented.

Images