Intelligent glasses audio processing method and system

Abstract

The invention discloses an intelligent glasses audio processing method and system. The method comprises the following steps: data preparation: collecting source domain data and target domain data; pre-training the model; converting the audio signal into sequence data; an RNN model is constructed and trained; model migration: migrating the pre-trained RNN model to target domain data, and processing noise in the target domain data by using parameters and features of the pre-trained model; intelligent analysis and processing, wherein noise in target domain data is analyzed and processed by using the migrated RNN model; real-time processing and feedback are carried out, audio signals captured by a microphone of the intelligent glasses are analyzed in real time, and a noise reduction strategy is dynamically adjusted to eliminate background noise; and optimizing noise reduction parameters and strategies according to the feedback. By collecting the source domain data and performing model pre-training, the RNN model can identify and process various known noise types and features and quickly adapt to different noise environments, so that the noise reduction effect is improved, and a user can obtain the optimal hearing experience in different noise environments.

Description

Technical Field

[0001] The present invention belongs to the technical field of smart wearable devices and relates to a smart glasses audio processing method and system. Background Art

[0002] With the rapid development of science and technology, smart wearable devices have gradually integrated into people's daily lives and become a bridge connecting the digital world and the physical world. Among them, smart glasses, as the leader of smart wearable devices, have successfully attracted the attention of consumers with their unique fashion appearance and powerful functions. Smart glasses can not only provide convenient information display, interactive control and other functions, but also bring users an immersive auditory experience through the built-in audio system.

[0003] However, despite the increasing functionality of smart glasses, there are still some urgent problems to be solved in audio processing. First of all, poor noise reduction is a major shortcoming in the current audio processing of smart glasses. In noisy environments, such as public transportation, outdoor activities and other scenes, smart glasses often find it difficult to effectively isolate external noise, resulting in users being unable to clearly hear the content of calls or music. Secondly, bland sound effects are also one of the problems that users report more frequently. Existing smart glasses often lack a sense of hierarchy and detail in sound processing, making it impossible for users to get a satisfactory auditory experience when enjoying music or watching videos. In addition, inaccurate sound positioning is also a difficult problem in the audio processing of smart glasses. In a complex sound environment, smart glasses often find it difficult to accurately judge the direction and distance of the sound source, which not only affects the user's call experience, but may also cause users to miss key information in scenes such as outdoor sports and meeting records.

[0004] In summary, existing smart glasses still have many deficiencies in audio processing, which not only limit the expansion of smart glasses in more application scenarios, but also seriously affect the user experience. Therefore, how to improve the audio processing capabilities of smart glasses has become a technical problem that needs to be solved urgently in the field of smart wearable devices. Summary of the invention

[0005] The purpose of the present invention is to solve the problem that the noise reduction effect of smart glasses in the prior art is poor, and to provide a smart glasses audio processing method and system.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions: A smart glasses audio processing method comprises the following steps: Data preparation: Collect source domain data, which contains known noise types and characteristics and is collected and annotated in a specific noise environment; collect target domain data, which is to be processed and comes from different noise environments with different noise types and characteristics; Model pre-training: training a noise processing model on source domain data to identify and process noise in source domain data; Audio signal preprocessing: converting the audio signal captured by the smart glasses microphone into sequence data; preprocessing the sequence data, including normalization and segmentation, to meet the input requirements of the subsequent RNN model; RNN model construction and training, construct the RNN model, the model includes input layer, hidden layer and output layer; the input layer receives the preprocessed audio sequence data; the hidden layer processes the data to capture the long-term dependencies in the audio signal; the output layer outputs the denoised audio signal; initialize the parameters of the RNN model, and use the noisy audio data and the corresponding noise-free audio data to train the RNN model; Model migration: migrate the pre-trained RNN model to the target domain data, use the parameters and features of the pre-trained model to process the noise in the target domain data; adjust the model according to the noise type and characteristics of the target domain data; Intelligent analysis and processing: Use the migrated RNN model to intelligently analyze and process the noise in the target domain data; the model automatically identifies the type and characteristics of the noise and adjusts the noise reduction parameters and strategies based on the information; Real-time processing and feedback: The RNN model analyzes the audio signal captured by the smart glasses microphone in real time and dynamically adjusts the noise reduction strategy to eliminate background noise; based on user feedback, it optimizes the noise reduction parameters and strategies.

[0007] The audio signal preprocessing specifically includes: When converting an audio signal into sequence data, the audio signal is first sampled, and the amplitude value of the audio signal is extracted at a set time interval, wherein the sampling rate is adjusted according to the required accuracy and the amount of calculation; then the continuous amplitude value obtained by sampling is converted into a discrete value, including integer or floating point representation; finally, the quantized amplitude value is stored as sequence data in chronological order, using an array or list structure for storage; The sequence data is then preprocessed. First, the maximum and minimum values ​​of the sequence data are calculated; based on the maximum and minimum values, each amplitude value of the sequence data is scaled to a specified range to eliminate the amplitude differences between different audio signals; the length of each subsequence, that is, the window size, is determined, and the sequence data is divided according to the window size to obtain multiple subsequences for model processing.

[0008] The RNN model construction and training specifically include: Model construction: The input layer is set up to receive the preprocessed audio sequence data as input, which exists in the form of a time series, with each time point corresponding to an amplitude value or spectral feature; convert the audio data into a format that the model can process, and determine the dimensions of the input data, including the time step and the number of features; Hidden layer construction: at least one hidden layer is set in the core part of the RNN model to capture the long-term dependencies in the audio signal; RNN units are used in the hidden layer to process sequence data and pass hidden states between time steps to capture the timing information in the audio signal; Output layer setting: Design the output layer according to the specific task to output the prediction results. In the denoising task, the output layer outputs the denoised audio signal. In the classification task, the output layer outputs the category label of the audio signal. The output layer uses the fully connected layer and the softmax layer to process the output of the hidden layer and generate the final prediction results. Model definition and training: Use a deep learning framework to define the RNN model, including the specific structure and parameters of the input layer, hidden layer, and output layer; Use the noisy audio data and the corresponding noise-free audio data as training data sets to optimize the model parameters; Define the loss function, select the optimizer, and perform iterative training; during the training process, calculate the loss through forward propagation, and update the model parameters through back propagation and the optimizer.

[0009] Parameter initialization: Initialize the parameters of the RNN model during model construction; Verification and testing: Use validation set data to verify the performance of the RNN model, including accuracy, stability, and generalization ability; Use the test set data to test the performance of the RNN model and evaluate the accuracy and efficiency of the model; in the classification task, use the softmax layer to convert the output of the hidden layer into a probability distribution, and select the category with the highest probability as the prediction result.

[0010] The model migration is specifically as follows: Select a noise processing model trained on the source domain data, and use the API provided by the deep learning framework to load the weights and architecture of the selected pre-trained model; Prepare the output layer of the target task. For audio signal classification tasks, the output layer needs to be able to predict the category to which the audio signal belongs, so the dimension of the output layer matches the number of target categories; Design the output layer. According to the number of target categories, design a fully connected layer with the number of output nodes equal to the number of categories. Add a softmax layer after the fully connected layer to convert the output into a probability distribution. Connect the pre-trained model with the new output layer and remove the original output layer of the pre-trained model; Connect the designed fully connected layer and softmax layer to the end of the pre-trained model to form a new model architecture to meet the processing requirements of the target domain data; During the migration process, the new model architecture is adjusted according to the noise type and characteristics of the target domain data to optimize the performance of the model when processing the target domain data.

[0011] The intelligent analysis and processing are specifically: Automatic noise recognition: The migrated model recognizes the noise type and features in the target domain data, extracts the time domain and frequency domain features of the audio signal, and compares them with the noise feature library built into the model to accurately identify the noise. Dynamic adjustment of noise reduction strategies and parameters: Based on the identified noise type and characteristics, the model selects and applies low-pass filters, high-pass filters or band-pass filters for noise reduction to adapt to different types of noise; The model uses spectral subtraction to reduce noise, by subtracting the estimated noise spectrum from the spectrum of noisy speech to obtain a purer speech signal; Real-time processing and user feedback optimization: During the speech recognition process, the model analyzes the input audio signal in real time and dynamically adjusts the noise reduction strategy to eliminate background noise in real time and improve the accuracy and clarity of speech recognition. At the same time, the model optimizes the noise reduction parameters and strategies based on user feedback to form a closed-loop optimization mechanism to continuously improve the noise reduction performance.

[0012] The spectral subtraction method is specifically: Preprocessing: obtaining a noisy speech signal containing a clean speech signal and a noise signal; framing processing, cutting a continuous audio signal into multiple frames; windowing, performing windowing processing on each frame signal to improve the accuracy of spectrum analysis; Noise estimation: Silence segment detection, detecting silence segments in speech signals; Noise spectrum calculation, calculating the spectrum of each frame signal within the detected silence segment and taking the average value as the estimated value of the noise spectrum; Spectral subtraction processing: calculation of the spectrum of noisy speech, fast Fourier transform of the noisy speech signal to obtain its spectrum representation; subtraction operation, for each frame of signal, subtract the estimated noise spectrum from the spectrum of the noisy speech; negative value processing, if a negative spectrum is generated after the subtraction operation, at least one of the following methods is used for processing: truncation to zero: set the negative value directly to zero; set lower limit: set a lower limit value for the spectrum, and when the spectrum value is lower than this lower limit, set it to this lower limit value; spectrum lower limit parameter: use the spectrum lower limit parameter to control the amount of residual noise and the size of the music noise; Post-processing: Inverse Fourier transform: convert the processed spectrum back to time domain signal through inverse Fourier transform; overlap and add, since overlap is used in frame processing, the processed frame signals are overlapped and added to restore the original continuous signal.

[0013] The following steps are also included: Using historical noise data and current environmental parameters, including temperature, humidity, and time, a machine learning algorithm is used to predict the noise environment change trend over a period of time in the future. According to the noise environment prediction results, the noise reduction strategy of the RNN model is adjusted in advance to cope with the upcoming changes in the noise environment and ensure the continuity and stability of the noise reduction effect.

[0014] A smart glasses audio processing system includes the following modules: Data collection module: collects source domain data, which contains known noise types and characteristics and is collected and labeled in a specific noise environment; collects target domain data, which comes from different noise environments, has different noise types and characteristics, and is to be processed; Model pre-training module: trains a noise processing model on source domain data. The model is used to identify and process noise in source domain data. Audio signal preprocessing module: captures audio signals through the microphone of smart glasses and converts them into sequence data; preprocesses the sequence data, including normalization and segmentation, to meet the input requirements of the subsequent RNN model; RNN model construction and training module: construct an RNN model, which includes an input layer, a hidden layer, and an output layer; the input layer receives the preprocessed audio sequence data; the hidden layer is used for data processing to capture the long-term dependencies in the audio signal; the output layer outputs the denoised audio signal; initialize the parameters of the RNN model, and use the noisy audio data and the corresponding noise-free audio data to train the RNN model; Model migration module: migrate the pre-trained RNN model to the target domain data, and use the parameters and features of the pre-trained model to process the noise in the target domain data; adjust the RNN model according to the noise type and characteristics of the target domain data to optimize its performance in the target domain; Intelligent analysis and processing module: Use the migrated RNN model to intelligently analyze and process the noise in the target domain data; the RNN model automatically identifies the type and characteristics of the noise, and adjusts the noise reduction parameters and strategies based on the identification results; Real-time processing and feedback module: The RNN model analyzes the audio signal captured by the smart glasses microphone in real time and dynamically adjusts the noise reduction strategy to eliminate background noise; based on user feedback, the noise reduction parameters and strategies of the RNN model are optimized.

[0015] The system further comprises: The noise environment prediction module uses historical noise data and current environmental parameters, such as temperature, humidity, and time, to predict the noise environment change trend in the future through machine learning algorithms; The denoising strategy module is pre-adjusted to adjust the denoising strategy of the RNN model in advance according to the output results of the noise environment prediction module to cope with the upcoming changes in the noise environment and ensure the continuity and stability of the denoising effect.

[0016] The system further comprises: The privacy protection module uses encryption technology and anonymization when collecting and processing user audio signals and sensor data to ensure the security and compliance of user privacy; The data compliance check module regularly performs compliance checks on the collected and processed data.

[0017] Compared with the prior art, the present invention has the following beneficial effects: The smart glasses audio processing method of the present invention collects source domain data and performs model pre-training, so that the RNN model can recognize and process a variety of known noise types and features. In the target domain data, the model migration technology enables the RNN model to use pre-trained parameters and features to quickly adapt to different noise environments, thereby significantly improving the noise reduction effect. The intelligent analysis and processing steps enable the RNN model to automatically identify the type and characteristics of noise, and dynamically adjust the noise reduction parameters and strategies based on the information. This intelligent processing method not only improves the accuracy of noise reduction, but also enables users to obtain the best auditory experience in different noise environments. This method supports real-time processing and feedback mechanism. The RNN model can analyze the audio signal captured by the smart glasses microphone in real time, and dynamically adjust the noise reduction strategy to eliminate background noise. At the same time, based on user feedback, the system can further optimize the noise reduction parameters and strategies to ensure the continuity and stability of the noise reduction effect.

[0018] Since this method uses model migration technology, the RNN model can easily adapt to different noise environments without the need for tedious model training for each new environment. This enables the smart glasses audio processing method to be widely used in various complex and changeable noise environments. By using source domain data for model pre-training and migrating and adjusting the model in the target domain data, the cost and time of model training in the new environment are significantly reduced. This not only improves development efficiency, but also reduces overall development costs.

[0019] This method provides advanced audio processing technology for smart wearable devices such as smart glasses, making these devices more competitive in noise reduction, voice recognition, etc. It helps to promote the rapid development of the smart wearable device market and provide users with a more intelligent and convenient life experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 This is a flow chart of the smart glasses audio processing method of the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0023] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0025] The present invention is further described in detail below in conjunction with the accompanying drawings: See also Figure 1 , is a flow chart of the smart glasses audio processing method of the present invention, which specifically includes the following steps: S1, data preparation, collect source domain data, the data contains known noise types and characteristics, and is collected and annotated in a specific noise environment, such as a laboratory environment or the noise generated by a specific device, and these data sets have been annotated and possibly pre-processed, such as denoising and filtering; collect target domain data, the data is to be processed, comes from different noise environments, such as natural environments or industrial environments, has different noise types and characteristics, and may not be annotated or processed.

[0026] Data cleaning: remove noise from the data using filtering, denoising, etc.; process outliers in the data by using statistical methods to determine the range of outliers and delete or replace them with reasonable values; process missing values ​​in the data by filling, deleting, or using interpolation methods.

[0027] Data integration and transformation steps: Data integration integrates the source domain data and the target domain data into a unified data set, considering the consistency and integrity of the data, and solving the problems of inconsistent data formats and data redundancy; data transformation, including data smoothing, data aggregation, data normalization, data discretization, etc., to discover patterns and regularities in the data.

[0028] Data reduction steps: sampling, randomly selecting a part of samples from the data set as representatives; clustering, dividing the samples in the data set into multiple categories or clusters; dimensionality reduction, reducing the complexity of the data by reducing the dimension of the data while retaining the key information of the data.

[0029] Data segmentation step: Divide the data set into training set, validation set and test set; the training set is used to train the machine learning model, the validation set is used to adjust the model parameters and select the best model configuration, and the test set is used to evaluate the performance of the model.

[0030] Data labeling (for target domain data): manual labeling, adding labels or categories to data manually; automatic labeling, using machine learning algorithms or tools to automatically add labels or categories to data.

[0031] Data validation and storage: Data validation ensures the correctness and effectiveness of the data preparation process through visualization, statistical analysis, model evaluation and other technologies; data storage saves the processed data to a suitable storage medium, taking into account the security, reliability and scalability of the data.

[0032] S2, model pre-training, trains a noise processing model on the source domain data to identify and process the noise in the source domain data.

[0033] S3, audio signal preprocessing, converts the audio signal captured by the smart glasses microphone into sequence data; preprocesses the sequence data, including normalization and segmentation, to meet the input requirements of the subsequent RNN model.

[0034] When converting an audio signal into sequence data, the audio signal is first sampled and the amplitude value of the audio signal is extracted at a set time interval, where the sampling rate is adjusted according to the required accuracy and computational complexity; the continuous amplitude values ​​obtained by sampling are then converted into discrete numerical values, including integer or floating-point representations; finally, the quantized amplitude values ​​are stored as sequence data in chronological order, using an array or list structure for storage.

[0035] The sequence data is then preprocessed. First, the maximum and minimum values ​​of the sequence data are calculated; based on the maximum and minimum values, each amplitude value of the sequence data is scaled to a specified range to eliminate the amplitude differences between different audio signals; the length of each subsequence, that is, the window size, is determined, and the sequence data is divided according to the window size to obtain multiple subsequences for model processing.

[0036] S4, RNN model construction and training, construct the RNN model, the model includes input layer, hidden layer and output layer; the input layer receives the preprocessed audio sequence data; the hidden layer performs data processing to capture the long-term dependencies in the audio signal; the output layer outputs the denoised audio signal; initialize the parameters of the RNN model, and use the noisy audio data and the corresponding noise-free audio data to train the RNN model.

[0037] Model construction: The input layer is set up to receive the preprocessed audio sequence data as input. The data exists in the form of a time series, and each time point corresponds to an amplitude value or spectral feature; convert the audio data into a format that the model can process, and determine the dimensions of the input data, including the time step and the number of features.

[0038] Hidden layer construction: at least one hidden layer is set in the core part of the RNN model to capture the long-term dependencies in the audio signal; RNN units are used in the hidden layer to process sequence data and pass hidden states between time steps to capture the timing information in the audio signal.

[0039] The output layer is set up according to the specific task to output the prediction results. In the denoising task, the output layer outputs the denoised audio signal. In the classification task, the output layer outputs the category label of the audio signal. The output layer uses the fully connected layer and the softmax layer to process the output of the hidden layer and generate the final prediction results.

[0040] Model definition and training: Use the deep learning framework to define the RNN model, including the specific structure and parameters of the input layer, hidden layer, and output layer.

[0041] Use the noisy audio data and the corresponding noise-free audio data as training data sets to optimize the parameters of the model.

[0042] Define the loss function, select the optimizer, and perform iterative training; during the training process, calculate the loss through forward propagation, and update the model parameters through back propagation and the optimizer.

[0043] Parameter initialization, during the model construction process, initialize the parameters of the RNN model.

[0044] Verification and testing: The validation set data is used to verify the performance of the RNN model, including accuracy, stability, and generalization ability.

[0045] Use the test set data to test the performance of the RNN model and evaluate the accuracy and efficiency of the model; in the classification task, use the softmax layer to convert the output of the hidden layer into a probability distribution, and select the category with the highest probability as the prediction result.

[0046] S5, model migration, migrate the pre-trained RNN model to the target domain data, use the parameters and features of the pre-trained model to process the noise in the target domain data; adjust the model according to the noise type and characteristics of the target domain data.

[0047] Select a noise processing model trained on the source domain data, and use the API provided by the deep learning framework to load the weights and architecture of the selected pre-trained model.

[0048] Prepare the output layer of the target task. For audio signal classification tasks, the output layer needs to have the ability to predict the category to which the audio signal belongs, so the dimension of the output layer matches the number of target categories.

[0049] Design the output layer. According to the number of target categories, design a fully connected layer whose number of output nodes is equal to the number of categories; and add a softmax layer after the fully connected layer to convert the output into a probability distribution.

[0050] Connect the pre-trained model with the new output layer and remove the original output layer of the pre-trained model.

[0051] The designed fully connected layer and softmax layer are connected to the end of the pre-trained model to form a new model architecture to adapt to the processing requirements of the target domain data.

[0052] During the migration process, the new model architecture is adjusted according to the noise type and characteristics of the target domain data to optimize the performance of the model when processing the target domain data.

[0053] S6, intelligent analysis and processing, uses the migrated RNN model to intelligently analyze and process the noise in the target domain data; the model automatically identifies the type and characteristics of the noise, and adjusts the noise reduction parameters and strategies based on the information.

[0054] Automatic noise recognition: The migrated model recognizes the noise type and characteristics in the target domain data, extracts the time domain and frequency domain features of the audio signal, and compares them with the model's built-in noise feature library to achieve accurate noise recognition.

[0055] The noise reduction strategy and parameters are adjusted dynamically. According to the identified noise type and characteristics, the model selects and applies low-pass filters, high-pass filters or band-pass filters for noise reduction processing to adapt to different types of noise.

[0056] The model applies spectral subtraction to reduce noise by subtracting the estimated noise spectrum from the spectrum of noisy speech to obtain a purer speech signal.

[0057] Preprocessing: Obtain a noisy speech signal containing a clean speech signal and a noise signal; frame processing, cut the continuous audio signal into multiple frames; windowing, perform windowing processing on each frame signal to improve the accuracy of spectrum analysis.

[0058] Noise estimation: Silence segment detection, detecting silence segments in speech signals; noise spectrum calculation, calculating the spectrum of each frame signal within the detected silence segment, and taking the average value as the estimated value of the noise spectrum.

[0059] Spectral subtraction processing: calculation of the spectrum of noisy speech, fast Fourier transform of the noisy speech signal to obtain its spectral representation; subtraction operation, for each frame of signal, subtract the estimated noise spectrum from the spectrum of the noisy speech; negative value processing, if a negative spectrum is generated after the subtraction operation, at least one of the following methods is used for processing: truncation to zero: set the negative value directly to zero; set a lower limit: set a lower limit value for the spectrum, and when the spectrum value is lower than this lower limit, set it to this lower limit value; spectrum lower limit parameter: use the spectrum lower limit parameter to control the amount of residual noise and the size of the music noise.

[0060] Post-processing: Inverse Fourier transform: convert the processed spectrum back to time domain signal through inverse Fourier transform; overlap and add, since overlap is used in frame processing, the processed frame signals are overlapped and added to restore the original continuous signal.

[0061] Real-time processing and user feedback optimization: During the speech recognition process, the model analyzes the input audio signal in real time and dynamically adjusts the noise reduction strategy to eliminate background noise in real time and improve the accuracy and clarity of speech recognition. At the same time, the model optimizes the noise reduction parameters and strategies based on user feedback to form a closed-loop optimization mechanism to continuously improve the noise reduction performance.

[0062] S7, real-time processing and feedback, the RNN model analyzes the audio signal captured by the smart glasses microphone in real time, and dynamically adjusts the noise reduction strategy to eliminate background noise; based on user feedback, the noise reduction parameters and strategies are optimized.

[0063] Using historical noise data and current environmental parameters, including temperature, humidity, and time, a machine learning algorithm is used to predict the noise environment change trend over a period of time in the future. According to the noise environment prediction results, the noise reduction strategy of the RNN model is adjusted in advance to cope with the upcoming changes in the noise environment and ensure the continuity and stability of the noise reduction effect.

[0064] An embodiment of the present invention is a smart glasses audio processing system, which specifically includes the following modules: Data collection module: collects source domain data, which contains known noise types and characteristics and is collected and labeled in a specific noise environment; collects target domain data, which comes from different noise environments, has different noise types and characteristics, and is to be processed.

[0065] Model pre-training module: Train a noise processing model on the source domain data. The model is used to identify and process the noise in the source domain data.

[0066] Audio signal preprocessing module: captures audio signals through the microphone of smart glasses and converts them into sequence data; preprocesses the sequence data, including normalization and segmentation, to meet the input requirements of the subsequent RNN model.

[0067] RNN model construction and training module: construct an RNN model, which includes an input layer, a hidden layer and an output layer; the input layer receives the preprocessed audio sequence data; the hidden layer is used for data processing to capture the long-term dependencies in the audio signal; the output layer outputs the denoised audio signal; initialize the parameters of the RNN model, and train the RNN model using noisy audio data and the corresponding noise-free audio data.

[0068] Model migration module: migrate the pre-trained RNN model to the target domain data, and use the parameters and features of the pre-trained model to process the noise in the target domain data; adjust the RNN model according to the noise type and characteristics of the target domain data to optimize its performance in the target domain.

[0069] Intelligent analysis and processing module: Use the migrated RNN model to intelligently analyze and process the noise in the target domain data; the RNN model automatically identifies the type and characteristics of the noise, and adjusts the noise reduction parameters and strategies based on the identification results.

[0070] Real-time processing and feedback module: The RNN model analyzes the audio signal captured by the smart glasses microphone in real time and dynamically adjusts the noise reduction strategy to eliminate background noise; based on user feedback, the noise reduction parameters and strategies of the RNN model are optimized.

[0071] The noise environment prediction module uses historical noise data and current environmental parameters, including temperature, humidity and time, to predict the noise environment change trend in the future through machine learning algorithms.

[0072] The denoising strategy module is pre-adjusted to adjust the denoising strategy of the RNN model in advance according to the output results of the noise environment prediction module to cope with the upcoming changes in the noise environment and ensure the continuity and stability of the denoising effect.

[0073] The privacy protection module uses encryption technology and anonymization when collecting and processing user audio signals and sensor data to ensure the security and compliance of user privacy.

[0074] The data compliance check module regularly performs compliance checks on the collected and processed data.

[0075] The smart glasses audio processing system in the present invention can learn a variety of known noise types and characteristics through the source domain data collected by the data collection module, providing a solid foundation for subsequent noise processing. The model pre-training module uses these data for training, so that the RNN model has higher efficiency and accuracy when processing similar noise. In addition, the model migration module can further adjust the RNN model according to the characteristics of the target domain data, so that it can perform well in different noise environments.

[0076] The audio signal preprocessing module performs preprocessing operations such as normalization and segmentation on the captured audio signal to ensure that the input data meets the input requirements of the RNN model. The RNN model construction and training module captures the long-term dependencies in the audio signal and outputs the denoised audio signal, thereby significantly improving the quality of the audio signal.

[0077] The system can process audio data from different noise environments and has a wide range of applicability. Whether indoors, outdoors or in other complex environments, the system can automatically identify noise types and characteristics through intelligent analysis and processing modules, and adjust noise reduction parameters and strategies to meet the needs of different scenarios.

[0078] The real-time processing and feedback module enables the RNN model to analyze the audio signals captured by the smart glasses microphone in real time and dynamically adjust the noise reduction strategy to eliminate background noise. This improves the real-time performance of audio processing and further enhances the user experience by optimizing noise reduction parameters and strategies based on user feedback.

[0079] The system realizes intelligent audio processing by integrating advanced RNN models and transfer learning technology. With the continuous advancement of technology and the continuous accumulation of data, the performance of the system will continue to improve, opening up new possibilities for the application of wearable devices such as smart glasses in the field of audio processing.

[0080] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A smart glasses audio processing method, characterized in that: The following steps are involved: Data preparation: collecting source domain data. The data contains known noise types and characteristics and is collected and annotated in a specific noise environment. Collect target domain data. The data to be processed comes from different noise environments and has different noise types and characteristics. Model pre-training: training a noise processing model on source domain data to identify and process noise in source domain data; Audio signal preprocessing: converting the audio signal captured by the smart glasses microphone into sequence data; preprocessing the sequence data, including normalization and segmentation, to meet the input requirements of the subsequent RNN model; RNN model construction and training, construct the RNN model, the model includes input layer, hidden layer and output layer; the input layer receives the preprocessed audio sequence data; the hidden layer processes the data to capture the long-term dependencies in the audio signal; the output layer outputs the denoised audio signal; initialize the parameters of the RNN model, and use the noisy audio data and the corresponding noise-free audio data to train the RNN model; Model migration: migrate the pre-trained RNN model to the target domain data, use the parameters and features of the pre-trained model to process the noise in the target domain data; adjust the model according to the noise type and characteristics of the target domain data; Intelligent analysis and processing: Use the migrated RNN model to intelligently analyze and process the noise in the target domain data; The model automatically identifies the type and characteristics of noise and adjusts noise reduction parameters and strategies based on the information; Real-time processing and feedback: The RNN model analyzes the audio signal captured by the smart glasses microphone in real time and dynamically adjusts the noise reduction strategy to eliminate background noise; Optimize noise reduction parameters and strategies based on user feedback.

2. The smart glasses audio processing method according to claim 1, characterized in that: The audio signal preprocessing specifically includes: When converting an audio signal into sequence data, the audio signal is first sampled, and the amplitude value of the audio signal is extracted at a set time interval, wherein the sampling rate is adjusted according to the required accuracy and the amount of calculation; then the continuous amplitude value obtained by sampling is converted into a discrete value, including integer or floating point representation; finally, the quantized amplitude value is stored as sequence data in chronological order, using an array or list structure for storage; The sequence data is then preprocessed. First, the maximum and minimum values ​​of the sequence data are calculated; based on the maximum and minimum values, each amplitude value of the sequence data is scaled to a specified range to eliminate the amplitude differences between different audio signals; the length of each subsequence, that is, the window size, is determined, and the sequence data is divided according to the window size to obtain multiple subsequences for model processing.

3. The smart glasses audio processing method according to claim 1, characterized in that: The RNN model construction and training specifically include: Model construction: The input layer is set up to receive the preprocessed audio sequence data as input, which exists in the form of a time series, with each time point corresponding to an amplitude value or spectral feature; convert the audio data into a format that the model can process, and determine the dimensions of the input data, including the time step and the number of features; Hidden layer construction: at least one hidden layer is set in the core part of the RNN model to capture the long-term dependencies in the audio signal; RNN units are used in the hidden layer to process sequence data and pass hidden states between time steps to capture the timing information in the audio signal; Output layer setting: Design the output layer according to the specific task to output the prediction results. In the denoising task, the output layer outputs the denoised audio signal. In the classification task, the output layer outputs the category label of the audio signal. The output layer uses the fully connected layer and the softmax layer to process the output of the hidden layer and generate the final prediction results. Model definition and training: Use a deep learning framework to define the RNN model, including the specific structure and parameters of the input layer, hidden layer, and output layer; Use the noisy audio data and the corresponding noise-free audio data as training data sets to optimize the model parameters; Define the loss function, select the optimizer, and perform iterative training; during the training process, calculate the loss through forward propagation, and update the model parameters through back propagation and the optimizer; Parameter initialization: Initialize the parameters of the RNN model during model construction; Verification and testing: Use validation set data to verify the performance of the RNN model, including accuracy, stability, and generalization ability; Use the test set data to test the performance of the RNN model and evaluate the accuracy and efficiency of the model; in the classification task, use the softmax layer to convert the output of the hidden layer into a probability distribution, and select the category with the highest probability as the prediction result.

4. The smart glasses audio processing method according to claim 1, characterized in that: The model migration is specifically as follows: Select a noise processing model trained on the source domain data, and use the API provided by the deep learning framework to load the weights and architecture of the selected pre-trained model; Prepare the output layer of the target task. For audio signal classification tasks, the output layer needs to be able to predict the category to which the audio signal belongs, so the dimension of the output layer matches the number of target categories; Design the output layer. According to the number of target categories, design a fully connected layer with the number of output nodes equal to the number of categories. Add a softmax layer after the fully connected layer to convert the output into a probability distribution. Connect the pre-trained model with the new output layer and remove the original output layer of the pre-trained model; Connect the designed fully connected layer and softmax layer to the end of the pre-trained model to form a new model architecture to meet the processing requirements of the target domain data; During the migration process, the new model architecture is adjusted according to the noise type and characteristics of the target domain data to optimize the performance of the model when processing the target domain data.

5. The smart glasses audio processing method according to claim 1, characterized in that: The intelligent analysis and processing are specifically: Automatic noise recognition: The migrated model recognizes the noise type and features in the target domain data, extracts the time domain and frequency domain features of the audio signal, and compares them with the noise feature library built into the model to accurately identify the noise. Dynamic adjustment of noise reduction strategies and parameters: Based on the identified noise type and characteristics, the model selects and applies low-pass filters, high-pass filters or band-pass filters for noise reduction to adapt to different types of noise; The model uses spectral subtraction to reduce noise, by subtracting the estimated noise spectrum from the spectrum of noisy speech to obtain a purer speech signal; Real-time processing and user feedback optimization: During the speech recognition process, the model analyzes the input audio signal in real time and dynamically adjusts the noise reduction strategy to eliminate background noise in real time and improve the accuracy and clarity of speech recognition. At the same time, the model optimizes the noise reduction parameters and strategies based on user feedback to form a closed-loop optimization mechanism to continuously improve the noise reduction performance.

6. The smart glasses audio processing method according to claim 5, characterized in that: The spectral subtraction method is specifically: Preprocessing: obtaining a noisy speech signal containing a clean speech signal and a noise signal; framing processing, cutting a continuous audio signal into multiple frames; windowing, performing windowing processing on each frame signal to improve the accuracy of spectrum analysis; Noise estimation: Silence segment detection, detecting silence segments in speech signals; Noise spectrum calculation, calculating the spectrum of each frame signal within the detected silence segment and taking the average value as the estimated value of the noise spectrum; Spectral subtraction processing: calculation of the spectrum of noisy speech, fast Fourier transform of the noisy speech signal to obtain its spectrum representation; subtraction operation, for each frame of signal, subtract the estimated noise spectrum from the spectrum of the noisy speech; negative value processing, if a negative spectrum is generated after subtraction, at least one of the following methods is used for processing: truncation to zero: set the negative value directly to zero; set lower limit: set a lower limit value for the spectrum, and when the spectrum value is lower than this lower limit, set it to this lower limit value; spectrum lower limit parameter: use the spectrum lower limit parameter to control the amount of residual noise and the size of the music noise; Post-processing: Inverse Fourier transform: convert the processed spectrum back to time domain signal through inverse Fourier transform; overlap and add, since overlap is used in frame processing, the processed frame signals are overlapped and added to restore the original continuous signal.

7. The smart glasses audio processing method according to claim 1, characterized in that: The following steps are also included: Using historical noise data and current environmental parameters, including temperature, humidity, and time, a machine learning algorithm is used to predict the noise environment change trend over a period of time in the future. According to the noise environment prediction results, the noise reduction strategy of the RNN model is adjusted in advance to cope with the upcoming changes in the noise environment and ensure the continuity and stability of the noise reduction effect.

8. A smart glasses audio processing system, characterized in that: Includes the following modules: Data collection module: collects source domain data, which contains known noise types and characteristics, and is collected and annotated in a specific noise environment; Collect target domain data, which comes from different noise environments, has different noise types and characteristics, and needs to be processed; Model pre-training module: trains a noise processing model on source domain data. The model is used to identify and process noise in source domain data. Audio signal preprocessing module: captures audio signals through the microphone of smart glasses and converts them into sequence data; preprocesses the sequence data, including normalization and segmentation, to meet the input requirements of the subsequent RNN model; RNN model construction and training module: construct an RNN model, which includes an input layer, a hidden layer, and an output layer; the input layer receives the preprocessed audio sequence data; the hidden layer is used for data processing to capture the long-term dependencies in the audio signal; the output layer outputs the denoised audio signal; initialize the parameters of the RNN model, and use the noisy audio data and the corresponding noise-free audio data to train the RNN model; Model migration module: migrate the pre-trained RNN model to the target domain data, and use the parameters and features of the pre-trained model to process the noise in the target domain data; adjust the RNN model according to the noise type and characteristics of the target domain data to optimize its performance in the target domain; Intelligent analysis and processing module: Use the migrated RNN model to intelligently analyze and process the noise in the target domain data; the RNN model automatically identifies the type and characteristics of the noise, and adjusts the noise reduction parameters and strategies based on the identification results; Real-time processing and feedback module: The RNN model analyzes the audio signal captured by the smart glasses microphone in real time and dynamically adjusts the noise reduction strategy to eliminate background noise; Based on user feedback, optimize the denoising parameters and strategies of the RNN model.

9. The smart glasses audio processing system according to claim 8, characterized in that: The system further comprises: The noise environment prediction module uses historical noise data and current environmental parameters, such as temperature, humidity, and time, to predict the noise environment change trend in the future through machine learning algorithms; The denoising strategy module is pre-adjusted to adjust the denoising strategy of the RNN model in advance according to the output results of the noise environment prediction module to cope with the upcoming changes in the noise environment and ensure the continuity and stability of the denoising effect.

10. The smart glasses audio processing system according to claim 8, characterized in that: The system further comprises: The privacy protection module uses encryption technology and anonymization when collecting and processing user audio signals and sensor data to ensure the security and compliance of user privacy; The data compliance check module regularly performs compliance checks on the collected and processed data.

Images