Intercom audio processing method, system and device based on edge AI, and medium

Abstract

This application relates to the field of speech processing technology, and in particular to a method, system, device, and medium for audio processing of walkie-talkies based on edge AI. The method includes: acquiring and parsing raw audio through an audio acquisition component to obtain an audio feature set containing noise and scene features; inputting this feature set into an edge-side audio analysis model to obtain noise classification and scene recognition results; dynamically adjusting filter parameters accordingly to generate a filter parameter set adapted to the current environment; extracting user voiceprint features to construct a dedicated voiceprint library, and optimizing the audio using the filter parameter set to generate personalized processed audio; training a model locally based on the personalized processed audio and the audio feature set, generating updated parameters, and updating the audio analysis model for subsequent analysis. This application solves the technical problem that existing technologies cannot simultaneously achieve a balance between dynamic adaptation, personalized optimization, and low power consumption and high performance.

Description

Technical Field

[0001] This application relates to the field of speech processing technology, and in particular to a walkie-talkie audio processing method, system, device and medium based on edge AI. Background Technology

[0002] With the rapid development of industrial production and mobile operations, the demand for audio communication in complex noise environments using walkie-talkies is becoming increasingly urgent. To address this, existing technologies generally employ fixed-parameter analog filtering (such as LC resonant circuits), DSP-based digital filtering (such as FIR / IIR filters), or rely on cloud-based AI for audio processing. However, fixed filtering parameters cannot adapt to dynamic noise and scene changes, DSP solutions lack intelligent feature extraction capabilities resulting in limited signal-to-noise ratio improvement, and cloud-based AI suffers from high latency. Furthermore, existing solutions do not consider user voiceprint differences and face the contradiction between high-performance processing and low-power consumption requirements. Therefore, the technical problem of simultaneously achieving a balance between dynamic adaptation, personalized optimization, and low-power high performance urgently needs to be solved.

[0003] Therefore, the technical problem that existing technologies cannot simultaneously achieve a balance between dynamic adaptation, personalized optimization, and low power consumption and high performance urgently needs to be solved. Summary of the Invention

[0004] The main purpose of this application is to provide a walkie-talkie audio processing method, system, device and medium based on edge AI, which aims to solve the technical problem that the existing technology cannot simultaneously achieve a balance between dynamic adaptation, personalized optimization and low power consumption and high performance.

[0005] To achieve the aforementioned objectives, this application proposes an adaptive walkie-talkie audio processing method based on edge AI, the method comprising:

[0006] The raw audio signal is acquired by the audio acquisition component and analyzed in real time to obtain an audio feature set containing noise features and scene features;

[0007] The audio feature set is input into an audio analysis model deployed on the edge side, and noise classification results and scene recognition results are output.

[0008] The operating parameters of the audio processing filter are dynamically adjusted based on the noise classification results and scene recognition results to generate a set of filter parameters adapted to the current environment.

[0009] The user's voice features are extracted by an embedded voiceprint learning unit to build a user-specific voiceprint library. The filtered audio signal is then optimized in a personalized manner based on the filtering parameter group to generate personalized processed audio.

[0010] Based on the personalized audio processing and audio feature set, the model is trained locally and model update parameters are generated.

[0011] The audio analysis model is updated based on the model update parameters for subsequent analysis of the collected audio signals.

[0012] Furthermore, the step of acquiring the original audio signal through the audio acquisition component and analyzing it in real time to obtain an audio feature set containing noise features and scene features includes:

[0013] Raw audio signals were acquired using a multi-microphone array.

[0014] Real-time spectrum analysis is performed on the original audio signal to obtain the audio signal in the frequency domain.

[0015] Based on the audio signal represented in the frequency domain, noise features and scene features are extracted by the audio analysis model to generate an audio feature set.

[0016] Furthermore, the audio analysis model includes a ResNet8 model comprising an improved VDCNN network and TinyML compression. The step of inputting the audio feature set into the edge-deployed audio analysis model and outputting noise classification results and scene recognition results includes:

[0017] Input the audio feature set into the audio analysis model deployed on the edge side;

[0018] The improved VDCNN network is used to classify noise features, wherein the VDCNN network contains a 12-class industrial noise sample library, and the current environmental noise type is identified through matching and classification algorithms, and the noise classification result is output.

[0019] A ResNet8 model using TinyML compression is used to extract environmental features from the audio feature set, including background noise power spectral density and reverberation time, to identify the current acoustic scene and output the scene identification results.

[0020] Furthermore, the step of dynamically adjusting the operating parameters of the audio processing filter based on the noise classification result and scene recognition result to generate a filter parameter set adapted to the current environment includes:

[0021] Select the corresponding filtering strategy based on the noise classification results;

[0022] Optimize different parameters of the filter based on scene recognition results;

[0023] Real-time parameter adjustment of adaptive filter banks is achieved through a TinyML-optimized LSTM network;

[0024] A residual signal feedback mechanism is introduced to automatically adjust the filter parameters based on the difference between the original signal and the filtered signal.

[0025] Based on the above adjustments, a set of filter parameters adapted to the current environment is generated.

[0026] Furthermore, the steps of extracting voiceprint features from user speech samples through an embedded voiceprint learning unit, constructing a user-specific voiceprint library, and performing personalized optimization on the filtered audio signal based on the filtering parameter set to generate personalized processed audio include:

[0027] Collect user voice samples using the device's microphone;

[0028] The embedded voiceprint learning unit is used to extract voiceprint features and store them in the device to build a user-specific voiceprint library;

[0029] The original audio signal is initially filtered based on the filter parameter set generated by the noise classification results and scene recognition results;

[0030] The audio signal after initial filtering is optimized based on the user's voiceprint database to generate personalized processed audio.

[0031] Furthermore, the step of completing model training and generating model update parameters locally based on the personalized processed audio and audio feature set includes:

[0032] On edge devices, local model training is performed based on personalized processed audio and audio feature sets to generate model update parameters;

[0033] The model update parameters are encrypted using an encryption algorithm and are prepared to be sent to the central server or other associated devices via a distributed learning architecture with encrypted transmission.

[0034] Furthermore, the step of updating the audio analysis model based on the model update parameters for subsequent analysis of the acquired audio signals includes:

[0035] The model update parameters are encrypted using an encryption algorithm and sent to the central server or associated device through a distributed learning architecture with encrypted transmission.

[0036] Homomorphic encryption technology is used to securely aggregate the updated parameters of the encrypted model to generate an updated audio analysis model;

[0037] The updated model is distributed to each edge device to replace the original model.

[0038] The second aspect of this application proposes an adaptive walkie-talkie audio processing system based on edge AI:

[0039] The audio acquisition module acquires raw audio signals through audio acquisition components and analyzes them in real time to obtain an audio feature set containing noise features and scene features;

[0040] The edge computing module inputs the audio feature set into the audio analysis model deployed on the edge side and outputs noise classification results and scene recognition results;

[0041] The adaptive filtering module dynamically adjusts the operating parameters of the audio processing filter based on the noise classification results and scene recognition results, generating a set of filtering parameters that are adapted to the current environment.

[0042] The personalized optimization module extracts the voiceprint features of user speech samples through an embedded voiceprint learning unit, constructs a user-exclusive voiceprint library, and performs personalized optimization on the filtered audio signal based on the filtering parameter group to generate personalized processed audio.

[0043] The local model training module, based on the personalized processed audio and audio feature set, completes model training locally and generates model update parameters.

[0044] The model update module updates the audio analysis model based on the model update parameters for subsequent analysis of the collected audio signals.

[0045] A third aspect of this application also provides an apparatus comprising a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of any of the methods described above.

[0046] A fourth aspect of this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the methods described above.

[0047] Beneficial effects

[0048] The achievable technical effects are as follows: Audio feature sets are obtained in real-time through audio acquisition components; noise and scene recognition results are output through the edge-side audio analysis model; filtering parameters are adaptively and dynamically adjusted; and audio is optimized by combining a user-specific voiceprint library, achieving adaptive audio processing for the current environment. Simultaneously, through local model training and updates, the analysis capabilities are continuously improved, effectively addressing the poor audio processing effects caused by fixed filtering and lack of personalized adaptation in existing technologies, thus enhancing the audio communication quality of walkie-talkies in complex environments. Attached Figure Description

[0049] Figure 1 This is a schematic flowchart of an edge AI-based walkie-talkie audio processing method according to an embodiment of this application.

[0050] Figure 2 This is a schematic block diagram of a walkie-talkie audio processing system based on edge AI, according to an embodiment of this application.

[0051] Figure 3 This is a schematic block diagram of the structure of a computer device according to an embodiment of this application;

[0052] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0054] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of features, integers, steps, operations, elements, modules, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, modules, components, and / or groups thereof. It should be understood that when an element is referred to as “connected” or “coupled” to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein may include wireless connection or wireless coupling. The term “and / or” as used herein includes all or any modules and all combinations of one or more associated listed items.

[0055] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0056] Reference Figure 1 This invention provides a walkie-talkie audio processing method based on edge AI, including steps S1-S6, specifically:

[0057] S1. Acquire the raw audio signal through the audio acquisition component and analyze it in real time to obtain an audio feature set containing noise features and scene features;

[0058] S2. Input the audio feature set into the audio analysis model deployed on the edge side, and output noise classification results and scene recognition results;

[0059] S3. Dynamically adjust the working parameters of the audio processing filter based on the noise classification results and scene recognition results to generate a filter parameter group that adapts to the current environment;

[0060] S4. Extract the voiceprint features of user voice samples through the embedded voiceprint learning unit, construct a user-specific voiceprint library, and perform personalized optimization on the filtered audio signal based on the filtering parameter group to generate personalized processed audio.

[0061] S5. Based on the personalized processed audio and audio feature set, complete the model training locally and generate model update parameters;

[0062] S6. Update the audio analysis model based on the model update parameters for subsequent analysis of the collected audio signals.

[0063] Step S1 uses a multi-microphone array consisting of 4-6 MEMS microphones as the audio acquisition component to collect raw audio signals. Utilizing the spatial sound field capture capability of the multi-microphone array, it can acquire more comprehensive environmental acoustic information compared to a single microphone. Subsequently, real-time spectrum analysis is performed on the raw audio signal, using the FFT algorithm to convert the time-domain signal into a frequency-domain representation with a resolution of up to 5Hz for fine analysis of different frequency components. Based on the frequency-domain signal, noise features (such as specific frequency peaks of mechanical noise in industrial environments and the energy distribution of sudden noise) and scene features (such as background noise power spectral density and reverberation time) are extracted and integrated to form an audio feature set containing both types of features. The advantage of this step is that the multi-microphone array improves the spatial accuracy of sound field acquisition, and the real-time high-resolution spectrum analysis ensures the timeliness and accuracy of feature extraction, providing reliable input for subsequent noise classification and scene recognition in the edge-side model. For example, in a factory workshop setting, a multi-microphone array can simultaneously collect the 750Hz periodic noise from the lathe and the worker's voice. After FFT conversion, the 750Hz noise peak (noise feature) and the 0.8-second reverberation time (scene feature) can be extracted. The resulting audio feature set can accurately reflect the acoustic characteristics of the current environment.

[0064] Step S2 inputs the audio feature set generated in S1 into the audio analysis model deployed on the Rockchip RK1808SoC (equipped with a 1.2 TOPS NPU). The audio analysis model is a lightweight intelligent analysis model (with less than 100KB of parameters). The lightweight intelligent analysis model consists of the following two core branches, as detailed below:

[0065] An improved VDCNN network branch: This branch is based on a sample library containing 12 types of industrial noise and is used to classify noise features. By comparing feature vectors to match noise types, it can accurately identify the type of noise in the current environment, such as quickly identifying periodic impact noise on an oil drilling platform.

[0066] TinyML compressed ResNet8 model branch: This model is compressed to 48KB using TinyML technology. It is used to extract environmental features from the audio feature set, such as background noise power spectral density and reverberation time (RT60). Then, it is combined with the LSTM network to classify 6 preset scene modes such as industrial, outdoor, and indoor, providing scene basis for subsequent filtering adjustments.

[0067] Meanwhile, this lightweight audio analysis model features a parameter count of <100KB and employs INT8 asymmetric quantization (APoT algorithm) with an accuracy loss of <2%. It can achieve a 20ms response time on Rockchip RK1808SoC (equipped with a 1.2TOPS NPU), adapting to the low computing power requirements of edge devices and avoiding latency issues caused by cloud processing.

[0068] Step S3, based on the noise classification and scene recognition results output from S2, dynamically adjusts the operating parameters of the 32nd-order FIR+IIR hybrid filter bank using a TinyML-optimized LSTM network. This prioritizes suppression of corresponding frequency bands for different noise levels and optimizes filter parameters according to the scene. Simultaneously, a Kalman filter is used to smooth parameter switching, avoiding speech distortion caused by abrupt changes. It supports coefficient updates at 15 times / second and introduces residual signal feedback to correct filtering biases. In data processing, the filter parameter set includes the 32nd-order coefficients of the FIR filter, the cutoff frequency of the IIR filter, and its gain value. The advantage is that it overcomes the limitations of traditional fixed parameters, achieving dual adaptation to noise and scene. For example, in a factory workshop, when sudden impact noise occurs, the filter can quickly adjust the high-frequency attenuation coefficient while retaining the 300-3400Hz speech frequency band.

[0069] Step S4 collects 30 seconds of user speech samples using an embedded voiceprint learning unit (supporting 1000 users / device capacity), extracts voiceprint features from the 2.4-3.6kHz frequency band to construct a dedicated voiceprint library, and stores it using AES-256 encryption. After initial filtering based on the filtering parameter set generated in S3, the filtered audio is personalized and optimized using the voiceprint library. For example, the gain in the 2.4-3.6kHz frequency band is enhanced for female users to prevent high-frequency speech from being mistakenly filtered out. In terms of data processing, the optimization coefficients are dynamically adjusted through PESQ scoring to ensure improved speech intelligibility after personalized processing. The advantage is that it solves the problem of lack of personalization in general filtering. For example, when a female user uses the system, after recognizing her voiceprint features, the suppression intensity of the relevant frequency bands is adjusted to make the high-frequency components of the command clearer.

[0070] Step S5, based on the personalized audio processing in S4 and the audio feature set in S1, trains the model locally using a federated learning architecture: the audio analysis model is incrementally trained using a gradient descent algorithm on a 24-hour cycle. The generated model update parameters are then compressed by 80% gradient, retaining only the key weight adjustment values. During data processing, the training process consumes ≤30% of CPU resources to avoid impacting real-time audio processing. The advantage is that it enables localized model evolution without uploading raw voice data, protecting privacy and reducing power consumption. For example, oilfield inspection equipment can train its model daily using idle time at night based on the data collected that day.

[0071] Step S6 transmits the model update parameters generated in S5 using AES-256 encryption. After secure aggregation using Paillier homomorphic encryption, the network weights and classification thresholds of the audio analysis model are updated. The update process does not interrupt audio processing, with a switching latency of <50ms. In terms of data processing, the updated model parameters are immediately applied to the next audio analysis, forming a closed loop of "collection-analysis-optimization-training-update". The advantage is that it ensures the model continuously adapts to changes in the environment and users. For example, through iteration, the device's response speed to specific noise improves, and the speech signal-to-noise ratio is further optimized.

[0072] In one embodiment, the step of acquiring the raw audio signal through the audio acquisition component and analyzing it in real time to obtain an audio feature set containing noise features and scene features includes:

[0073] s20. Use a multi-microphone array to acquire raw audio signals;

[0074] s21. Perform real-time spectrum analysis on the original audio signal to obtain the audio signal in frequency domain representation;

[0075] s22. Based on the audio signal represented in the frequency domain, noise features and scene features are extracted by the audio analysis model to generate an audio feature set.

[0076] In this embodiment, a circular array of 4-6 MEMS microphones is used as the audio acquisition component. Utilizing the spatial distribution characteristics of multiple microphones, it achieves omnidirectional acquisition of the spatial sound field. Compared to a single microphone, it can more accurately capture sound sources and noise information from different directions, making it particularly suitable for industrial environments where multi-directional noise coexists. The raw audio signal is processed by a real-time spectrum analysis module, using an FFT algorithm to convert the time-domain signal into a frequency-domain representation. The FFT resolution reaches 5Hz, allowing for precise analysis of frequency components in the 0-8kHz band, providing high-resolution data support for subsequent feature extraction. The audio analysis model (with <100KB of parameters) extracts noise features (such as the frequency peaks of specific industrial noises and the energy distribution of sudden noises) and scene features (such as background noise power spectral density and reverberation time RT60) based on the frequency-domain signal, integrating them into an audio feature set. This lays the foundation for subsequent noise classification and scene recognition. The advantages of this process are that multi-microphone acquisition improves the spatial accuracy of sound field capture, high-resolution spectrum analysis ensures the accuracy of features, and the lightweight model adapts to the computing power requirements of edge devices.

[0077] In one embodiment, the audio analysis model includes a ResNet8 model comprising an improved VDCNN network and TinyML compression. The step of inputting the audio feature set into the edge-deployed audio analysis model and outputting noise classification results and scene recognition results includes:

[0078] s30. Input the audio feature set into the audio analysis model deployed on the edge side;

[0079] s31. Use an improved VDCNN network to classify noise features, wherein the VDCNN network contains a 12-class industrial noise sample library, identifies the current environmental noise type through matching and classification algorithms, and outputs the noise classification result;

[0080] s32. Using a TinyML-compressed ResNet8 model, environmental features, including background noise power spectral density and reverberation time, are extracted from the audio feature set to identify the current acoustic scene and output the scene identification results.

[0081] In this embodiment, the audio analysis model is deployed on a SoC with an integrated AI acceleration core (such as Rockchip RK1808), relying on NPU computing power to achieve efficient inference and avoid cloud transmission latency. The improved VDCNN network is based on a sample library containing 12 types of industrial noise (such as mechanical roar and explosion sounds). Through feature vector matching and classification algorithms, it accurately identifies the current noise type. For example, periodic impact noise can be identified on an oil drilling platform. A ResNet8 model (48KB in size) compressed using TinyML technology extracts environmental features such as background noise power spectral density (reflecting the overall energy distribution of noise) and reverberation time RT60 (reflecting the acoustic characteristics of the environment) from the audio feature set, thereby identifying acoustic scenes such as industrial, outdoor, and indoor environments, and providing six preset industrial environment modes for switching. The advantage of this process is that edge-side localized processing achieves a 20ms-level response speed, and the two types of models working together can simultaneously output noise and scene information, providing dual basis for subsequent filtering adjustments.

[0082] In one embodiment, the step of dynamically adjusting the operating parameters of the audio processing filter based on the noise classification result and scene recognition result to generate a filter parameter set adapted to the current environment includes:

[0083] s40. Select the corresponding filtering strategy based on the noise classification results;

[0084] s41. Optimize different parameters of the filter based on scene recognition results;

[0085] s42. Real-time parameter adjustment of adaptive filter banks is achieved through a TinyML-optimized LSTM network;

[0086] s43. Introduce a residual signal feedback mechanism to automatically adjust filter parameters based on the difference between the original signal and the filtered signal;

[0087] s44. Based on the above adjustments, generate a set of filter parameters adapted to the current environment.

[0088] In this embodiment, a targeted filtering strategy is selected based on the noise classification results (e.g., mechanical noise, sudden impact noise). For example, mechanical noise is suppressed in specific low-frequency bands, and sudden noise is attenuated more effectively. Parameters are optimized based on scene recognition results (e.g., industrial scenes, indoor scenes). For example, the dereverberation coefficient is enhanced in reverberant scenes, and the high-frequency filtering intensity is increased in outdoor strong wind scenes. A TinyML-optimized LSTM network is used to update the parameters of a 32nd-order FIR+IIR hybrid filter bank, supporting an adjustment frequency of 15 times / second to ensure real-time adaptation to environmental changes. A residual signal feedback mechanism is introduced to calculate the difference between the original signal and the filtered signal, and to correct the filter parameters in reverse, reducing filtering bias, such as avoiding excessive suppression of key audio segments. The above adjustments generate a filter parameter set, including key parameters such as FIR coefficients and IIR cutoff frequency. The advantage of this process is that it breaks through the limitations of traditional fixed parameters, enabling the filter to adapt to both noise type and scene characteristics through multi-dimensional dynamic adjustment.

[0089] In one embodiment, the steps of extracting voiceprint features from user speech samples using an embedded voiceprint learning unit, constructing a user-specific voiceprint library, and performing personalized optimization on the filtered audio signal based on the filtering parameter set to generate personalized processed audio include:

[0090] s50. Collect user voice samples through the device's microphone;

[0091] s51. Use an embedded voiceprint learning unit to extract voiceprint features and store them in the device to build a user-specific voiceprint library;

[0092] s52. The filtering parameter set generated based on the noise classification results and scene recognition results performs preliminary filtering on the original audio signal;

[0093] s53. Based on the user-specific voiceprint library, perform personalized optimization on the initially filtered audio signal to generate personalized processed audio.

[0094] In this embodiment, a 30-second voice sample from the user is collected via the device's microphone, covering voice features across different frequency bands to ensure comprehensive voiceprint extraction. An embedded voiceprint learning unit extracts voiceprint features from the sample (e.g., high-frequency features of female users in the 2.4-3.6kHz range) and stores them in a device-built voiceprint library, supporting a capacity of 1000 users / devices. The original audio signal is initially filtered based on a set of filtering parameters to remove most environmental noise, such as suppressing mechanical noise in industrial settings. The pre-filtered signal is then individually optimized based on the voiceprint library; for example, the gain in the 2.4-3.6kHz frequency band is increased for female users to prevent their high-frequency speech from being mistakenly filtered out, improving the current 65% satisfaction rate among female users. The advantage of this process is that it achieves personalized adaptation through voiceprint learning, solving the problem of voice distortion for specific users caused by general filtering and improving speech intelligibility.

[0095] In one embodiment, the step of completing model training locally and generating model update parameters based on the personalized processed audio and audio feature set includes:

[0096] s60. Incremental training is performed on edge devices using personalized processed audio and corresponding audio feature sets;

[0097] s61. Calculate the gradient changes of model parameters using the gradient descent algorithm to generate model update parameters for collaborative updating.

[0098] In this embodiment, incremental training is performed on edge devices (such as walkie-talkies equipped with RK1808 chips) on a 24-hour cycle, using personalized processed audio collected daily and corresponding audio feature sets to avoid the high computational cost of full training. Gradient changes in model parameters (such as VDCNN network weights and ResNet8 feature extraction coefficients) are calculated using the gradient descent algorithm to generate local model update parameters, reflecting the model's optimization direction on new data. The update parameters are limited to a transferable key subset (such as classifier thresholds and feature extraction kernels), prioritizing core parameters that significantly impact noise classification and scene recognition accuracy, and reducing data transmission volume, for example, compressing it to 20% of the original parameter volume. Model update parameters for collaborative updates are generated based on this key parameter subset, preparing for subsequent inter-device model collaborative optimization. The advantages of this process are that local training avoids uploading raw data, protects privacy, and reduces power consumption; incremental training and parameter simplification allow the model to continuously evolve and adapt to the computational power of edge devices.

[0099] In one embodiment, the step of updating the audio analysis model based on the model update parameters for subsequent analysis of the acquired audio signals includes:

[0100] s70. The model update parameters are encrypted using an encryption algorithm and sent to the central server or associated device through a distributed learning architecture with encrypted transmission.

[0101] s71. Use homomorphic encryption technology to securely aggregate the updated parameters of the encrypted model to generate an updated audio analysis model;

[0102] s72. Distribute the updated model to each edge device to replace the original model and complete the update.

[0103] In this embodiment, a distributed architecture of federated learning is adopted to achieve collaborative sharing of model update parameters among multiple devices. For example, multiple walkie-talkies used in oilfield inspections can collaboratively optimize the model. The model update parameters are encrypted using the AES-256 encryption algorithm and sent to the central server or associated devices through a secure transmission channel, ensuring secure parameter transmission. Paillier homomorphic encryption technology is used to securely aggregate the encrypted parameters uploaded from multiple devices, completing parameter fusion without decryption to generate an updated audio analysis model, preventing parameter leakage. The updated model is distributed to each edge device, replacing the original model. The update process does not interrupt audio processing, with a switching latency of <50ms. The advantages of this process are that encrypted transmission and secure aggregation ensure data privacy, and the distributed architecture allows the model to evolve based on data from multiple devices, continuously improving analysis accuracy and adapting to more complex scenarios.

[0104] Reference Figure 2 This is a block diagram of a walkie-talkie audio processing system based on edge AI in one embodiment of this application. The system includes:

[0105] The audio acquisition module 100 is used to acquire raw audio signals through the audio acquisition component and analyze them in real time to obtain an audio feature set containing noise features and scene features.

[0106] The edge computing module 200 is used to input the audio feature set into an audio analysis model deployed on the edge side and output noise classification results and scene recognition results;

[0107] The adaptive filtering module 300 is used to dynamically adjust the working parameters of the audio processing filter based on the noise classification results and scene recognition results, and generate a set of filtering parameters that are adapted to the current environment.

[0108] The personalized optimization module 400 is used to extract the voiceprint features of user voice samples through an embedded voiceprint learning unit, build a user-exclusive voiceprint library, and perform personalized optimization on the filtered audio signal based on the filtering parameter group to generate personalized processed audio.

[0109] The local model training module 500 is used to complete model training locally and generate model update parameters based on the personalized processed audio and audio feature set.

[0110] The model update module 600 is used to update the audio analysis model based on the model update parameters for subsequent analysis of the collected audio signals.

[0111] In one embodiment, the audio acquisition module 100 includes:

[0112] A signal acquisition unit is used to acquire raw audio signals using a multi-microphone array;

[0113] The spectrum analysis unit is used to perform real-time spectrum analysis on the original audio signal to obtain the audio signal in frequency domain representation;

[0114] The feature extraction unit is used to extract noise features and scene features from the audio signal represented in the frequency domain using an audio analysis model to generate an audio feature set.

[0115] In one embodiment, the edge computing module 200 includes:

[0116] The model input unit is used to input the audio feature set into the audio analysis model deployed at the edge.

[0117] The noise classification unit is used to classify noise features using an improved VDCNN network, wherein the VDCNN network contains a 12-class industrial noise sample library, identifies the current environmental noise type through matching and classification algorithms, and outputs the noise classification result;

[0118] The scene recognition unit is used to extract environmental features from the audio feature set using a TinyML-compressed ResNet8 model, including background noise power spectral density and reverberation time, to identify the current acoustic scene and output the scene recognition results.

[0119] In one embodiment, the adaptive filtering module 300 includes:

[0120] The strategy selection unit is used to select the corresponding filtering strategy based on the noise classification results.

[0121] The parameter optimization unit is used to optimize different parameters of the filter based on the scene recognition results.

[0122] A real-time adjustment unit is used to achieve real-time parameter adjustment of the adaptive filter bank through a TinyML-optimized LSTM network;

[0123] The feedback adjustment unit is used to introduce a residual signal feedback mechanism to automatically adjust the filter parameters based on the difference between the original signal and the filtered signal.

[0124] The parameter set generation unit is used to combine the above adjustments to generate a filter parameter set that is adapted to the current environment.

[0125] In one embodiment, the personalization optimization module 400 includes:

[0126] The sample acquisition unit is used to acquire user voice samples through the device's microphone;

[0127] The voiceprint library construction unit is used to extract voiceprint features using the embedded voiceprint learning unit and store them in the device to build a user-specific voiceprint library.

[0128] The preliminary filtering unit is used to perform preliminary filtering on the original audio signal based on the filtering parameter set generated by noise classification results and scene recognition results;

[0129] The personalized adjustment unit is used to perform personalized optimization on the initially filtered audio signal based on the user-specific voiceprint library, and generate personalized processed audio.

[0130] In one embodiment, the local model training module 500 includes:

[0131] Incremental training unit, used for incremental training on edge devices using personalized processed audio and corresponding audio feature sets;

[0132] The gradient calculation unit is used to calculate the gradient changes of model parameters using the gradient descent algorithm, and generate model update parameters for collaborative updating.

[0133] In one embodiment, the model update module 600 includes:

[0134] The parameter transmission unit is used to encrypt the model update parameters using an encryption algorithm and send them to the central server or associated devices through the encrypted transmission distributed learning architecture.

[0135] The secure aggregation unit is used to securely aggregate the encrypted model update parameters using homomorphic encryption technology to generate an updated audio analysis model.

[0136] The model distribution unit is used to distribute the updated model to each edge device to replace the original model and complete the update.

[0137] Reference Figure 3 This application also provides a computer device, which may be a server, and its internal structure may be as follows: Figure 3As shown, the computer device includes a processor, internal memory, a storage medium (non-volatile storage medium), and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes the aforementioned storage medium (non-volatile storage medium) and the internal memory. The storage medium (non-volatile storage medium) stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the storage medium (non-volatile storage medium). The database stores usage data, such as data from a walkie-talkie audio processing method based on edge AI. The network interface allows communication with external terminals via a network connection. Furthermore, the computer device may also include input devices and a display screen. When the aforementioned computer program is executed by a processor, it implements a walkie-talkie audio processing method based on edge AI, comprising the following steps: acquiring raw audio signals through an audio acquisition component and analyzing them in real time to obtain an audio feature set containing noise features and scene features; inputting the audio feature set into an audio analysis model deployed on the edge side, and outputting noise classification results and scene recognition results; dynamically adjusting the working parameters of the audio processing filter according to the noise classification results and scene recognition results to generate a filter parameter set adapted to the current environment; extracting voiceprint features of user voice samples through an embedded voiceprint learning unit to construct a user-specific voiceprint library, and performing personalized optimization on the filtered audio signal based on the filter parameter set to generate personalized processed audio; completing model training locally and generating model update parameters based on the personalized processed audio and the audio feature set; updating the audio analysis model based on the model update parameters for subsequent analysis of the acquired audio signals. Those skilled in the art will understand that... Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer equipment on which the present application is applied.

[0138] One embodiment of this application also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements an audio processing method for a walkie-talkie based on edge AI, including the following steps: acquiring raw audio signals through an audio acquisition component and analyzing them in real time to obtain an audio feature set containing noise features and scene features; inputting the audio feature set into an audio analysis model deployed on the edge side, and outputting noise classification results and scene recognition results; dynamically adjusting the working parameters of the audio processing filter according to the noise classification results and scene recognition results to generate a filter parameter set adapted to the current environment; extracting voiceprint features of user voice samples through an embedded voiceprint learning unit to construct a user-specific voiceprint library, and performing personalized optimization on the filtered audio signal based on the filter parameter set to generate personalized processed audio; completing model training locally and generating model update parameters based on the personalized processed audio and the audio feature set; updating the audio analysis model based on the model update parameters for subsequent analysis of the acquired audio signals. It is understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0139] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in this application and in the embodiments can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-speed SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0140] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0141] The above description is only a preferred embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural changes made based on the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A walkie-talkie audio processing method based on edge AI, characterized in that, The method includes: The raw audio signal is acquired by the audio acquisition component and analyzed in real time to obtain an audio feature set containing noise features and scene features; The audio feature set is input into an audio analysis model deployed on the edge side, and noise classification results and scene recognition results are output. The operating parameters of the audio processing filter are dynamically adjusted based on the noise classification results and scene recognition results to generate a set of filter parameters adapted to the current environment. The user's voice features are extracted by an embedded voiceprint learning unit to build a user-specific voiceprint library. The filtered audio signal is then optimized in a personalized manner based on the filtering parameter group to generate personalized processed audio. Based on the personalized audio processing and audio feature set, the model is trained locally and model update parameters are generated. The audio analysis model is updated based on the model update parameters for subsequent analysis of the collected audio signals.

2. The walkie-talkie audio processing method based on edge AI according to claim 1, characterized in that, The step of acquiring raw audio signals through an audio acquisition component and analyzing them in real time to obtain an audio feature set containing noise and scene features includes: Raw audio signals were acquired using a multi-microphone array. Real-time spectrum analysis is performed on the original audio signal to obtain the audio signal in the frequency domain. Based on the audio signal represented in the frequency domain, noise features and scene features are extracted by the audio analysis model to generate an audio feature set.

3. The walkie-talkie audio processing method based on edge AI according to claim 1, characterized in that, The audio analysis model includes an improved VDCNN network and a ResNet8 model with TinyML compression. The step of inputting the audio feature set into the edge-deployed audio analysis model and outputting noise classification results and scene recognition results includes: Input the audio feature set into the audio analysis model deployed on the edge side; The improved VDCNN network is used to classify noise features, wherein the VDCNN network contains a 12-class industrial noise sample library, and the current environmental noise type is identified through matching and classification algorithms, and the noise classification result is output. A ResNet8 model using TinyML compression is used to extract environmental features from the audio feature set, including background noise power spectral density and reverberation time, to identify the current acoustic scene and output the scene identification results.

4. The walkie-talkie audio processing method based on edge AI according to claim 1, characterized in that, The step of dynamically adjusting the operating parameters of the audio processing filter based on the noise classification results and scene recognition results to generate a filter parameter set adapted to the current environment includes: Select the corresponding filtering strategy based on the noise classification results; Optimize different parameters of the filter based on scene recognition results; Real-time parameter adjustment of adaptive filter banks is achieved through a TinyML-optimized LSTM network; A residual signal feedback mechanism is introduced to automatically adjust the filter parameters based on the difference between the original signal and the filtered signal. Based on the above adjustments, a set of filter parameters adapted to the current environment is generated.

5. The walkie-talkie audio processing method based on edge AI according to claim 1, characterized in that, The steps of extracting voiceprint features from user speech samples using an embedded voiceprint learning unit, constructing a user-specific voiceprint library, and then performing personalized optimization on the filtered audio signal based on the filtering parameter set to generate personalized processed audio include: Collect user voice samples using the device's microphone; The embedded voiceprint learning unit is used to extract voiceprint features and store them in the device to build a user-specific voiceprint library; The original audio signal is initially filtered based on the filter parameter set generated by the noise classification results and scene recognition results; The user-specific voiceprint library is used to perform personalized optimization on the initially filtered audio signal to generate personalized processed audio.

6. The walkie-talkie audio processing method based on edge AI according to claim 1, characterized in that, The steps of training the model locally and generating model update parameters based on the personalized processed audio and audio feature set include: Incremental training is performed on edge devices using personalized processed audio and corresponding audio feature sets. The gradient changes of model parameters are calculated using the gradient descent algorithm to generate model update parameters for collaborative updating.

7. The walkie-talkie audio processing method based on edge AI according to claim 1, characterized in that, The step of updating the audio analysis model based on the model update parameters for subsequent analysis of the acquired audio signals includes: The model update parameters are encrypted using an encryption algorithm and sent to the central server or associated device through a distributed learning architecture with encrypted transmission. Homomorphic encryption technology is used to securely aggregate the updated parameters of the encrypted model to generate an updated audio analysis model; The updated model is distributed to each edge device to replace the original model and complete the update.

8. A walkie-talkie audio processing system based on edge AI, characterized in that, include: The audio acquisition module acquires raw audio signals through audio acquisition components and analyzes them in real time to obtain an audio feature set containing noise features and scene features; The edge computing module inputs the audio feature set into the audio analysis model deployed on the edge side and outputs noise classification results and scene recognition results; The adaptive filtering module dynamically adjusts the operating parameters of the audio processing filter based on the noise classification results and scene recognition results, generating a set of filtering parameters that are adapted to the current environment. The personalized optimization module extracts the voiceprint features of user speech samples through an embedded voiceprint learning unit, constructs a user-exclusive voiceprint library, and performs personalized optimization on the filtered audio signal based on the filtering parameter group to generate personalized processed audio. The local model training module, based on the personalized processed audio and audio feature set, completes model training locally and generates model update parameters. The model update module updates the audio analysis model based on the model update parameters for subsequent analysis of the collected audio signals.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images