Interphone audio processing method, system and device based on edge AI and medium

Abstract

The invention relates to the technical field of voice processing, in particular to an edge AI-based interphone audio processing method, system and device and a medium, and the method comprises the steps: collecting and analyzing an original audio through an audio collection assembly, and obtaining an audio feature set containing noise and scene features; inputting the image into an edge side audio analysis model to obtain a noise classification and scene recognition result; dynamically adjusting filter parameters, and generating a filtering parameter group adaptive to the current environment; extracting user voiceprint features to construct an exclusive voiceprint library, and optimizing the audio in combination with the filtering parameter group to generate a personalized processing audio; and training the model locally based on the personalized processing audio and the audio feature set, generating update parameters, and updating the audio analysis model for subsequent re-analysis. The technical problem that dynamic adaptation, personalized optimization and low-power-consumption and high-performance balance cannot be achieved at the same time in the prior art can be solved.

Description

Technical Field

[0001] The present application relates to the field of voice processing technology, and in particular to an edge AI-based walkie-talkie audio processing method, system, device, and medium. Background Art

[0002] With the rapid development of industrial production and mobile work scenarios, the demand for audio communication for walkie-talkies in complex noisy environments is becoming increasingly urgent. To address this, existing technologies generally use fixed-parameter analog filtering (such as LC resonant circuits), DSP-based digital filtering (such as FIR / IIR filters), or rely on cloud-based AI to process audio. However, fixed filtering parameters cannot adapt to dynamic noise and scene changes. DSP solutions lack intelligent feature extraction capabilities, resulting in limited improvement in signal-to-noise ratio. Cloud-based AI suffers from high latency, and existing solutions fail to consider user voiceprint differences. Furthermore, they face the conflict between high-performance processing and low-power consumption. This leads to the urgent need to address the technical issues of achieving dynamic adaptation, personalized optimization, and a balance between low power consumption and high performance.

[0003] Therefore, the technical problem that existing technologies cannot simultaneously achieve dynamic adaptation, personalized optimization and low power consumption and high performance balance needs to be solved urgently. Summary of the Invention

[0004] The main purpose of this application is to provide a walkie-talkie audio processing method, system, device and medium based on edge AI, aiming to solve the technical problem that the existing technology cannot simultaneously achieve dynamic adaptation, personalized optimization and low power consumption and high performance balance.

[0005] In order to achieve the above-mentioned invention objectives, the present application proposes an adaptive intercom audio processing method based on edge AI, the method comprising: The audio acquisition component collects the original audio signal and analyzes it in real time to obtain an audio feature set containing noise features and scene features; Input the audio feature set into the audio analysis model deployed on the edge side, and output noise classification results and scene recognition results; Dynamically adjusting the operating parameters of the audio processing filter according to the noise classification result and the scene recognition result to generate a filter parameter group adapted to the current environment; Extracting voiceprint features of user voice samples through an embedded voiceprint learning unit to build a user-specific voiceprint library, and performing personalized optimization on the filtered audio signal based on the filtering parameter group to generate personalized processed audio; Based on the personalized processed audio and audio feature set, completing model training locally and generating model update parameters; The audio analysis model is updated based on the model update parameters for subsequent analysis of the collected audio signal.

[0006] Furthermore, the step of collecting the original audio signal through the audio collection component and performing real-time analysis on the original audio signal to obtain an audio feature set including noise features and scene features includes: Use a multi-microphone array to collect raw audio signals; Performing real-time spectrum analysis on the original audio signal to obtain an audio signal represented in the frequency domain; Based on the audio signal represented in the frequency domain, an audio analysis model extracts noise features and scene features to generate an audio feature set.

[0007] Furthermore, the audio analysis model includes a ResNet8 model including an improved VDCNN network and TinyML compression, and the step of inputting the audio feature set into the audio analysis model deployed on the edge side and outputting the noise classification result and the scene recognition result includes: Input the audio feature set into the audio analysis model deployed on the edge side; The improved VDCNN network is used to classify noise features, wherein the VDCNN network contains a library of 12 industrial noise samples, identifies the current environmental noise type through matching and classification algorithms, and outputs noise classification results; The ResNet8 model compressed with TinyML is used to extract environmental features from the audio feature set, including background noise power spectrum density and reverberation time, identify the current acoustic scene, and output the scene recognition results.

[0008] Furthermore, the step of dynamically adjusting the working parameters of the audio processing filter according to the noise classification result and the scene recognition result to generate a filter parameter group adapted to the current environment includes: Select the corresponding filtering strategy according to the noise classification results; Optimize different parameters of the filter based on the scene recognition results; Real-time parameter adjustment of adaptive filter banks using TinyML-optimized LSTM networks; A residual signal feedback mechanism is introduced to automatically adjust the filter parameters according to the difference between the original signal and the filtered signal; Based on the above adjustments, a filtering parameter group adapted to the current environment is generated.

[0009] Furthermore, the steps of extracting voiceprint features of user voice samples through the embedded voiceprint learning unit, building a user-specific voiceprint library, and performing personalized optimization on the filtered audio signal based on the filtering parameter group to generate personalized processed audio include: Collect user voice samples through the device microphone; Use the embedded voiceprint learning unit to extract voiceprint features and store them in the device to build a user-specific voiceprint library; Perform preliminary filtering on the original audio signal based on the filter parameter group generated by the noise classification result and the scene recognition result; The audio signal after preliminary filtering is personalized optimized according to the user voiceprint library to generate personalized processed audio.

[0010] Furthermore, the step of completing model training locally and generating model update parameters based on the personalized processed audio and audio feature set includes: Perform local model training on edge devices based on personalized processed audio and audio feature sets to generate model update parameters; The model update parameters are encrypted using an encryption algorithm and prepared to be sent to a central server or other associated devices via a distributed learning architecture with encrypted transmission.

[0011] Furthermore, the step of updating the audio analysis model based on the model update parameter for subsequent analysis of the collected audio signal includes: The model update parameters are encrypted using an encryption algorithm and sent to the central server or associated devices through a distributed learning architecture with encrypted transmission; Use homomorphic encryption technology to securely aggregate the encrypted model update parameters to generate an updated audio analysis model; Distribute the updated model to each edge device, replacing the original model.

[0012] The second aspect of this application proposes an adaptive intercom audio processing system based on edge AI: The audio acquisition module collects the original audio signal through the audio acquisition component and analyzes it in real time to obtain an audio feature set containing noise features and scene features; An edge computing module inputs the audio feature set into an audio analysis model deployed on the edge side and outputs noise classification results and scene recognition results; An adaptive filtering module dynamically adjusts the working parameters of the audio processing filter according to the noise classification result and the scene recognition result to generate a filtering parameter group adapted to the current environment; A personalized optimization module, which extracts voiceprint features of user voice samples through an embedded voiceprint learning unit, builds a user-specific voiceprint library, and performs personalized optimization on the filtered audio signal based on the filtering parameter group to generate personalized processed audio; A local model training module, which completes model training locally and generates model update parameters based on the personalized processed audio and audio feature set; The model updating module updates the audio analysis model based on the model updating parameters for subsequent analysis of the collected audio signal.

[0013] The third aspect of the present application further proposes a device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the above methods when executing the computer program.

[0014] The fourth aspect of the present application further proposes a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of any of the above methods when executed by a processor.

[0015] Beneficial effects The technical effects that can be achieved are as follows: the audio feature set is obtained through real-time analysis by the audio acquisition component, the noise and scene recognition results are output by the edge-side audio analysis model, the filtering parameters are adaptively and dynamically adjusted, and the audio is optimized in combination with the user's exclusive voiceprint library, thereby realizing adaptive audio processing for the current environment; at the same time, through local model training and updating, the analysis capability is continuously improved, which effectively improves the poor audio processing effect caused by fixed filtering and lack of personalized adaptation in existing technologies, and improves the audio communication quality of walkie-talkies in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a flow chart of an edge AI-based intercom audio processing method according to an embodiment of the present application; Figure 2 This is a schematic block diagram of the structure of an edge AI-based intercom audio processing system according to an embodiment of the present application; Figure 3 This is a schematic block diagram of the structure of a computer device according to an embodiment of the present application; The realization of the objectives, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0018] Those skilled in the art will appreciate that, unless expressly stated otherwise, the singular forms "a", "an", "above", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of features, integers, steps, operations, elements, modules, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, modules, components, and / or groups thereof. It should be understood that when an element is said to be "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, "connected" or "coupled" as used herein may include wireless connections or wireless couplings. The term "and / or" used herein includes all or any module and all combinations of one or more associated listed items.

[0019] Those skilled in the art will understand that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art in the art to which this invention belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with their meanings in the context of the prior art and, unless specifically defined as such, will not be interpreted in an idealized or overly formal sense.

[0020] Reference Figure 1 , an embodiment of the present invention provides an intercom audio processing method based on edge AI, including steps S1-S6, specifically: S1. Collect the original audio signal through the audio acquisition component and analyze it in real time to obtain an audio feature set including noise features and scene features; S2. Input the audio feature set into an audio analysis model deployed on the edge side, and output noise classification results and scene recognition results; S3. Dynamically adjusting the operating parameters of the audio processing filter according to the noise classification result and the scene recognition result to generate a filter parameter group adapted to the current environment; S4. Extracting voiceprint features of the user's voice sample through the embedded voiceprint learning unit, building a user-specific voiceprint library, and performing personalized optimization on the filtered audio signal based on the filtering parameter group to generate personalized processed audio; S5. Based on the personalized processed audio and audio feature set, complete model training locally and generate model update parameters; S6. Update the audio analysis model based on the model update parameters for subsequent analysis of the collected audio signal.

[0021] Step S1 uses a multi-microphone array consisting of 4-6 MEMs microphones as an audio acquisition component to collect raw audio signals. Leveraging the spatial sound field capture capabilities of multiple microphones, this method allows for more comprehensive acquisition of environmental acoustic information than a single microphone. Real-time spectrum analysis is then performed on the raw audio signals, using an FFT algorithm to convert the time-domain signal into a frequency-domain representation with an FFT resolution of 5Hz, enabling precise analysis of different frequency components. Based on the frequency-domain signal, noise features (such as specific frequency peaks of mechanical noise in industrial environments and the energy distribution of burst noise) and scene features (such as background noise power spectrum density and reverberation time) are extracted, integrating these features to form an audio feature set containing both types of features. The advantage of this step is that the multi-microphone array improves the spatial accuracy of sound field acquisition, while real-time, high-resolution spectrum analysis ensures the timeliness and accuracy of feature extraction, providing reliable input for noise classification and scene recognition in subsequent edge-side models. For example, in a factory workshop scenario, a multi-microphone array simultaneously collects the 750Hz periodic noise of a running lathe and the worker's voice. After FFT conversion, the 750Hz noise peak (noise feature) and 0.8-second reverberation time (scene feature) can be extracted. The resulting audio feature set can accurately reflect the acoustic characteristics of the current environment.

[0022] In step S2, the audio feature set generated in step S1 is input into the audio analysis model deployed on the Rockchip RK1808 SoC (equipped with a 1.2TOPS computing power NPU). The audio analysis model is a lightweight intelligent analysis model (parameter size <100KB). The lightweight intelligent analysis model consists of the following two core branches: Improved VDCNN network branch: This branch is based on a sample library containing 12 types of industrial noise and is used to classify noise features. By matching feature vectors to match noise types, it can accurately identify the type of noise in the current environment. For example, it can quickly identify periodic impact noise on an oil drilling platform.

[0023] TinyML-compressed ResNet8 model branch: This model, compressed with TinyML technology to 48KB, is used to extract environmental features from the audio feature set, such as background noise power spectral density and reverberation time (RT60). It then uses an LSTM network to classify six preset scene modes, including industrial, outdoor, and indoor, providing a scenario-based basis for subsequent filtering adjustments.

[0024] At the same time, this lightweight audio analysis model has the characteristics of parameter size <100KB, and adopts INT8 asymmetric quantization (APoT algorithm), with accuracy loss <2%. It can achieve 20ms response on Rockchip RK1808SoC (equipped with 1.2TOPS computing power NPU), adapting to the low computing power requirements of edge devices and avoiding the delay problems caused by cloud processing.

[0025] Based on the noise classification and scene recognition results output by S2, step S3 dynamically adjusts the operating parameters of the 32nd-order FIR+IIR hybrid filter group through a TinyML-optimized LSTM network. This suppresses the corresponding frequency bands for different noises and optimizes the filter parameters based on the scenario. A Kalman filter is used to smooth parameter switching to avoid speech distortion caused by sudden changes. The filter supports 15 coefficient updates per second and introduces residual signal feedback to correct filter deviations. During data processing, the filter parameter group includes the 32nd-order coefficients of the FIR filter, the cutoff frequency and gain value of the IIR filter, and more. This method breaks through the limitations of traditional fixed parameters and achieves dual adaptation to both noise and scenarios. For example, in a factory workshop, when sudden impact noise occurs, the filter can quickly adjust the high-frequency attenuation coefficient while retaining the 300-3400Hz speech frequency band.

[0026] Step S4 uses the embedded voiceprint learning unit to collect a 30-second voice sample from the user (supporting 1,000 users / device capacity), extracts voiceprint features in characteristic frequency bands such as 2.4-3.6kHz, and constructs a dedicated voiceprint library, which is stored using AES-256 encryption. After completing preliminary filtering based on the filter parameter group generated in S3, the filtered audio is personalized and optimized in combination with the voiceprint library. For example, for female users, the gain of the 2.4-3.6kHz frequency band is enhanced to prevent high-frequency speech from being mistakenly filtered out. In data processing, the optimization coefficient is dynamically corrected through the PESQ score to ensure improved voice intelligibility after personalized processing. The advantage is that it solves the problem of lack of personalization in general filtering. For example, when a female user uses it, the system recognizes her voiceprint features and adjusts the suppression strength of the relevant frequency bands to make the high-frequency components of the command clearer.

[0027] In step S5, based on the personalized audio processed in S4 and the audio feature set in S1, a federated learning architecture is used to train the model locally. The audio analysis model is incrementally trained using a gradient descent algorithm over a 24-hour period. The generated model update parameters undergo 80% gradient compression, retaining only the key weight adjustment values. During data processing, the training process occupies ≤30% of CPU resources to avoid impacting real-time audio processing. This advantage is that the model can be evolved locally without uploading raw voice data, which protects privacy and reduces power consumption. For example, oilfield inspection equipment uses idle time every night to train models based on data collected that day.

[0028] In step S6, the model update parameters generated in S5 are transmitted via AES-256 encryption. After secure aggregation using Paillier homomorphic encryption, the network weights and classification thresholds of the audio analysis model are updated. This update process does not interrupt audio processing, and the switching delay is less than 50ms. Regarding data processing, the updated model parameters are immediately applied to the next audio analysis, forming a closed loop of "collection-analysis-optimization-training-update." This advantage ensures that the model continuously adapts to changes in the environment and users. For example, after iteration, the device's recognition response speed to specific noises increases, and the speech signal-to-noise ratio is further optimized.

[0029] In one embodiment, the step of collecting the original audio signal by the audio collection component and performing real-time analysis on the original audio signal to obtain the audio feature set including the noise feature and the scene feature includes: s20, using a multi-microphone array to collect original audio signals; s21. Performing real-time spectrum analysis on the original audio signal to obtain an audio signal represented in the frequency domain; s22. Based on the audio signal represented in the frequency domain, an audio analysis model extracts noise features and scene features to generate an audio feature set.

[0030] In this embodiment, a circular array of 4-6 MEMs microphones is used as the audio acquisition component. Leveraging the spatial distribution of multiple microphones, this approach enables omnidirectional acquisition of the spatial sound field. Compared to a single microphone, it can more accurately capture sound sources and noise information from different directions, making it particularly suitable for industrial scenarios where multi-directional noise coexists. A real-time spectrum analysis module processes the raw audio signal, converting the time-domain signal into a frequency-domain representation using an FFT algorithm. With an FFT resolution of 5Hz, it can precisely analyze frequency components in the 0-8kHz range, providing high-resolution data support for subsequent feature extraction. The audio analysis model (with less than 100KB of parameters) extracts noise features (such as the frequency peak of specific industrial noise and the energy distribution of burst noise) and scene features (such as the power spectral density of background noise and the reverberation time (RT60)) based on the frequency-domain signal. These features are integrated into an audio feature set, laying the foundation for subsequent noise classification and scene recognition. The advantages of this process are that multi-microphone acquisition improves the spatial accuracy of sound field capture, high-resolution spectrum analysis ensures feature accuracy, and the lightweight model adapts to the computing power requirements of edge devices.

[0031] In one embodiment, the audio analysis model includes a ResNet8 model including an improved VDCNN network and TinyML compression, and the step of inputting the audio feature set into the audio analysis model deployed on the edge side and outputting a noise classification result and a scene recognition result includes: s30. Input the audio feature set into the audio analysis model deployed on the edge side; s31. Use an improved VDCNN network to classify noise features, where the VDCNN network contains a library of 12 industrial noise samples, identifies the current environmental noise type through matching and classification algorithms, and outputs a noise classification result; s32. Use the ResNet8 model compressed by TinyML to extract environmental features from the audio feature set, including background noise power spectrum density and reverberation time, identify the current acoustic scene, and output the scene recognition result.

[0032] In this embodiment, the audio analysis model is deployed on an SoC with an integrated AI accelerator (such as the Rockchip RK1808), leveraging the computing power of the NPU for efficient inference and avoiding cloud-to-cloud transmission latency. The improved VDCNN network, based on a sample library containing 12 types of industrial noise (such as mechanical roar and explosions), uses feature vector matching and classification algorithms to accurately identify the current noise type. For example, it can identify periodic impact noise on an oil drilling platform. A ResNet8 model (48KB) compressed using TinyML technology extracts environmental features such as background noise power spectral density (reflecting the overall noise energy distribution) and reverberation time (RT60) (reflecting the acoustic characteristics of the environment) from the audio feature set. It then identifies acoustic scenes such as industrial, outdoor, and indoor, and offers six preset industrial environment modes for switching. The advantage of this process lies in the localized edge processing, which achieves a response speed of 20ms. The two models work together to simultaneously output noise and scene information, providing dual basis for subsequent filtering adjustments.

[0033] In one embodiment, the step of dynamically adjusting the operating parameters of the audio processing filter according to the noise classification result and the scene recognition result to generate a filter parameter group adapted to the current environment includes: s40, selecting a corresponding filtering strategy according to the noise classification result; s41, optimizing different parameters of the filter based on the scene recognition results; s42. Implement real-time parameter adjustment of adaptive filter banks through LSTM networks optimized by TinyML; s43. Introduce a residual signal feedback mechanism to automatically adjust the filter parameters according to the difference between the original signal and the filtered signal; s44. Based on the above adjustments, a filtering parameter group adapted to the current environment is generated.

[0034] In this embodiment, targeted filtering strategies are selected based on noise classification results (e.g., mechanical noise, sudden impact noise). For example, for mechanical noise, specific low-frequency band suppression is emphasized, while for sudden noise, transient attenuation is enhanced. Parameters are optimized based on scene recognition results (e.g., industrial and indoor scenes). For example, the dereverberation coefficient is enhanced in reverberant scenes, while high-frequency filtering is enhanced in outdoor strong wind scenes. A TinyML-optimized LSTM network is used to update the parameters of a 32nd-order FIR+IIR hybrid filter bank, supporting an adjustment frequency of 15 times per second to ensure real-time adaptation to environmental changes. A residual signal feedback mechanism is introduced to calculate the difference between the original and filtered signals, and filter parameters are reversely corrected to reduce filtering deviation, for example, to prevent over-suppression of critical speech bands. These adjustments are combined to generate a filter parameter set containing key parameters such as FIR coefficients and IIR cutoff frequency. This process breaks through the limitations of traditional fixed parameters and, through multi-dimensional dynamic adjustment, enables the filter to adapt to both noise type and scene characteristics.

[0035] In one embodiment, the steps of extracting voiceprint features of user voice samples through the embedded voiceprint learning unit, building a user-specific voiceprint library, and performing personalized optimization on the filtered audio signal based on the filtering parameter group to generate personalized processed audio include: s50, collect user voice samples through the device microphone; s51. Use the embedded voiceprint learning unit to extract voiceprint features and store them in the device to build a user-specific voiceprint library; s52, performing preliminary filtering on the original audio signal based on the filter parameter group generated based on the noise classification result and the scene recognition result; s53. Perform personalized optimization on the audio signal after preliminary filtering according to the user-specific voiceprint library to generate personalized processed audio.

[0036] In this embodiment, a 30-second voice sample of the user is collected through the device microphone, covering voice features in different frequency bands to ensure the comprehensiveness of voiceprint extraction. The embedded voiceprint learning unit extracts voiceprint features from the sample (such as the 2.4-3.6kHz high-frequency features of female users) and stores them in the device to build a dedicated voiceprint library, which supports a capacity of 1,000 users / devices. The original audio signal is preliminarily filtered based on the filter parameter group to remove most of the environmental noise, such as suppressing mechanical noise in industrial scenarios. The signal after preliminary filtering is personalized and optimized according to the voiceprint library. For example, the gain of the 2.4-3.6kHz frequency band is increased for female users to avoid their high-frequency voice being mistakenly filtered out, thereby improving the current 65% satisfaction rate of female users. The advantage of this process is that it can achieve personalized adaptation through voiceprint learning, solve the problem of voice distortion of specific users caused by general filtering, and improve voice intelligibility.

[0037] In one embodiment, the step of locally completing model training and generating model update parameters based on the personalized processed audio and audio feature set includes: s60, performing incremental training on the edge device using the personalized processed audio and the corresponding audio feature set; s61. Calculate the gradient change of model parameters through the gradient descent algorithm to generate model update parameters for collaborative update.

[0038] In this embodiment, incremental training is performed on edge devices (such as walkie-talkies equipped with the RK1808 chip) over a 24-hour period using personalized processed audio collected that day and the corresponding audio feature set, avoiding the high computing power consumption of full training. The gradient change of model parameters (such as VDCNN network weights and ResNet8 feature extraction coefficients) is calculated using a gradient descent algorithm to generate local model update parameters, reflecting the optimization direction of the model on the new data. The update parameters are limited to a key subset that can be transferred (such as classifier thresholds and feature extraction kernels), prioritizing core parameters that significantly impact noise classification and scene recognition accuracy. This reduces data transmission volume, for example, by compressing them to 20% of the original parameter volume. Based on this key parameter subset, model update parameters are generated for collaborative updating, paving the way for subsequent cross-device model collaborative optimization. The advantages of this process are that local training avoids uploading raw data, protecting privacy and reducing power consumption. Incremental training and parameter reduction enable sustainable model evolution and adaptability to edge device computing power.

[0039] In one embodiment, the step of updating the audio analysis model based on the model update parameter for subsequent analysis of the collected audio signal includes: s70, encrypting the model update parameters using an encryption algorithm and sending them to a central server or associated device via a distributed learning architecture with encrypted transmission; s71. Use homomorphic encryption technology to securely aggregate the encrypted model update parameters to generate an updated audio analysis model; s72. Distribute the updated model to each edge device, replacing the original model to complete the update.

[0040] In this embodiment, a distributed architecture of federated learning is adopted to achieve collaborative sharing of model update parameters among multiple devices. For example, multiple walkie-talkies in oil field inspections can collaboratively optimize the model. The model update parameters are encrypted using the AES-256 encryption algorithm and sent to the central server or associated devices through a secure transmission channel to ensure the security of parameter transmission. The encrypted parameters uploaded by multiple devices are securely aggregated using Paillier homomorphic encryption technology, and parameter fusion is completed without decryption to generate an updated audio analysis model to avoid parameter leakage. The updated model is distributed to each edge device to replace the original model. The update process does not interrupt audio processing, and the switching delay is <50ms. The advantage of this process is that encrypted transmission and secure aggregation ensure data privacy. The distributed architecture enables the model to evolve based on multi-device data, continuously improve analysis accuracy, and adapt to more complex scenarios.

[0041] Reference Figure 2 , is a block diagram of the structure of an intercom audio processing system based on edge AI in one embodiment of the present application. The system includes: The audio acquisition module 100 is used to collect the original audio signal through the audio acquisition component and perform real-time analysis on it to obtain an audio feature set including noise features and scene features; The edge computing module 200 is configured to input the audio feature set into an audio analysis model deployed on the edge side, and output noise classification results and scene recognition results; An adaptive filtering module 300 is configured to dynamically adjust the operating parameters of the audio processing filter according to the noise classification result and the scene recognition result, and generate a filtering parameter group adapted to the current environment; The personalized optimization module 400 is used to extract the voiceprint features of the user's voice sample through the embedded voiceprint learning unit, build a user-specific voiceprint library, and perform personalized optimization on the filtered audio signal based on the filtering parameter group to generate personalized processed audio; A local model training module 500 is configured to locally complete model training and generate model update parameters based on the personalized processed audio and audio feature set; The model updating module 600 is configured to update the audio analysis model based on the model updating parameters for subsequent analysis of the collected audio signal.

[0042] In one embodiment, the audio acquisition module 100 includes: A signal acquisition unit, configured to acquire original audio signals using a multi-microphone array; A spectrum analysis unit, configured to perform real-time spectrum analysis on the original audio signal to obtain an audio signal represented in the frequency domain; The feature extraction unit is configured to extract noise features and scene features based on the audio signal represented in the frequency domain by an audio analysis model to generate an audio feature set.

[0043] In one embodiment, the edge computing module 200 includes: A model input unit, used to input the audio feature set into the audio analysis model deployed on the edge side; A noise classification unit is used to classify noise features using an improved VDCNN network, wherein the VDCNN network contains a library of 12 types of industrial noise samples, identifies the current environmental noise type through a matching and classification algorithm, and outputs a noise classification result; The scene recognition unit uses a ResNet8 model compressed with TinyML to extract environmental features from the audio feature set, including background noise power spectrum density and reverberation time, identify the current acoustic scene, and output the scene recognition result.

[0044] In one embodiment, the adaptive filtering module 300 includes: A strategy selection unit, used to select a corresponding filtering strategy according to the noise classification result; A parameter optimization unit, used to optimize different parameters of the filter based on the scene recognition results; A real-time adjustment unit for real-time parameter adjustment of adaptive filter banks using TinyML-optimized LSTM networks; Feedback adjustment unit, used to introduce a residual signal feedback mechanism to automatically adjust the filter parameters according to the difference between the original signal and the filtered signal; The parameter group generating unit is used to synthesize the above adjustments and generate a filtering parameter group adapted to the current environment.

[0045] In one embodiment, the personalized optimization module 400 includes: A sample collection unit, used to collect user voice samples through the device microphone; A voiceprint library construction unit is used to extract voiceprint features using an embedded voiceprint learning unit and store them in the device to build a user-specific voiceprint library; A preliminary filtering unit, configured to perform preliminary filtering on the original audio signal based on a filtering parameter group generated based on a noise classification result and a scene recognition result; The personalized adjustment unit is used to perform personalized optimization on the audio signal after preliminary filtering according to the user-specific voiceprint library to generate personalized processed audio.

[0046] In one embodiment, the local model training module 500 includes: An incremental training unit, configured to perform incremental training on edge devices using personalized processed audio and corresponding audio feature sets; The gradient calculation unit is used to calculate the gradient changes of model parameters through the gradient descent algorithm and generate model update parameters for collaborative updating.

[0047] In one embodiment, the model updating module 600 includes: A parameter transmission unit is used for an encrypted sending unit, which is used to encrypt the model update parameters using an encryption algorithm and send them to a central server or an associated device through a distributed learning architecture with encrypted transmission; A secure aggregation unit, configured to securely aggregate the encrypted model update parameters using homomorphic encryption technology to generate an updated audio analysis model; The model distribution unit is used to distribute the updated model to each edge device, replacing the original model to complete the update.

[0048] Reference Figure 3 In the embodiment of the present application, a computer device is also provided. The computer device may be a server, and its internal structure may be as follows: Figure 3As shown. The computer device includes a processor, internal memory, storage medium (non-volatile storage medium) and network interface connected via a system bus. The processor of the computer design is used to provide computing and control capabilities. The memory of the computer device includes the above-mentioned storage medium (non-volatile storage medium) and internal memory. The storage medium (non-volatile storage medium) stores an operating system, computer programs and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the storage medium (non-volatile storage medium). The database of the computer device is used to store usage data, etc. in the process of a walkie-talkie audio processing method based on edge AI. The network interface of the computer device is used to communicate with an external terminal via a network connection. Furthermore, the above-mentioned computer device can also be provided with an input device and a display screen, etc. When the above computer program is executed by the processor, it implements an edge AI-based walkie-talkie audio processing method, including the following steps: collecting the original audio signal through the audio acquisition component, and performing real-time analysis on it to obtain an audio feature set containing noise features and scene features; inputting the audio feature set into the audio analysis model deployed on the edge side, and outputting the noise classification result and the scene recognition result; dynamically adjusting the working parameters of the audio processing filter according to the noise classification result and the scene recognition result, and generating a filter parameter group adapted to the current environment; extracting the voiceprint features of the user voice sample through the embedded voiceprint learning unit, building a user-exclusive voiceprint library, and performing personalized optimization on the filtered audio signal based on the filter parameter group to generate personalized processed audio; based on the personalized processed audio and audio feature set, completing model training locally and generating model update parameters; updating the audio analysis model based on the model update parameters for subsequent analysis of the collected audio signal. Those skilled in the art will understand that Figure 3 The structure shown in is merely a block diagram of a portion of the structure related to the present application solution and does not constitute a limitation on the computer device to which the present application solution is applied.

[0049] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program implements an edge AI-based walkie-talkie audio processing method, comprising the following steps: collecting raw audio signals through an audio acquisition component and performing real-time analysis on the raw audio signals to obtain an audio feature set including noise features and scene features; inputting the audio feature set into an audio analysis model deployed on the edge side to output a noise classification result and a scene recognition result; dynamically adjusting the operating parameters of the audio processing filter based on the noise classification result and the scene recognition result to generate a filter parameter set adapted to the current environment; extracting the voiceprint features of the user's voice sample through an embedded voiceprint learning unit to construct a user-specific voiceprint library, and performing personalized optimization on the filtered audio signal based on the filter parameter set to generate personalized processed audio; based on the personalized processed audio and the audio feature set, completing model training locally and generating model update parameters; and updating the audio analysis model based on the model update parameters for subsequent analysis of the collected audio signal. It is understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0050] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided in this application and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct RAM bus dynamic RAM (DRDRAM), and RAM bus dynamic RAM (RDRAM), etc.

[0051] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, apparatus, article, or method comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, apparatus, article, or method. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, apparatus, article, or method comprising the element.

[0052] The above description is only a preferred embodiment of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A walkie-talkie audio processing method based on edge AI, characterized in that: The method comprises: The audio acquisition component collects the original audio signal and analyzes it in real time to obtain an audio feature set containing noise features and scene features; Input the audio feature set into the audio analysis model deployed on the edge side, and output noise classification results and scene recognition results; Dynamically adjusting the operating parameters of the audio processing filter according to the noise classification result and the scene recognition result to generate a filter parameter group adapted to the current environment; Extracting voiceprint features of user voice samples through an embedded voiceprint learning unit to build a user-specific voiceprint library, and performing personalized optimization on the filtered audio signal based on the filtering parameter group to generate personalized processed audio; Based on the personalized processed audio and audio feature set, completing model training locally and generating model update parameters; The audio analysis model is updated based on the model update parameters for subsequent analysis of the collected audio signal.

2. The intercom audio processing method based on edge AI according to claim 1 is characterized in that: The steps of collecting the original audio signal by the audio collection component and performing real-time analysis on the original audio signal to obtain an audio feature set including noise features and scene features include: Use a multi-microphone array to collect raw audio signals; Performing real-time spectrum analysis on the original audio signal to obtain an audio signal represented in the frequency domain; Based on the audio signal represented in the frequency domain, an audio analysis model extracts noise features and scene features to generate an audio feature set.

3. The intercom audio processing method based on edge AI according to claim 1 is characterized in that: The audio analysis model includes a ResNet8 model including an improved VDCNN network and TinyML compression. The step of inputting the audio feature set into the audio analysis model deployed on the edge side and outputting the noise classification result and the scene recognition result includes: Input the audio feature set into the audio analysis model deployed on the edge side; The improved VDCNN network is used to classify noise features, wherein the VDCNN network contains a library of 12 industrial noise samples, identifies the current environmental noise type through matching and classification algorithms, and outputs noise classification results; The ResNet8 model compressed with TinyML is used to extract environmental features from the audio feature set, including background noise power spectrum density and reverberation time, identify the current acoustic scene, and output the scene recognition results.

4. The intercom audio processing method based on edge AI according to claim 1 is characterized in that: The step of dynamically adjusting the working parameters of the audio processing filter according to the noise classification result and the scene recognition result to generate a filter parameter group adapted to the current environment includes: Select the corresponding filtering strategy according to the noise classification results; Optimize different parameters of the filter based on the scene recognition results; Real-time parameter adjustment of adaptive filter banks using TinyML-optimized LSTM networks; A residual signal feedback mechanism is introduced to automatically adjust the filter parameters according to the difference between the original signal and the filtered signal; Based on the above adjustments, a filtering parameter group adapted to the current environment is generated.

5. The intercom audio processing method based on edge AI according to claim 1 is characterized in that: The steps of extracting voiceprint features of user voice samples through the embedded voiceprint learning unit, building a user-specific voiceprint library, and performing personalized optimization on the filtered audio signal based on the filtering parameter group to generate personalized processed audio include: Collect user voice samples through the device microphone; Use the embedded voiceprint learning unit to extract voiceprint features and store them in the device to build a user-specific voiceprint library; Perform preliminary filtering on the original audio signal based on the filter parameter group generated by the noise classification result and the scene recognition result; The audio signal after preliminary filtering is personalized optimized according to the user-specific voiceprint library to generate personalized processed audio.

6. The intercom audio processing method based on edge AI according to claim 1 is characterized in that: The step of completing model training locally and generating model update parameters based on the personalized processed audio and audio feature set includes: Incremental training is performed on edge devices using personalized processed audio and corresponding audio feature sets; The gradient change of model parameters is calculated through the gradient descent algorithm to generate model update parameters for collaborative updating.

7. The intercom audio processing method based on edge AI according to claim 1 is characterized in that: The step of updating the audio analysis model based on the model update parameters for subsequent analysis of the collected audio signal includes: The model update parameters are encrypted using an encryption algorithm and sent to the central server or associated devices through a distributed learning architecture with encrypted transmission; Use homomorphic encryption technology to securely aggregate the encrypted model update parameters to generate an updated audio analysis model; Distribute the updated model to each edge device, replacing the original model to complete the update.

8. An edge AI-based intercom audio processing system, characterized in that: include: The audio acquisition module collects the original audio signal through the audio acquisition component and analyzes it in real time to obtain an audio feature set containing noise features and scene features; An edge computing module inputs the audio feature set into an audio analysis model deployed on the edge side and outputs noise classification results and scene recognition results; An adaptive filtering module dynamically adjusts the working parameters of the audio processing filter according to the noise classification result and the scene recognition result to generate a filtering parameter group adapted to the current environment; A personalized optimization module, which extracts voiceprint features of user voice samples through an embedded voiceprint learning unit, builds a user-specific voiceprint library, and performs personalized optimization on the filtered audio signal based on the filtering parameter group to generate personalized processed audio; A local model training module, which completes model training locally and generates model update parameters based on the personalized processed audio and audio feature set; The model updating module updates the audio analysis model based on the model updating parameters for subsequent analysis of the collected audio signal.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images