Android device audio noise reduction method, controller, device and medium
By integrating and calling the DTLN noise reduction model into the audio hardware abstraction layer of Android devices, the problem that existing technologies cannot adapt to diverse and complex scenarios is solved, achieving efficient audio noise reduction and good system adaptability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ALLWINNER TECH CO LTD
- Filing Date
- 2026-05-09
- Publication Date
- 2026-07-31
AI Technical Summary
Existing noise reduction algorithms cannot adapt to the diverse and complex scenarios of Android devices and cannot effectively suppress non-steady-state noise.
The DTLN noise reduction model is trained based on an open-source dataset, and after its format is converted, it is integrated into the audio hardware abstraction layer of the Android device. The DTLN noise reduction model is then called through the audio hardware abstraction layer to perform real-time noise reduction processing.
It achieves efficient audio noise reduction in diverse and complex scenarios on Android devices, with strong adaptability, excellent noise reduction effect, low resource consumption, and good system stability.
Smart Images
Figure CN122493870A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer technology, and in particular to an audio noise reduction method, controller, device, and medium for Android devices. Background Technology
[0002] With the rapid development of mobile internet and AI technology, audio noise reduction technology is being used more and more widely on Android devices, especially in voice interaction or voice acquisition scenarios such as voice calls, application recording, and voice assistants, where users' requirements for audio quality are increasing.
[0003] Existing noise reduction algorithms can only suppress steady-state noise at specific frequencies and cannot adapt to the diverse and complex scenarios of Android devices. Summary of the Invention
[0004] In view of this, the purpose of this invention is to provide an audio noise reduction method, controller, device and medium for Android devices, which can complete audio noise reduction in diverse and complex scenarios of Android devices.
[0005] In a first aspect, embodiments of the present invention provide an audio noise reduction method for an Android device, comprising: The DTLN denoising model was trained based on an open-source dataset. The DTLN noise reduction model is converted into a format and then encapsulated. The encapsulated DTLN noise reduction model is then integrated into the audio hardware abstraction layer of the Android device. The target audio data is obtained by calling the DTLN noise reduction model through the audio hardware abstraction layer to perform real-time noise reduction on the audio acquisition data.
[0006] In some optional embodiments, the DTLN denoising model trained on an open-source dataset includes: Obtain open-source audio data from the aforementioned open-source dataset; The model training dataset is obtained by filtering from the open-source audio data according to the preset noise reduction requirements. The model training data includes a target type of human voice dataset and a target scene noise dataset. The DTLN denoising model is trained using the model training dataset and a preset DTLN model training environment.
[0007] In some optional embodiments, training the DTLN denoising model based on the model training dataset and a preset DTLN model training environment includes: The model training dataset is divided into a training set and a validation set; Input the training set and the validation set into the DTLN model training environment; Configure the model training parameters of the DTLN model training environment, and obtain the initial DTLN model by training the DTLN model based on the training set; The initial DTLN model was evaluated for noise reduction based on the validation set to obtain the evaluation results. If the evaluation results indicate that the noise reduction is satisfactory, the initial DTLN model will be configured as the DTLN noise reduction model. If the evaluation results indicate that the noise reduction is unsatisfactory, the data components and proportions of the model training dataset are adjusted, and the initial DTLN model is trained and updated through the DTLN model training environment.
[0008] In some optional embodiments, the step of converting the format of the DTLN denoising model and then encapsulating it includes: The first format model weight file of the DTLN denoising model is converted into a second format model weight file. The first format model weight file represents the model weight file corresponding to the TensorFlow architecture, and the second format model weight file represents the ONNX format model weight file. The second format model weight file is converted into a third format model weight file using a preset conversion tool. The third format model weight file represents an NCNN format model weight file. After encapsulating the DTLN denoising model with the third format model weight file into functional blocks, a DTLN denoising algorithm library and an API interface for calling the DTLN denoising algorithm library are obtained.
[0009] In some optional embodiments, the step of obtaining the target audio data by calling the DTLN noise reduction model through the audio hardware abstraction layer to perform real-time noise reduction processing on the audio acquisition data includes: When the input stream of the audio hardware abstraction layer is open, the first audio format of the current recording is compared with the preset second audio format to obtain the comparison result; If the comparison result indicates that the first audio format and the second audio format are the same, the API interface is initialized; When the audio hardware abstraction layer reads the recording data through its input stream reading interface, it calls the initialized API interface to perform real-time noise reduction processing on the recording data to obtain the target audio data.
[0010] In some optional embodiments, after calling the initialized API interface to perform real-time noise reduction processing on the recording data to obtain the target audio data, the method further includes: When the input stream of the audio hardware abstraction layer is closed, the calling resources of the API interface are destructed and released.
[0011] In some optional embodiments, the step of obtaining the target audio data by calling the DTLN noise reduction model through the audio hardware abstraction layer to obtain the target audio data further includes: When the input stream of the audio hardware abstraction layer is open, the first audio format of the current recording is compared with the preset second audio format to obtain the comparison result; If the comparison result indicates that the first audio format and the second audio format are the same, the real-time application scenario of the recording data is obtained. If the real-time application scenario is the same as the preset scenario, the initialized API interface is called to perform real-time noise reduction processing on the recording data to obtain the target audio data. If the comparison results indicate that the first audio format and the second audio format are different, or if the real-time application scenario is different from the preset scenario, the initialized API interface will not be called.
[0012] Secondly, embodiments of the present invention provide a controller, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described method.
[0013] Thirdly, embodiments of the present invention provide an audio noise reduction device for an Android device, including the controller described above.
[0014] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a processor-executable program, which, when executed by a processor, is used to perform the method described above.
[0015] Implementing the embodiments of the present invention has the following beneficial effects: The embodiments of the present invention provide an audio noise reduction method for Android devices, comprising: training a DTLN noise reduction model based on an open-source dataset; converting the format of the DTLN noise reduction model and encapsulating it, and integrating the encapsulated DTLN noise reduction model into the audio hardware abstraction layer of the Android device; and using the audio hardware abstraction layer to call the DTLN noise reduction model to perform real-time noise reduction processing on the audio acquisition data to obtain the target audio data. By integrating and calling the DTLN noise reduction model in the audio hardware abstraction layer for noise reduction processing, the method is simple to use, easy to port, and can adapt to the diverse and complex scenarios of Android devices. Attached Figure Description
[0016] Figure 1This is a flowchart of an audio noise reduction method for Android devices provided in an embodiment of the present invention; Figure 2 This is a flowchart illustrating the deployment process of the DTLN noise reduction model provided in this embodiment of the invention. Figure 3 This is a comparison chart showing the effects of traditional noise reduction provided in the embodiments of the present invention and DTLN noise reduction of this application; Figure 4 This is a schematic block diagram of a controller provided in an embodiment of the present invention. Detailed Implementation
[0017] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.
[0018] In the description of this invention, it should be understood that the orientation descriptions, such as up, down, front, back, left, right, etc., are based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention.
[0019] In the description of this invention, "several" means one or more, "more than" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. The use of "first" and "second" in the description is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.
[0020] In the description of this invention, unless otherwise explicitly defined, terms such as "set up," "install," and "connect" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this invention in conjunction with the specific content of the technical solution.
[0021] This invention provides an audio noise reduction method, controller, device, and medium for Android devices. The Android device audio noise reduction method includes: training a DTLN noise reduction model based on an open-source dataset; converting the format of the DTLN noise reduction model and encapsulating it; integrating the encapsulated DTLN noise reduction model into the audio hardware abstraction layer (HIPL) of the Android device; and using the HIPL to call the DTLN noise reduction model to perform real-time noise reduction processing on the acquired audio data to obtain the target audio data. By integrating and calling the DTLN noise reduction model at the HIPL, the noise reduction process is simple to implement, easy to port, and adaptable to the diverse and complex scenarios of Android devices.
[0022] Reference Figure 1 , Figure 1 This is a flowchart of an audio noise reduction method for an Android device provided by an embodiment of the present invention. The audio noise reduction method for an Android device includes, but is not limited to, the following steps: S100, a DTLN denoising model trained on an open-source dataset; S200, after converting the format of the DTLN noise reduction model, it is encapsulated and the encapsulated DTLN noise reduction model is integrated into the audio hardware abstraction layer of the Android device. S300, the target audio data is obtained by calling the DTLN noise reduction model through the audio hardware abstraction layer to perform real-time noise reduction processing on the audio acquisition data.
[0023] Specifically, model training is conducted based on open-source speech and noise datasets to obtain DTLN denoising models adapted to multiple scenarios. The trained DTLN denoising models are then subjected to format conversion and algorithm library encapsulation to form a lightweight denoising algorithm library that can run on the Android platform CPU. The encapsulated DTLN denoising algorithm library is integrated into the audio hardware abstraction layer of the Android device. The DTLN denoising algorithm library is called through the audio hardware abstraction layer to perform real-time denoising processing on the audio data collected by the device and output the denoised target audio data.
[0024] The DTLN (Dual-Transform Learning Network) architecture, an open-source dual-transform learning network, is used to train the DTLN denoising model. This enables the DTLN denoising model to achieve real-time denoising results with relatively few parameters and low computational requirements, while ensuring effective operation on resource-constrained Android devices. This application requires preliminary preparations for model deployment on a local server, including setting up the model training environment, training a vendor-customized performance model, validating model performance, and implementing the NCNN transformation.
[0025] In the audio hardware abstraction layer of Android devices, the native API is deployed and integrated with the DTLN noise reduction model and customized model effect files. Relevant scene condition judgments are added in the adaptation implementation to adapt to the switching requirements of various AI noise reduction scenarios.
[0026] In some optional embodiments, the step of training the DTLN noise reduction model based on the open-source dataset includes: obtaining open-source audio data from the open-source dataset; selecting a model training dataset from the open-source audio data according to preset noise reduction requirements, wherein the model training data includes a target type of human voice dataset and a target scene noise dataset; and training the DTLN noise reduction model according to the model training dataset and a preset DTLN model training environment.
[0027] Specifically, the required open-source audio data should be obtained from publicly available audio data repositories (such as the audio datasets from the Open DNS Challenge or the OpenSLR website). Open-source audio data should include at least: a clean speech dataset for model learning of human voice features, and a diverse noise dataset for simulating real-world interference environments. Introducing open-source datasets effectively reduces the data acquisition cost for model training while ensuring the diversity and coverage of the training data.
[0028] Based on the preset noise reduction requirements of the actual application of the device, the above-acquired open-source audio data is precisely screened and organized to construct a dedicated model training dataset.
[0029] Voice dataset screening: Extract target type voice data (such as Mandarin voice, voice of a specific language, etc.) from open source voice data to ensure the quality and standardization of the voice data.
[0030] Noise dataset filtering: Based on the target scenarios in which the product is actually used (such as environmental noise in offices, outdoors, natural weather, traffic roads, airports, homes, shopping malls, etc.), noise samples for the corresponding scenarios are filtered out from open source noise libraries.
[0031] Data fusion: The selected target human voice dataset is matched and fused with the target scene noise dataset to generate noisy speech training samples, and finally a model training dataset covering the target type of human voice and the target application scenario is obtained.
[0032] The DTLN model training environment in this application is built on a deep learning framework (such as TensorFlow) and supported by local server computing power. The completed model training dataset is input into this DTLN model training environment, and iterative training is performed according to the set network structure and hyperparameters. By continuously optimizing the model parameters, the trained DTLN denoising model can accurately identify noise features in the target scene and achieve high-quality noise reduction of the target audio, ultimately resulting in a high-performance DTLN denoising model.
[0033] In some optional embodiments, the step of training the DTLN denoising model based on the model training dataset and a preset DTLN model training environment includes: dividing the model training dataset into a training set and a validation set; inputting the training set and the validation set into the DTLN model training environment; configuring the model training parameters of the DTLN model training environment, and training the DTLN model based on the training set to obtain an initial DTLN model; evaluating the initial DTLN model for denoising based on the validation set to obtain an evaluation result; if the evaluation result indicates that the denoising is qualified, configuring the initial DTLN model as the DTLN denoising model; if the evaluation result indicates that the denoising is unqualified, adjusting the data components and proportions of the model training dataset, and then training and updating the initial DTLN model through the DTLN model training environment.
[0034] Specifically, the pre-built model training dataset is divided into a training set and a validation set according to a certain ratio or a preset strategy. The training set is used for the main model training and parameter iterative optimization, while the validation set is used for model performance evaluation. The divided training and validation sets are then input into a preset DTLN model training environment. Training parameters such as the learning rate, batch size, number of iterations, and loss function are configured for the DTLN model training environment. Subsequently, the DTLN model structure is trained based on the training set, and the model weights are continuously optimized through backpropagation to obtain the initial DTLN model.
[0035] The initial DTLN model is tested for denoising performance using noisy frequency data in the validation set. Evaluation metrics (such as signal-to-noise ratio improvement and perceptual quality assessment metrics) are used to generate denoising evaluation results to determine if the model's denoising performance meets application requirements. If the evaluation results show that the initial DTLN model's denoising performance meets the preset standard (i.e., denoising performance is qualified), then the initial DTLN model is determined as the final usable DTLN denoising model. If the evaluation results show that the model's denoising performance is unqualified, the data composition and noise proportions of the model training dataset are adjusted in reverse based on the evaluation analysis results. The training dataset is then re-screened and reconstructed, and then input into the DTLN model training environment for a new round of training and model updates until the denoising performance meets the requirements.
[0036] In one specific embodiment, after the DTLN model is trained, a Keras model weight file in .h5 format corresponding to the TensorFlow architecture is generated. This model weight file is loaded via a Python script, and a noisy speech (.wav) file for verification is imported to perform preliminary verification and evaluation of the model's denoising effect, obtaining the denoising effect evaluation result. If the denoising effect evaluation result is unsatisfactory, the process returns to the dataset construction step to make targeted adjustments to the data components and proportions of the model training dataset. Based on the adjusted dataset, model training and effect verification are carried out again, iterating until the denoising effect meets the standard. If the denoising effect evaluation result is satisfactory, the subsequent model format conversion and deployment operations are performed based on the current satisfactory model weight file.
[0037] In some optional embodiments, the process of converting and encapsulating the DTLN denoising model includes: converting a first-format model weight file of the DTLN denoising model into a second-format model weight file, wherein the first-format model weight file represents a model weight file corresponding to the TensorFlow architecture, and the second-format model weight file represents a model weight file in ONNX format; converting the second-format model weight file into a third-format model weight file using a preset conversion tool, wherein the third-format model weight file represents a model weight file in NCNN format; and encapsulating the DTLN denoising model with the third-format model weight file into functional blocks to obtain a DTLN denoising algorithm library and an API interface for calling the DTLN denoising algorithm library.
[0038] Specifically, the trained and evaluated DTLN denoising model is output as a first-format model weight file, which is a .h5 format Keras model weight file generated based on the TensorFlow deep learning architecture. This first-format model weight file is then converted across frameworks to obtain a second-format model weight file, which is a general intermediate ONNX format model file. Using a pre-defined PNNX format conversion tool, the ONNX format second-format model weight file is further converted into a third-format model weight file suitable for Android terminal CPU inference. This third-format model weight file is a lightweight mobile inference framework NCNN format model file, containing a .param model parameter file and a .bin model weight binary file.
[0039] Based on the third-format model weight file of NCNN, the DTLN denoising model is encapsulated into functional blocks using C / C++. The libncnn inference library and libfftw3f fast Fourier transform library are integrated to realize model loading, audio data block processing, denoising inference calculation, and automatic release of runtime resources. After encapsulation, a DTLN denoising algorithm library that can run directly on the Android system is generated, and a standardized, directly callable API interface is provided for the implementation and integration of denoising functions in the audio hardware abstraction layer.
[0040] Reference Figure 2 The DTLN noise reduction algorithm only supports fixed-format audio data input and fixed-size single-processing data blocks. The fixed audio format is 16000Hz sampling rate, single channel, 16-bit bit depth, and the single-processing data block size is 256 bytes. To enable the DTLN noise reduction algorithm to adapt to and be compatible with large blocks of audio input data transmitted in the Android audio hardware abstraction layer, data adaptation logic needs to be added to the algorithm processing flow. The input audio data buffer is divided into blocks and cached according to a fixed size of 256 bytes, and noise reduction processing is completed in multiple batches to avoid processing abnormalities and runtime errors caused by mismatches in data input format or length.
[0041] After completing the model deployment and algorithm encapsulation implementation, the corresponding Android solution's native development kit version (Android NDK) is used for compilation and construction to generate the encapsulated noise reduction algorithm library file and its accompanying header file. This can be directly provided to multiple different Android device solutions for integration and application, improving solution reusability and integration efficiency.
[0042] In some optional embodiments, the target audio data is obtained by calling the DTLN noise reduction model through the audio hardware abstraction layer to perform real-time noise reduction processing on the audio acquisition data. This includes: when the input stream of the audio hardware abstraction layer is open, comparing the first audio format of the current recording with a preset second audio format to obtain a comparison result; when the comparison result indicates that the first audio format and the second audio format are the same, initializing the API interface; and when the recording data is read by the input stream reading interface of the audio hardware abstraction layer, calling the initialized API interface to perform real-time noise reduction processing on the recording data to obtain the target audio data.
[0043] Specifically, when the recording input stream of the native interface of the audio hardware abstraction layer is opened, the first audio format used by the current recording stream is obtained, and then compared with a preset second audio format (determining whether the current recording stream's sampling rate, bit depth, and number of channels are a fixed audio acquisition format of 16000Hz sampling rate, 16-bit bit depth, and single channel) to obtain the corresponding comparison result; where the second audio format is a fixed audio format adapted to the running requirements of the DTLN noise reduction model. If the comparison result indicates that the first audio format and the second audio format are consistent, the initialization operation of the DTLN noise reduction algorithm library is executed, completing the loading and startup of the corresponding API interface. When the recording input stream reading interface of the audio hardware abstraction layer successfully reads the recording data acquired by the device, the initialized API interface is called to perform real-time AI noise reduction processing on the recording data, and after noise reduction algorithm processing, clear and usable target audio data is finally obtained.
[0044] In some optional embodiments, after the API interface is initialized and performs real-time noise reduction processing on the recording data to obtain the target audio data, the method further includes: destructing and releasing the API interface's call resources when the input stream of the audio hardware abstraction layer is closed.
[0045] Specifically, when the recording input stream of the audio hardware abstraction layer is detected to be closed, all call resources such as model instances, memory buffers, and inference resources occupied by the DTLN noise reduction algorithm library and API interface during operation are uniformly destructed and released to avoid system resource leakage and ensure the stable operation of the audio hardware abstraction layer.
[0046] In some optional embodiments, the step of calling the DTLN noise reduction model through the audio hardware abstraction layer to perform real-time noise reduction processing on the audio acquisition data to obtain the target audio data further includes: when the input stream of the audio hardware abstraction layer is open, comparing the first audio format of the current stream recording with a preset second audio format to obtain a comparison result; if the comparison result indicates that the first audio format and the second audio format are the same, obtaining the real-time application scenario of the recording data; if the real-time application scenario is the same as the preset scenario, calling the initialized API interface to perform real-time noise reduction processing on the recording data to obtain the target audio data; if the comparison result indicates that the first audio format and the second audio format are different, or the real-time application scenario is different from the preset scenario, not calling the initialized API interface.
[0047] Specifically, when the recording input stream of the native interface of the audio hardware abstraction layer is opened, the first audio format used by the current recording stream is obtained, and the first audio format is matched and compared with the preset second audio format (determining whether the sampling rate, bit depth, and number of channels of the current recording stream are a fixed audio acquisition format with a sampling rate of 16000Hz, a bit depth of 16bit, and a single channel) to obtain the corresponding comparison result; where the second audio format is a fixed audio format adapted to the running requirements of the DTLN noise reduction model. If the comparison result indicates that the first audio format and the second audio format are consistent, the real-time application scenario of the current recording data is obtained, and the scenario type of the current recording service is determined according to the system running status or upper-layer application configuration; the identified real-time application scenario is matched and compared with the preset scenario that supports AI noise reduction processing; if the real-time application scenario is consistent with the preset scenario, the initialization operation of the DTLN noise reduction algorithm library is executed to complete the loading and startup of the corresponding API interface. If the audio hardware abstraction layer's recording input stream reading interface successfully reads the recording data collected by the device, it calls the initialized API interface to perform real-time AI noise reduction processing on the recording data. After noise reduction algorithm processing, clear and usable target audio data is finally obtained. If the comparison results indicate that the first audio format and the second audio format are different, or if the real-time application scenario is different from the preset scenario, it indicates that AI noise reduction processing is not required, the initialized API interface is not called, and the recording data is read and filled normally.
[0048] When processing recording input data using the DTLN noise reduction algorithm via the `audio_stream_in->read` interface (using `dtln_ns_process()`), to simultaneously meet the DTLN model algorithm's constraints on input data and the adaptability of the Android system's recording data stream, the parameters of the recording input stream in the Audio HAL (Audio Hardware Abstraction Layer) need to be configured appropriately. The product of the input stream's `period_size` and `period_count` is set to 1920 × 2, allowing the `audio_stream_in->read` interface to read 1920 frames, totaling 3840 bytes of audio data, in a single session. The DTLN model algorithm requires a single processing size of 128 frames, corresponding to a data length of 256 bytes. This stream parameter configuration enables the algorithm library to segment, cache, and batch process large blocks of audio data, ensuring the model's normal operation. Testing shows that a single audio noise reduction process takes approximately 30 milliseconds, demonstrating good real-time performance and application effects in typical use cases such as local recording and voice calls.
[0049] The above adaptation logic and calling process are all based on the standard API of Android's native HAL layer. They are compatible with and adaptable to most standard audio hardware abstraction layer implementation schemes. The interface calls are simple and the algorithm is easy to port. This can significantly shorten the development cycle on the manufacturer's side and facilitate the rapid integration, deployment and effect verification of the noise reduction function.
[0050] Reference Figure 3 Noise reduction tests comparing the DTLN-based noise reduction model (AI noise reduction) of this invention with traditional noise reduction methods show that the DTLN noise reduction proposed in this application has excellent suppression effects on various complex environmental noises. Furthermore, the noise reduction performance can be continuously improved through optimization of the training dataset, model structure tuning, and training iterations. Actual measurements and statistics on process resources such as CPU usage, bandwidth usage, and memory consumption on Android devices running AI noise reduction confirm that the overall resource consumption is at a low level, the system runs stably and smoothly, and it has good practicality.
[0051] Implementing the embodiments of the present invention has the following beneficial effects: The embodiments of the present invention provide an audio noise reduction method for Android devices, comprising: training a DTLN noise reduction model based on an open-source dataset; converting the format of the DTLN noise reduction model and encapsulating it, and integrating the encapsulated DTLN noise reduction model into the audio hardware abstraction layer of the Android device; and using the audio hardware abstraction layer to call the DTLN noise reduction model to perform real-time noise reduction processing on the audio acquisition data to obtain the target audio data. By integrating and calling the DTLN noise reduction model in the audio hardware abstraction layer for noise reduction processing, the method is simple to use, easy to port, and can adapt to the diverse and complex scenarios of Android devices.
[0052] Compared to existing noise reduction models that are based on application-side integration, this invention integrates the noise reduction model into the Audio HAL layer (audio hardware abstraction layer), enabling system-level audio noise reduction capabilities.
[0053] By designing an audio processing path that adapts to the input requirements of the DTLN noise reduction model, and through block caching and multi-batch calling mechanisms, the processing time per cycle is small, which can meet the real-time requirements of noise reduction.
[0054] By comparing real-time application scenarios with preset scenarios to select noise reduction processing, AI noise reduction can be dynamically controlled according to the application scenario (local recording, voice calls, etc.), resulting in good noise reduction effect.
[0055] By training the model with noise data from multiple scenarios, it is possible to suppress noise in a variety of complex noise scenarios. Furthermore, the noise reduction effect can be continuously optimized and more types of noise can be suppressed by iteratively training the noise reduction model on the training set.
[0056] The DTLN noise reduction model is integrated into the Audio HAL layer of Android devices. The integration only relies on the relevant implementation libraries, header files, and model parameter files encapsulated by the vendor, resulting in a small amount of code. This makes it easy for solution providers to port, integrate, and apply the model. It is seamlessly compatible with the existing native Android audio framework, requiring only modifications to the Audio HAL layer of the Android device for adaptation, resulting in good system stability. It can support multiple CPU architectures without relying on additional hardware and can be widely used in Android systems and devices.
[0057] like Figure 4 As shown, Figure 4 A structural block diagram of a controller 1000 according to an embodiment of this application is shown. The components of the controller 1000 include, but are not limited to, a memory 1200 and a processor 1100. The processor 1100 is connected to the memory 1200 via a bus, and the memory 1200 is used to store data.
[0058] The controller 1000 also includes an access device that enables the controller 1000 to communicate via one or more networks. Examples of such networks include a Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the Internet. The access device may include one or more of any type of wired or wireless network interface (e.g., a Network Interface Card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) interface, a Wi-MAX interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.
[0059] The controller 1000 can be any type of stationary or mobile electronic device, including mobile computers or mobile electronic devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable electronic devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary electronic devices such as desktop computers or PCs. The controller 1000 can also be a mobile or stationary server.
[0060] The processor 1100 is used to execute computer-executable instructions for the secure boot method.
[0061] The above is an illustrative scheme of a controller according to this embodiment. It should be noted that the technical solution of this controller and the technical solution of the above-described safe startup method belong to the same concept. For details not described in detail in the technical solution of the controller, please refer to the description of the technical solution of the above-described safe startup method.
[0062] According to an embodiment of this application, an audio noise reduction device for an Android device is also provided. The secure boot device includes a controller 1000. It should be noted that the technical solution of this secure boot device belongs to the same concept as the technical solution of the aforementioned secure boot method. Details not described in detail in the technical solution of the secure boot device can be found in the description of the technical solution of the aforementioned secure boot method.
[0063] Furthermore, this application also discloses a computer program product or computer program stored in a computer-readable storage medium. A processor of a computer device can read the computer program from the computer-readable storage medium, execute the computer program, and cause the computer device to perform the described method. Similarly, the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.
[0064] It is understood that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, or suitable combinations thereof. Some or all of the physical components can be implemented as processors, such as central processing units, digital information processors, or microprocessors executing software, or as hardware, or as integrated circuits, such as application-specific integrated circuits. Such software can be distributed on computer-readable media, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data information such as carrier waves or other transmission mechanisms, and may include any information delivery medium.
[0065] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.
Claims
1. An Android device audio noise reduction method, characterized in that, include: The DTLN denoising model was trained based on an open-source dataset. The DTLN noise reduction model is converted into a format and then encapsulated. The encapsulated DTLN noise reduction model is then integrated into the audio hardware abstraction layer of the Android device. The target audio data is obtained by calling the DTLN noise reduction model through the audio hardware abstraction layer to perform real-time noise reduction on the audio acquisition data.
2. The method according to claim 1, characterized in that, The DTLN denoising model trained on the open-source dataset includes: Obtain open-source audio data from the aforementioned open-source dataset; The model training dataset is obtained by filtering from the open-source audio data according to the preset noise reduction requirements. The model training data includes a target type of human voice dataset and a target scene noise dataset. The DTLN denoising model is trained using the model training dataset and a preset DTLN model training environment.
3. The method according to claim 2, characterized in that, The step of training the DTLN denoising model based on the model training dataset and a preset DTLN model training environment includes: The model training dataset is divided into a training set and a validation set; Input the training set and the validation set into the DTLN model training environment; Configure the model training parameters of the DTLN model training environment, and obtain the initial DTLN model by training the DTLN model based on the training set; The initial DTLN model was evaluated for noise reduction based on the validation set to obtain the evaluation results. If the evaluation results indicate that the noise reduction is satisfactory, the initial DTLN model will be configured as the DTLN noise reduction model. If the evaluation results indicate that the noise reduction is unsatisfactory, the data components and proportions of the model training dataset are adjusted, and the initial DTLN model is trained and updated through the DTLN model training environment.
4. The method according to claim 1, characterized in that, The process of converting and encapsulating the DTLN denoising model includes: The first format model weight file of the DTLN denoising model is converted into a second format model weight file. The first format model weight file represents the model weight file corresponding to the TensorFlow architecture, and the second format model weight file represents the ONNX format model weight file. The second format model weight file is converted into a third format model weight file using a preset conversion tool. The third format model weight file represents an NCNN format model weight file. After encapsulating the DTLN denoising model with the third format model weight file into functional blocks, a DTLN denoising algorithm library and an API interface for calling the DTLN denoising algorithm library are obtained.
5. The method according to claim 4, characterized in that, The process of obtaining target audio data by calling the DTLN noise reduction model through the audio hardware abstraction layer to perform real-time noise reduction on the audio acquisition data includes: When the input stream of the audio hardware abstraction layer is open, the first audio format of the current recording is compared with the preset second audio format to obtain the comparison result; If the comparison result indicates that the first audio format and the second audio format are the same, the API interface is initialized; When the audio hardware abstraction layer reads the recording data through its input stream reading interface, it calls the initialized API interface to perform real-time noise reduction processing on the recording data to obtain the target audio data.
6. The method according to claim 5, characterized in that, After the API interface is initialized and real-time noise reduction is performed on the recording data to obtain the target audio data, the method further includes: When the input stream of the audio hardware abstraction layer is closed, the calling resources of the API interface are destructed and released.
7. The method according to claim 4, characterized in that, The process of obtaining target audio data by calling the DTLN noise reduction model through the audio hardware abstraction layer to perform real-time noise reduction processing on the audio acquisition data further includes: The process of obtaining target audio data by calling the DTLN noise reduction model through the audio hardware abstraction layer to perform real-time noise reduction processing on the audio acquisition data further includes: When the input stream of the audio hardware abstraction layer is open, the first audio format of the current recording is compared with the preset second audio format to obtain the comparison result; If the comparison result indicates that the first audio format and the second audio format are the same, the real-time application scenario of the recording data is obtained. If the real-time application scenario is the same as the preset scenario, the initialized API interface is called to perform real-time noise reduction processing on the recording data to obtain the target audio data. If the comparison results indicate that the first audio format and the second audio format are different, or if the real-time application scenario is different from the preset scenario, the initialized API interface will not be called.
8. A controller, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method of any one of claims 1-7.
9. An audio noise reduction device for an Android device, comprising the controller of claim 8.
10. A computer-readable storage medium storing a processor-executable program, characterized in that, The processor-executable program, when executed by the processor, is used to perform the method as described in any one of claims 1-7.