Data acquisition method and device of audio processor

Abstract

The invention relates to the technical field of audio processing, and discloses a data acquisition method and device for an audio processor, and the method comprises the steps: obtaining initial audio signal source data and initial environment noise data, and carrying out the calculation to obtain an initial audio signal feature and an initial environment noise feature; constructing an initial compensation model according to the initial audio signal features and the initial environmental noise features, and obtaining an initial compensation coefficient; finely adjusting the audio input equipment according to the initial compensation coefficient, and collecting initial audio output data of the audio input equipment as a feedback error characteristic value; updating the initial compensation model according to the feedback error characteristic value to obtain an optimized compensation model; and inputting the updated audio signal data into the optimization compensation model to obtain a target audio output result. The method can calculate the compensation value in real time while processing the audio data, and ensures that the output audio signal of the audio processor is less affected by environmental noise.

Description

Technical Field

[0001] The present invention relates to the technical field of audio processing, in particular to a data acquisition method and device for an audio processor. Background Art

[0002] At present, with the development of science and technology, the field of audio processing has entered the digital age. As an important device for audio processing, audio processors play a huge role in the digital age. Digital audio processing improves the efficiency and quality of data processing, and also makes the storage, transmission and editing of audio content more convenient and flexible. The traditional audio processor data acquisition method mainly uses front-end acquisition equipment for data acquisition. The collected data is not accurate and is easily affected by environmental noise.

[0003] In one prior art, a noise elimination method based on a convolutional neural network is disclosed, which includes: obtaining audio data to be processed; establishing a first fully connected layer, a convolutional layer, a second fully connected layer and a first loss function; establishing a denoising model; the audio data to be processed is processed by the denoising model to generate a denoising result, and the denoising result is used to eliminate the noise in the audio data to be processed. By establishing a denoising model and inputting the audio data to be processed into the denoising model for processing, the noise in the audio data to be processed is eliminated.

[0004] However, in the prior art, audio data is processed only through a denoising model. Under different noise conditions, for example, when the noise fluctuates greatly, it is difficult to maintain a stable denoising performance, resulting in low data collection accuracy. Summary of the invention

[0005] The present invention provides a data acquisition method and device for an audio processor to solve the problem that it is difficult to maintain stable denoising performance when the noise fluctuation is large.

[0006] In a first aspect, in order to solve the above technical problem, the present invention provides a data acquisition method of an audio processor, comprising:

[0007] After the audio input device is turned off, initial audio signal source data and initial environmental noise data are obtained;

[0008] Performing feature calculation on the initial audio signal source data and the initial ambient noise data to obtain initial audio signal features and initial ambient noise features;

[0009] Constructing and training an initial compensation model according to the initial audio signal characteristics and the initial ambient noise characteristics;

[0010] Inputting preset audio input data into the initial compensation model, and outputting an initial compensation coefficient;

[0011] After the audio input device is turned on, obtaining updated audio signal data and updated environmental noise data;

[0012] Fine-tuning the audio input device according to the initial compensation coefficient, and collecting initial audio output data of the audio input device as a feedback error characteristic value;

[0013] Determining an update coefficient according to the feedback error characteristic value, and updating the initial compensation model according to the update coefficient, the updated audio signal data and the updated ambient noise data to obtain an optimized compensation model;

[0014] Inputting the updated audio signal data into the optimized compensation model and outputting a target compensation coefficient;

[0015] The audio input device is fine-tuned according to the target compensation coefficient, and audio output data of the fine-tuned audio input device is collected as a target audio output result.

[0016] In an optional implementation, after the audio input device is turned off, obtaining initial audio signal source data and initial ambient noise data includes:

[0017] Collecting environmental noise data through a digital signal processor to obtain M environmental noise sampling values, where M is a positive integer greater than 1;

[0018] Audio signal source data is collected through an audio signal source device to obtain k audio signal source sampling values, where k is a positive integer greater than 1.

[0019] In an optional implementation, performing feature calculation on the initial audio signal source data and the initial ambient noise data to obtain initial audio signal features and initial ambient noise features includes:

[0020] The initial audio signal features include the audio signal source sampling mean and the audio signal source sampling variance; the initial environmental noise features include the environmental noise sampling mean and the environmental noise sampling variance;

[0021] Preprocessing the initial audio signal source data and the initial environmental noise data to obtain preprocessed data;

[0022] The preprocessed data is calculated using the following mean and variance formulas:

[0023] Environmental noise sampling mean μ n :

[0024]

[0025] where n irepresents the i-th environmental noise sampling value, M is the number of environmental noise sampling values;

[0026] Environmental noise sampling variance

[0027]

[0028] where μ n is the mean value of the environmental noise sampling value;

[0029] Audio signal source sampling mean μ a :

[0030]

[0031] where a p represents the pth audio signal source sampling value, k is the number of audio signal source sampling values;

[0032] Audio signal source sampling variance

[0033]

[0034] where μ a is the audio signal source sampling mean.

[0035] In an optional implementation, constructing and training an initial compensation model according to the initial audio signal feature and the initial ambient noise feature includes:

[0036] Build an initial machine learning model;

[0037] Performing a preprocessing operation on the initial audio signal feature and the initial environmental noise feature to obtain preprocessed data;

[0038] Performing a data set division operation on the preprocessed data to obtain a training set, a validation set, and a test set;

[0039] The initial machine learning model is trained according to the training set, the validation set and the test set, and the training is determined to be completed when the number of training times reaches a preset maximum number, thereby obtaining an initial compensation model.

[0040] In an optional implementation, the step of fine-tuning the audio input device according to the initial compensation coefficient and collecting initial audio output data of the audio input device as a feedback error characteristic value includes:

[0041] Sending parameter adjustment instructions through the audio input device adjustment interface to adjust parameters, including sampling frequency, sampling amplitude, adjustment parameters and adjustment amplitude;

[0042] The adjustment amplitude of the adjustment interface is calculated by the amplitude adjustment formula:

[0043] A=b(x mean -x adj )

[0044] Where A represents the adjustment range of the audio input device, x mean represents the mean value of the audio input device fine-tuning, x adj represents the amplitude parameter of the audio input device fine-tuning, and b represents the adjustment ratio of the audio input device;

[0045] The audio input device interface collects the initial compensation coefficient as an adjustment parameter;

[0046] The audio input device performs fine adjustment based on the adjustment parameter and the adjustment amplitude;

[0047] The fine-tuned audio input device is sampled, m sampling values ​​are collected, features are extracted for the m sampling values, the weight difference between the pth sampling value and the sampling value is calculated, the weights are accumulated, and the parameter W of the fine-tuning of the audio input device is calculated by the following formula:

[0048]

[0049] Where W is the parameter of audio input device fine-tuning, m is the number of sample values ​​of audio input device fine-tuning, and w f Indicates the sample value of the fine-tuning of the f-th audio input device, w f+1 Indicates the sample value of the fine-tuning of the f+1th audio input device, (w f -w f+1 ) represents the weight difference of the sample value fine-tuned by the audio input device;

[0050] The parameter W fine-tuned by the audio input device is used as the initial audio output data, and the initial audio output data is used as the feedback error characteristic value.

[0051] In an optional implementation, determining an update coefficient according to the feedback error characteristic value includes:

[0052] The update coefficient α is calculated by the following formula:

[0053]

[0054] Wherein, w is the feedback error characteristic value, and m is the number of sampling values ​​of the audio input device fine-tuning.

[0055] In an optional implementation, the updating of the initial compensation model according to the update coefficient, the updated audio signal data and the updated ambient noise data to obtain an optimized compensation model includes:

[0056] Performing feature operations on the updated audio signal data and the updated environmental noise data to obtain a mean value, a variance of the updated environmental noise data, and a mean value of an audio input error feature;

[0057] The update coefficient is used as a reference value for update calculation to update the compensation model, wherein updating the compensation model includes updating the formula:

[0058]

[0059] Among them, μ' n Represents the average value of the environmental noise characteristic noise sampling value in the updated audio processor, Represents the variance of the ambient noise characteristic noise sample value in the updated audio processor, μ' e represents a mean value of an audio input error characteristic in an updated audio processor; and Respectively represent the updated mean value, variance and mean value of the audio input error feature of the environmental noise data; α is the update coefficient;

[0060] The update result will be used as the update value of the initial compensation model, and the update result will be replaced with the corresponding parameters of the initial compensation model to obtain the optimized compensation model.

[0061] In a second aspect, the present invention provides a data acquisition device for an audio processor, comprising:

[0062] A data acquisition module, used to obtain initial audio signal source data and initial environmental noise data after the audio input device is turned off;

[0063] A feature calculation module, used to perform feature calculation on the initial audio signal source data and the initial ambient noise data to obtain initial audio signal features and initial ambient noise features;

[0064] A model training module, used to construct and train an initial compensation model according to the initial audio signal characteristics and the initial ambient noise characteristics;

[0065] A compensation coefficient module, used for inputting preset audio input data into the initial compensation model and outputting an initial compensation coefficient;

[0066] A data updating module, used to obtain and update audio signal data and update environmental noise data after the audio input device is turned on;

[0067] A device fine-tuning module, used to fine-tune the audio input device according to the initial compensation coefficient, and collect initial audio output data of the audio input device as a feedback error characteristic value;

[0068] A model updating module, used to determine an update coefficient according to the feedback error characteristic value, and update the initial compensation model according to the update coefficient, the updated audio signal data and the updated ambient noise data to obtain an optimized compensation model;

[0069] A coefficient updating module, used for inputting the updated audio signal data into the optimized compensation model and outputting a target compensation coefficient;

[0070] The audio output module is used to fine-tune the audio input device according to the target compensation coefficient, and collect audio output data of the fine-tuned audio input device as a target audio output result.

[0071] In a third aspect, the present invention further provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the data acquisition method of the audio processor described in any one of the above when executing the computer program.

[0072] In a fourth aspect, the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the data acquisition methods for the audio processor described above.

[0073] Compared with the prior art, the present invention has the following beneficial effects:

[0074] The present invention provides a data acquisition method and device for an audio processor, comprising acquiring initial audio signal source data and initial environmental noise data, and calculating initial audio signal characteristics and initial environmental noise characteristics; constructing an initial compensation model according to the initial audio signal characteristics and initial environmental noise characteristics and obtaining an initial compensation coefficient; fine-tuning an audio input device according to the initial compensation coefficient, and acquiring initial audio output data of the audio input device as a feedback error characteristic value; updating the initial compensation model according to the feedback error characteristic value to obtain an optimized compensation model; and inputting the updated audio signal data into the optimized compensation model to obtain a target audio output result.

[0075] The method processes audio input data in real time by training and optimizing a compensation model using audio signal features and environmental noise features, and collects compensation coefficients to fine-tune an audio input device in real time, thereby achieving real-time calculation of compensation values ​​while processing audio data, ensuring that the output audio signal of the audio processor is less affected by environmental noise, and improving the applicability and stability of the audio processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1is a flowchart of a data acquisition method for an audio processor provided by a first embodiment of the present invention;

[0077] Figure 2 It is a structural schematic diagram of a data acquisition device of an audio processor provided in a second embodiment of the present invention. DETAILED DESCRIPTION

[0078] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0079] In one prior art, a noise elimination method based on a convolutional neural network is disclosed, which includes: obtaining audio data to be processed; establishing a first fully connected layer, a convolutional layer, a second fully connected layer and a first loss function; establishing a denoising model; the audio data to be processed is processed by the denoising model to generate a denoising result, and the denoising result is used to eliminate the noise in the audio data to be processed. By establishing a denoising model and inputting the audio data to be processed into the denoising model for processing, the noise in the audio data to be processed is eliminated.

[0080] However, in the prior art, audio data is processed only through a denoising model. Under different noise conditions, for example, when the noise fluctuates greatly, it is difficult to maintain a stable denoising performance, resulting in low data collection accuracy.

[0081] In order to solve the above technical problems, refer to Figure 1 The first embodiment of the present invention provides a data acquisition method for an audio processor, comprising the following steps:

[0082] S11, after turning off the audio input device, obtaining initial audio signal source data and initial environmental noise data;

[0083] S12, performing feature calculation on the initial audio signal source data and the initial ambient noise data to obtain initial audio signal features and initial ambient noise features;

[0084] S13, constructing and training an initial compensation model according to the initial audio signal characteristics and the initial ambient noise characteristics;

[0085] S14, inputting the preset audio input data into the initial compensation model, and outputting an initial compensation coefficient;

[0086] S15, after turning on the audio input device, obtaining updated audio signal data and updated environmental noise data;

[0087] S16, fine-tuning the audio input device according to the initial compensation coefficient, and collecting initial audio output data of the audio input device as a feedback error characteristic value;

[0088] S17, determining an update coefficient according to the feedback error characteristic value, and updating the initial compensation model according to the update coefficient, the updated audio signal data and the updated ambient noise data to obtain an optimized compensation model;

[0089] S18, inputting the updated audio signal data into the optimized compensation model, and outputting a target compensation coefficient;

[0090] S19, fine-tuning the audio input device according to the target compensation coefficient, and collecting audio output data of the fine-tuned audio input device as a target audio output result.

[0091] In step S11, after the audio input device is turned off, initial audio signal source data and initial ambient noise data are obtained, including:

[0092] After the audio input device is turned off, obtaining initial audio signal source data and initial ambient noise data includes:

[0093] Collecting environmental noise data through a digital signal processor to obtain M environmental noise sampling values, where M is a positive integer greater than 1;

[0094] Audio signal source data is collected through an audio signal source device to obtain k audio signal source sampling values, where k is a positive integer greater than 1.

[0095] It is worth noting that before turning off the audio input device, it is necessary to make preliminary adjustments to the audio input device and the environment, initialize various settings of the digital signal processor and the analog-to-digital signal converter, determine the input voltage amplitude range of the audio input device, and avoid the analog-to-digital signal converter being unable to correctly process and accurately convert due to excessive or too small input voltage. Then start the audio input device and the audio processor, wait for the audio input device and the audio processor to enter a stable operating state, and then turn off the audio input device. Stable operation and stable output state are conducive to data collection and analysis in the subsequent audio processing process, ensuring the accuracy and reliability of data collection. The acquisition of the initial audio signal source data and the initial environmental noise data requires several groups to make the data analysis results reliable. Among them, the initial audio signal source data includes but is not limited to the original signal of the audio processor; the acquisition of the initial environmental noise data includes: acquisition through the audio processor, acquisition through a removable device (such as a mobile phone, a tablet computer, etc.), and acquisition through a microphone array.

[0096] In step S12, feature calculation is performed on the initial audio signal source data and the initial ambient noise data to obtain initial audio signal features and initial ambient noise features.

[0097] It is worth noting that before performing the feature operation, the initial audio signal source data and the initial environmental noise data need to be preprocessed, including filtering and feature extraction; wherein filtering includes denoising, frequency selection and signal separation; feature extraction is time series extraction of the initial audio signal source data and the initial environmental noise data, including spectrum features, time domain features and statistical features;

[0098] The initial audio signal features include the audio signal source sampling mean and the audio signal source sampling variance; the initial environmental noise features include the environmental noise sampling mean and the environmental noise sampling variance;

[0099] Construct a mean formula and a variance formula, perform feature operations on the preprocessed data, and obtain the initial audio signal features and the initial ambient noise features; the formula is as follows

[0100] Environmental noise sampling mean μ n :

[0101]

[0102] where n i represents the i-th environmental noise sampling value, M is the number of environmental noise sampling values;

[0103] Environmental noise sampling variance

[0104]

[0105] where μ n is the mean value of the environmental noise sampling value;

[0106] Audio signal source sampling mean μ a :

[0107]

[0108] where a p represents the pth audio signal source sampling value, k is the number of audio signal source sampling values;

[0109] Audio signal source sampling variance

[0110]

[0111] where μ a is the sampling mean of the audio signal source;

[0112] In steps S13 and S14, an initial compensation model is constructed and trained according to the initial audio signal characteristics and the initial ambient noise characteristics, and the preset audio input data is input into the initial compensation model to output an initial compensation coefficient.

[0113] In one implementation, constructing and training an initial compensation model includes:

[0114] Build an initial machine learning model;

[0115] Performing a preprocessing operation on the initial audio signal feature and the initial environmental noise feature to obtain preprocessed data;

[0116] Performing a data set division operation on the preprocessed data to obtain a training set, a validation set, and a test set;

[0117] The initial machine learning model is trained according to the training set, the validation set and the test set, and the training is determined to be completed when the number of training times reaches a preset maximum number, thereby obtaining an initial compensation model.

[0118] It is worth noting that the machine learning model type is a reinforcement learning model. The preprocessing operation includes: outlier detection and processing, filling the outliers caused by missing values ​​or errors with the mean or median; the audio signal source characteristics and environmental noise characteristics that have been detected and processed by outliers are calculated and normalized, where the normalization formula is as follows:

[0119]

[0120] Wherein z is the value before normalization, z'' is the result after normalization, max(z) and min(z) are the maximum and minimum values ​​in the feature set, respectively. The training set, validation set and test set are composed of several groups of audio data collected previously after feature budgeting and data preprocessing. The initial compensation model training process includes: using the training set and validation set for iterative training, and when the number of training times is greater than or equal to the preset maximum number of training times, the training is determined to be completed; an indicator evaluation is performed on the overfitting or underfitting situation, and the next step is carried out according to the evaluation result, such as: when overfitting occurs, the model complexity can be reduced by reducing the number of model layers or the number of neurons, and the model complexity can be reduced by using L1 or L2 regularization; when underfitting occurs, the opposite method to overfitting is adopted; when there is no overfitting or underfitting, the training can be determined to be completed, and the initial compensation model is obtained.

[0121] It is worth noting that the initial compensation coefficient, that is, the initial compensation value, is calculated by inputting the preset audio input data into the trained initial compensation model.

[0122] In step S15, it is necessary to obtain updated audio signal data and updated environmental noise data after the audio input device is turned on.

[0123] Among them, the updated audio signal data and updated environmental noise data in this step are obtained through the audio input device instead of the audio processor, including: collecting through a removable device (such as a mobile phone, a tablet computer, etc.) and collecting through a microphone array.

[0124] In step S16, the audio input device is fine-tuned according to the initial compensation coefficient, and initial audio output data of the audio input device is collected as a feedback error characteristic value, including:

[0125] Sending parameter adjustment instructions through the audio input device adjustment interface to adjust parameters, including sampling frequency, sampling amplitude, adjustment parameters and adjustment amplitude;

[0126] The adjustment amplitude of the adjustment interface is calculated by the amplitude adjustment formula:

[0127] A=b(x mean -x adj )

[0128] Where A represents the adjustment range of the audio input device, x mean represents the mean value of the audio input device fine-tuning, x adj represents the amplitude parameter of the audio input device fine-tuning, and b represents the adjustment ratio of the audio input device;

[0129] The audio input device interface collects the initial compensation coefficient as an adjustment parameter;

[0130] The audio input device performs fine adjustment based on the adjustment parameter and the adjustment amplitude;

[0131] The fine-tuned audio input device is sampled, m sampling values ​​are collected, features are extracted for the m sampling values, the weight difference between the pth sampling value and the sampling value is calculated, the weight is accumulated, and the parameter W of the audio input device fine-tuning is calculated by the following formula:

[0132]

[0133] Where W is the parameter of audio input device fine-tuning, m is the number of sample values ​​of audio input device fine-tuning, and w f Indicates the sample value of the fine-tuning of the f-th audio input device, w f+1 Indicates the sample value of the fine-tuning of the f+1th audio input device, (w f -w f+1 ) represents the weight difference of the sample value fine-tuned by the audio input device;

[0134] Using the parameter W fine-tuned by the audio input device as the initial audio output data, and using the initial audio output data as the feedback error characteristic value;

[0135] The number m of sampling values ​​fine-tuned by the audio input device is the number of sampling values ​​of the audio output data.

[0136] It is worth noting that the sampling frequency refers to the number of sample points collected per second, with Hertz (Hz) as the unit. The higher the sampling frequency, the higher the time resolution of the audio signal and the larger the data volume; the sampling amplitude refers to the amplitude value of each sample, with bits as the unit, which determines the dynamic range and volume of the audio signal; the audio input device interface collects the initial compensation coefficient as an adjustment parameter; the audio input device performs fine-tuning based on the adjustment parameter and the adjustment amplitude; the parameter W fine-tuned by the audio input device is the initial audio output data, that is, the feedback error characteristic value; the number of sampling values ​​m fine-tuned by the audio input device is the number of sampling values ​​of the output data.

[0137] In step S17, it is necessary to determine an update coefficient according to the feedback error characteristic value, and update the initial compensation model according to the update coefficient, the updated audio signal data and the updated ambient noise data to obtain an optimized compensation model, including:

[0138] The update coefficient α is obtained by calculating the feedback error characteristic value using the following formula:

[0139]

[0140] Wherein W is the feedback error characteristic value, that is, the initial audio output data; m is the number of sampling values ​​of the output data.

[0141] Perform feature operations on the updated audio signal data and the updated ambient noise data to obtain the mean, variance and mean of the audio input error features of the updated ambient noise data; the method for using the feature operation is the same as step S12 and will not be repeated here.

[0142] The update coefficient is used as a reference value for update calculation to update the compensation model, wherein updating the compensation model includes updating the formula:

[0143]

[0144]

[0145] Among them, μ' n Represents the average value of the environmental noise characteristic noise sampling value in the updated audio processor, Represents the variance of the ambient noise characteristic noise sample value in the updated audio processor, μ'e represents a mean value of an audio input error characteristic in an updated audio processor; and Respectively represent the updated mean value, variance and mean value of the audio input error feature of the environmental noise data; α is the update coefficient;

[0146] The update result will be used as the update value of the initial compensation model, and the update result will be replaced with the corresponding parameters of the initial compensation model to obtain the optimized compensation model.

[0147] In step S18, the updated audio signal data is input into the optimization compensation model, and a target compensation coefficient is output.

[0148] It is worth noting that the target compensation coefficient, i.e., the target compensation value, is calculated by inputting the updated audio signal data into the optimized compensation model. This step is similar to step S14, except that the optimized compensation model is used in this step, so the target compensation coefficient can more accurately reflect the current environmental state, ensuring that the audio processor can maintain relatively stable performance under different environmental noise conditions, and at the same time, the fine-tuning of the audio input device in the next step will be more accurate and close to the actual situation.

[0149] In step S19, the audio input device needs to be fine-tuned according to the target compensation coefficient, and the audio output data of the fine-tuned audio input device is collected as the target audio output result.

[0150] It is worth noting that this step uses the compensation coefficient fine-tuning device and collects the fine-tuned audio output data, uses the target compensation coefficient that more accurately reflects the environmental status for final adjustment and optimization, and outputs the target audio output result. This reflects that the data acquisition method of the audio processor calculates the compensation value in real time while processing the audio data in real time, ensuring that the target audio output result is less affected by environmental noise and audio input equipment.

[0151] In order to facilitate the understanding of the present invention, some preferred embodiments of the present invention are further described below.

[0152] The following describes the working process of the present invention using a common scenario as an example. Figure 2 , especially Figure 1 Schematic diagram of the working scenario of the method.

[0153] In this embodiment, the data acquisition device of the audio processor is composed of an audio input device, an audio processor and an audio output device. The device is designed to process audio data in real time and calculate compensation values ​​in real time to ensure that the output audio signal of the audio processor is less affected by environmental noise.

[0154] The working process is as follows:

[0155] Step 1: The audio processor starts initialization, and after the state is stable, the audio input device is turned off to obtain the initial audio signal source data and the initial environmental noise data;

[0156] Step 2: performing feature calculation on the audio data to obtain initial audio signal features and initial ambient noise features;

[0157] Step 3: constructing and training an initial compensation model according to the initial audio signal characteristics and the initial ambient noise characteristics;

[0158] Step 4: inputting the preset audio input data stored in the memory or collected on site into the initial compensation model, outputting an initial compensation coefficient, and fine-tuning the audio input device according to the initial compensation coefficient;

[0159] Step 5: Open the audio input device to collect new audio data. The new audio data can be used as the initial audio output number of the feedback error characteristic value through the fine-tuned audio input device output;

[0160] Step 6: Collect updated audio data and environmental noise data, and update the initial compensation model in combination with the initial audio output data to obtain an optimized compensation model;

[0161] Step 7: Inputting the updated audio data into the optimized compensation model to obtain a target compensation coefficient;

[0162] Step eight: fine-tune the audio input device according to the target compensation coefficient, and collect audio output data of the fine-tune audio input device as the target audio output result.

[0163] Through the above steps, the data acquisition device of the audio processor can calculate the compensation value in real time while processing the audio data in real time, ensuring that the target audio output result is less affected by the environmental noise.

[0164] In summary, the present invention discloses a data acquisition method and device for an audio processor, including obtaining initial audio signal source data and initial environmental noise data, and calculating initial audio signal characteristics and initial environmental noise characteristics; constructing an initial compensation model according to the initial audio signal characteristics and initial environmental noise characteristics and obtaining an initial compensation coefficient; fine-tuning an audio input device according to the initial compensation coefficient, and collecting initial audio output data of the audio input device as a feedback error characteristic value; updating the initial compensation model according to the feedback error characteristic value to obtain an optimized compensation model; inputting the updated audio signal data into the optimized compensation model to obtain a target audio output result. The method processes audio input data in real time by using a compensation model trained and optimized by audio signal characteristics and environmental noise characteristics, and collects compensation coefficients to fine-tune the audio input device in real time, thereby realizing real-time calculation of compensation values ​​while processing audio data, ensuring that the output audio signal of the audio processor is less affected by environmental noise, and improving the applicability and stability of the audio processor.

[0165] Reference Figure 2 A second embodiment of the present invention provides a data acquisition device for an audio processor, comprising:

[0166] A data acquisition module, used to obtain initial audio signal source data and initial environmental noise data after the audio input device is turned off;

[0167] A feature calculation module, used to perform feature calculation on the initial audio signal source data and the initial ambient noise data to obtain initial audio signal features and initial ambient noise features;

[0168] A model training module, used to construct and train an initial compensation model according to the initial audio signal characteristics and the initial ambient noise characteristics;

[0169] A compensation coefficient module, used for inputting preset audio input data into the initial compensation model and outputting an initial compensation coefficient;

[0170] A data updating module, used to obtain and update audio signal data and update environmental noise data after the audio input device is turned on;

[0171] A device fine-tuning module, used to fine-tune the audio input device according to the initial compensation coefficient, and collect initial audio output data of the audio input device as a feedback error characteristic value;

[0172] A model updating module, used to determine an update coefficient according to the feedback error characteristic value, and update the initial compensation model according to the update coefficient, the updated audio signal data and the updated ambient noise data to obtain an optimized compensation model;

[0173] A coefficient updating module, used for inputting the updated audio signal data into the optimized compensation model and outputting a target compensation coefficient;

[0174] The audio output module is used to fine-tune the audio input device according to the target compensation coefficient, and collect audio output data of the fine-tuned audio input device as a target audio output result.

[0175] Preferably, the data acquisition module includes:

[0176] An environmental noise data acquisition unit is used to acquire environmental noise data through a digital signal processor to obtain M environmental noise sampling values, where M is a positive integer greater than 1;

[0177] The audio signal source data acquisition unit is used to acquire audio signal source data through an audio signal source device to obtain k audio signal source sampling values, where k is a positive integer greater than 1.

[0178] Preferably, the feature calculation module includes:

[0179] The initial audio signal features include the audio signal source sampling mean and the audio signal source sampling variance; the initial environmental noise features include the environmental noise sampling mean and the environmental noise sampling variance;

[0180] A preprocessing unit, used for preprocessing the initial audio signal source data and the initial environmental noise data to obtain preprocessed data;

[0181] The mean variance calculation unit is used to calculate the preprocessed data using the following mean formula and variance formula:

[0182] Environmental noise sampling mean μ n :

[0183]

[0184] where n i represents the i-th environmental noise sampling value, M is the number of environmental noise sampling values;

[0185] Environmental noise sampling variance

[0186]

[0187] where μ n is the mean value of the environmental noise sampling value;

[0188] Audio signal source sampling mean μ a :

[0189]

[0190] where a p represents the pth audio signal source sampling value, k is the number of audio signal source sampling values;

[0191] Audio signal source sampling variance

[0192]

[0193] where μ a is the audio signal source sampling mean.

[0194] Preferably, the model training module is specifically used for:

[0195] Build an initial machine learning model;

[0196] Performing a preprocessing operation on the initial audio signal feature and the initial environmental noise feature to obtain preprocessed data;

[0197] Performing a data set division operation on the preprocessed data to obtain a training set, a validation set, and a test set;

[0198] The initial machine learning model is trained according to the training set, the validation set and the test set, and the training is determined to be completed when the number of training times reaches a preset maximum number, thereby obtaining an initial compensation model.

[0199] Preferably, the device fine-tuning module is specifically used for:

[0200] Sending parameter adjustment instructions through the audio input device adjustment interface to adjust parameters, including sampling frequency, sampling amplitude, adjustment parameters and adjustment amplitude;

[0201] The adjustment amplitude of the adjustment interface is calculated by the amplitude adjustment formula:

[0202] A=b(x mean -x adj )

[0203] Where A represents the adjustment range of the audio input device, x mean represents the mean value of the audio input device fine-tuning, x adj represents the amplitude parameter of the audio input device fine-tuning, and b represents the adjustment ratio of the audio input device;

[0204] The audio input device interface collects the initial compensation coefficient as an adjustment parameter;

[0205] The audio input device performs fine adjustment based on the adjustment parameter and the adjustment amplitude;

[0206] The fine-tuned audio input device is sampled, m sampling values ​​are collected, features are extracted for the m sampling values, the weight difference between the pth sampling value and the sampling value is calculated, the weight is accumulated, and the parameter W of the audio input device fine-tuning is calculated by the following formula:

[0207]

[0208] Where W is the parameter of audio input device fine-tuning, m is the number of sample values ​​of audio input device fine-tuning, and w f Indicates the sample value of the fine-tuning of the f-th audio input device, w f+1 Indicates the sample value of the fine-tuning of the f+1th audio input device, (w f -w f+1 ) represents the weight difference of the sample value fine-tuned by the audio input device;

[0209] The parameter W fine-tuned by the audio input device is used as the initial audio output data, and the initial audio output data is used as the feedback error characteristic value.

[0210] Preferably, the model updating module is specifically used for:

[0211] The update coefficient α is calculated by the following formula:

[0212]

[0213] Wherein, W is the feedback error characteristic value, and m is the number of sampling values ​​of the audio input device fine-tuning.

[0214] Preferably, the model updating module is specifically used for:

[0215] Performing feature operations on the updated audio signal data and the updated environmental noise data to obtain a mean value, a variance of the updated environmental noise data, and a mean value of an audio input error feature;

[0216] The update coefficient is used as a reference value for update calculation to update the compensation model, wherein updating the compensation model includes updating the formula:

[0217]

[0218] Among them, μ' n Represents the average value of the environmental noise characteristic noise sampling value in the updated audio processor, Represents the variance of the ambient noise characteristic noise sample value in the updated audio processor, μ' e represents a mean value of an audio input error characteristic in an updated audio processor; and Respectively represent the updated mean value, variance and mean value of the audio input error feature of the environmental noise data; α is the update coefficient;

[0219] The update result will be used as the update value of the initial compensation model, and the update result will be replaced with the corresponding parameters of the initial compensation model to obtain the optimized compensation model.

[0220] It should be noted that the data acquisition device of an audio processor provided in an embodiment of the present invention is used to execute all the process steps of the data acquisition method of an audio processor in the above embodiment, and the working principles and beneficial effects of the two correspond one to one, so they will not be repeated.

[0221] The embodiment of the present invention further provides an electronic device. The electronic device includes: a processor, a memory, and a computer program stored in the memory and executable on the processor, such as a data acquisition program. When the processor executes the computer program, the steps of the data acquisition method embodiments of the above-mentioned audio processor are implemented, such as Figure 1 Alternatively, when the processor executes the computer program, the functions of the modules / units in the above-mentioned device embodiments are realized, such as the real-time audio processing and compensation module.

[0222] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program in the electronic device.

[0223] The electronic device may be a computing device such as a desktop computer, a notebook, a PDA, and a smart tablet. The electronic device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the above components are merely examples of electronic devices and do not constitute a limitation on the electronic device. The electronic device may include more or fewer components than the above components, or may combine certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0224] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the electronic device, and uses various interfaces and lines to connect various parts of the entire electronic device.

[0225] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the electronic device by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0226] Wherein, if the module / unit integrated in the electronic device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0227] It should be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art may understand and implement it without paying any creative effort.

[0228] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. A data acquisition method for an audio processor, characterized in that: Executed by the processor, including: After the audio input device is turned off, initial audio signal source data and initial environmental noise data are obtained; Performing feature calculation on the initial audio signal source data and the initial ambient noise data to obtain initial audio signal features and initial ambient noise features; Constructing and training an initial compensation model according to the initial audio signal characteristics and the initial ambient noise characteristics; Inputting preset audio input data into the initial compensation model, and outputting an initial compensation coefficient; After the audio input device is turned on, obtaining updated audio signal data and updated environmental noise data; Fine-tuning the audio input device according to the initial compensation coefficient, and collecting initial audio output data of the audio input device as a feedback error characteristic value; Determining an update coefficient according to the feedback error characteristic value, and updating the initial compensation model according to the update coefficient, the updated audio signal data and the updated ambient noise data to obtain an optimized compensation model; Inputting the updated audio signal data into the optimized compensation model and outputting a target compensation coefficient; The audio input device is fine-tuned according to the target compensation coefficient, and audio output data of the fine-tuned audio input device is collected as a target audio output result.

2. The data acquisition method of the audio processor according to claim 1, characterized in that: After the audio input device is turned off, obtaining initial audio signal source data and initial ambient noise data includes: Collecting environmental noise data through a digital signal processor to obtain M environmental noise sampling values, where M is a positive integer greater than 1; Audio signal source data is collected through an audio signal source device to obtain k audio signal source sampling values, where k is a positive integer greater than 1.

3. The data acquisition method of the audio processor according to claim 1, characterized in that: Performing feature calculation on the initial audio signal source data and the initial ambient noise data to obtain initial audio signal features and initial ambient noise features includes: The initial audio signal features include the audio signal source sampling mean and the audio signal source sampling variance; the initial environmental noise features include the environmental noise sampling mean and the environmental noise sampling variance; Preprocessing the initial audio signal source data and the initial environmental noise data to obtain preprocessed data; The preprocessed data is calculated using the following mean and variance formulas: Environmental noise sampling mean μ n : Where n i represents the i-th environmental noise sampling value, M is the number of environmental noise sampling values; Environmental noise sampling variance where μ n is the mean value of the environmental noise sampling value; Audio signal source sampling mean μ a : where a p represents the pth audio signal source sampling value, k is the number of audio signal source sampling values; Audio signal source sampling variance where μ a is the audio signal source sampling mean.

4. The data acquisition method of the audio processor according to claim 1, characterized in that: The constructing and training an initial compensation model according to the initial audio signal characteristics and the initial ambient noise characteristics comprises: Build an initial machine learning model; Performing a preprocessing operation on the initial audio signal feature and the initial environmental noise feature to obtain preprocessed data; Performing a data set division operation on the preprocessed data to obtain a training set, a validation set, and a test set; The initial machine learning model is trained according to the training set, the validation set and the test set, and the training is determined to be completed when the number of training times reaches a preset maximum number, thereby obtaining an initial compensation model.

5. The data acquisition method of the audio processor according to claim 1, characterized in that: The step of fine-tuning the audio input device according to the initial compensation coefficient and collecting initial audio output data of the audio input device as a feedback error characteristic value includes: Sending parameter adjustment instructions through the audio input device adjustment interface to adjust parameters, including sampling frequency, sampling amplitude, adjustment parameters and adjustment amplitude; The adjustment amplitude of the adjustment interface is calculated by the amplitude adjustment formula: A=b(x mean -x adj ) Where A represents the adjustment range of the audio input device, x mean represents the mean value of the audio input device fine-tuning, x adj represents the amplitude parameter of the audio input device fine-tuning, and b represents the adjustment ratio of the audio input device; The audio input device interface collects the initial compensation coefficient as an adjustment parameter; The audio input device performs fine adjustment based on the adjustment parameter and the adjustment amplitude; The fine-tuned audio input device is sampled, m sampling values ​​are collected, features are extracted for the m sampling values, the weight difference between the pth sampling value and the sampling value is calculated, the weights are accumulated, and the parameter W of the fine-tuning of the audio input device is calculated by the following formula: Where W is the parameter of audio input device fine-tuning, m is the number of sample values ​​of audio input device fine-tuning, and w f Indicates the sample value of the fine-tuning of the f-th audio input device, w f+1 Indicates the sample value of the fine-tuning of the f+1th audio input device, (w f -w f+1 ) represents the weight difference of the sample value fine-tuned by the audio input device; The parameter W fine-tuned by the audio input device is used as the initial audio output data, and the initial audio output data is used as the feedback error characteristic value.

6. The data acquisition method of the audio processor according to claim 5, characterized in that: The step of determining an update coefficient according to the feedback error characteristic value comprises: The update coefficient α is calculated by the following formula: Wherein, W is the feedback error characteristic value, and m is the number of sampling values ​​of the audio input device fine-tuning.

7. The data acquisition method of the audio processor according to claim 6, characterized in that: The updating of the initial compensation model according to the update coefficient, the updated audio signal data and the updated environmental noise data to obtain an optimized compensation model comprises: Performing feature operations on the updated audio signal data and the updated environmental noise data to obtain a mean value, a variance of the updated environmental noise data, and a mean value of an audio input error feature; The update coefficient is used as a reference value for update calculation to update the compensation model, wherein updating the compensation model includes updating the formula: Among them, μ' n Represents the average value of the environmental noise characteristic noise sampling value in the updated audio processor, Represents the variance of the ambient noise characteristic noise sample value in the updated audio processor, μ' e represents a mean value of an audio input error characteristic in an updated audio processor; and Respectively represent the updated mean value, variance and mean value of the audio input error feature of the environmental noise data; α is the update coefficient; The update result will be used as the update value of the initial compensation model, and the update result will be replaced with the corresponding parameters of the initial compensation model to obtain the optimized compensation model.

8. A data acquisition device for an audio processor, characterized in that: include: A data acquisition module, used to obtain initial audio signal source data and initial environmental noise data after the audio input device is turned off; A feature calculation module, used to perform feature calculation on the initial audio signal source data and the initial ambient noise data to obtain initial audio signal features and initial ambient noise features; A model training module, used to construct and train an initial compensation model according to the initial audio signal characteristics and the initial ambient noise characteristics; A compensation coefficient module, used for inputting preset audio input data into the initial compensation model and outputting an initial compensation coefficient; A data updating module, used to obtain and update audio signal data and update environmental noise data after the audio input device is turned on; A device fine-tuning module, used to fine-tune the audio input device according to the initial compensation coefficient, and collect initial audio output data of the audio input device as a feedback error characteristic value; A model updating module, used to determine an update coefficient according to the feedback error characteristic value, and update the initial compensation model according to the update coefficient, the updated audio signal data and the updated ambient noise data to obtain an optimized compensation model; A coefficient updating module, used for inputting the updated audio signal data into the optimized compensation model and outputting a target compensation coefficient; The audio output module is used to fine-tune the audio input device according to the target compensation coefficient, and collect audio output data of the fine-tuned audio input device as a target audio output result.

9. An electronic device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the data acquisition method of the audio processor according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the data acquisition method of the audio processor according to any one of claims 1 to 7.

Images