Electronic audio data processing method and device based on deep learning, and storage medium

Abstract

The invention discloses an electronic audio data processing method and device based on deep learning and a storage medium, and particularly relates to the technical field of data processing, and the method comprises the steps: obtaining an audio evaluation coefficient of each electronic audio device, and determining an abnormal audio device in a preset playing time period according to the audio evaluation coefficient; obtaining a frequency fluctuation diagram of the abnormal audio equipment, determining frequency type data of the abnormal audio equipment according to the frequency fluctuation diagram, and obtaining corresponding sound quality abnormal data according to the frequency type data; according to the method and the device, the frequency data of the audio frequency in the preset time range is acquired according to the audio frequency zone bit of the sound quality abnormal data, the audio frequency spectrum of the abnormal audio equipment is optimized based on the frequency data, the audio frequency spectrum is optimized according to the frequency data acquired according to the audio frequency zone bit, and an adjustment strategy is formulated in combination with different frequency types, so that the audio frequency spectrum is more reasonable.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and more specifically, to a method, device and storage medium for electronic audio data processing based on deep learning. Background Art

[0002] In large-scale events such as concerts, music concerts, and various multimedia application scenarios, a large number of electronic audio devices are often used. These devices may have problems such as abnormal sound quality when running for a long time or in complex environments. Traditional audio device detection methods are mostly based on simple parameter settings or threshold judgments. For complex and changeable audio signals, it is difficult to accurately identify whether there are abnormalities in the equipment. For example, during the equipment debugging stage before a large-scale concert, traditional methods may not be able to quickly and accurately determine which of the many electronic audio devices have potential risks of abnormal sound quality during different preset playback time periods, which may affect the audio quality of the performance.

[0003] At present, the prior art discloses a multi-source long-distance wireless speaker transmission control method and device through a patent application with publication number CN101087485A. The device has a microprocessor 1 in the wireless sound source connected to a memory 1, a common channel modulation / demodulation circuit, an audio channel modulation circuit and a sound source interface circuit respectively; a memory 2 in the wireless speaker is connected to a microprocessor 2; a digital / analog conversion module power amplifier and a speaker are connected in sequence, and the digital / analog conversion module is also connected to the microprocessor 2. The control method includes the sending of audio signals and the receiving and playing of audio signals. Although the above method can enable the wireless speaker to receive audio signals sent by different wireless sound sources without confusion, the above method cannot adapt well to the complex characteristics and changes of different types of audio.

[0004] To this end, the present invention provides an electronic audio data processing method, device and storage medium based on deep learning. Summary of the invention

[0005] In order to overcome the above-mentioned defects of the prior art, the present invention provides an electronic audio data processing method, device and storage medium based on deep learning to solve the problems raised in the above-mentioned background technology.

[0006] To achieve the above purpose, the present invention provides the following technical solutions: In a first aspect, the present invention provides an electronic audio data processing method based on deep learning, comprising: Step 1: obtaining an audio evaluation coefficient of each electronic audio device, and determining abnormal audio devices within a preset playback time period according to the audio evaluation coefficient; Step 2: Obtain a frequency fluctuation graph of the abnormal audio device, determine the frequency type data of the abnormal audio device according to the frequency fluctuation graph, and obtain corresponding sound quality abnormality data according to the frequency type data; Step 3: Collect frequency data of the audio frequency within a preset time range according to the audio frequency location of the abnormal sound quality data, and optimize the audio spectrum of the abnormal audio device based on the frequency data.

[0007] The generation logic of the audio evaluation coefficient is as follows: Acquire audio recording data of the electronic audio device; the audio recording data includes the total number of audios within a preset time period and the audio fluctuation coefficient of each audio; The total number of audios in the preset time period and the audio fluctuation coefficient of each audio are calculated to obtain the audio evaluation coefficient, and the calculation formula is: , is the audio evaluation coefficient, is the audio fluctuation coefficient of the i-th audio, It is the total number of audios in the preset time period.

[0008] Furthermore, the generation logic of the audio fluctuation coefficient is as follows: S101. Extracting the audio waveform data of the i-th audio from the audio file, and performing time-frequency analysis on it to generate a time series containing frequency components; S102. Normalize the extracted audio waveform data and scale it to a standardized interval ; S103. Segment the audio waveform data according to the preset time window parameters into K segments of sub-signals, and calculate the audio feature data of each segment of the signal, wherein the audio feature data includes the local fluctuation mean and the spectrum fluctuation mean; S104. Calculate the audio fluctuation coefficient based on the audio feature data; Furthermore, the calculation formula of the local fluctuation mean is: , where is the kth segment relative to the The energy fluctuation value of the segment, is the energy value of the kth audio segment, is the energy value of the k-1th audio segment; , where is the mean of all local fluctuations, K is the total number of segments, is the local fluctuation value of each segment; The calculation formula of the spectrum fluctuation mean is: , where is the mean value of spectrum fluctuation, K is the total number of segments, is the fluctuation value of each spectrum segment, is the main frequency amplitude of the kth spectrum, is the main frequency amplitude of the k-1th spectrum.

[0009] Furthermore, the main frequency amplitude of the kth spectrum is obtained, including: S201. Input a discrete frequency amplitude, and convert the discrete frequency amplitude from the time domain to the frequency domain through Fourier transform;

[0010] S202. Calculate the The amplitude of the frequency component;

[0011] S203. Get the main frequency amplitude: Traverse all frequency components and find the component with the largest amplitude; S204. Obtain the frequency corresponding to the main frequency amplitude; Identify abnormal audio devices within a preset playback time period, including: Extract audio evaluation standard coefficients of normal audio equipment; Comparing the audio evaluation standard coefficient with the audio evaluation coefficient, if the audio evaluation coefficient is greater than or equal to the audio evaluation standard coefficient, determining that the corresponding electronic audio device is a normal electronic audio device; If the audio evaluation coefficient is less than the audio evaluation standard coefficient, the corresponding electronic audio device is determined to be an abnormal audio device; According to the preset relationship between the frequency type data and the abnormal sound quality data, the abnormal sound quality data corresponding to the frequency type data is determined, and the abnormal sound quality data covers multiple abnormal sound quality reasons and the standard audio fluctuation range associated with each reason.

[0012] Furthermore, a frequency fluctuation diagram of the abnormal audio device is obtained, including: S301: Obtain frequency data of an abnormal audio device; construct a frequency fluctuation graph with time in the frequency data as the horizontal axis and frequency value in the frequency data as the vertical axis; S302: dividing the frequency fluctuation graph into equal parts according to H audio rhythms to obtain a frequency waveform set, wherein the frequency waveform set includes G frequency sub-waveforms, where G is an integer greater than zero; S303: extracting the g-th frequency sub-waveform in the frequency waveform set, where g is an integer greater than zero and the initial value of g is 1; S304: retrieve the standard frequency waveform, calculate the difference between the frequency sub-waveform and the standard frequency waveform, and if the difference between the frequency sub-waveform and the standard frequency waveform is less than the preset frequency difference threshold, transfer to S305; if the difference between the frequency sub-waveform and the standard frequency waveform is greater than or equal to the preset frequency difference threshold, mark the frequency sub-waveform as an abnormal frequency waveform, and jump to S305; S305: Set g=g+1 and jump back to S303; S306: Repeat S303 to S305 until g=G, then end the cycle and obtain multiple abnormal frequency waveforms; S307: extracting the degree of distinction corresponding to each abnormal frequency waveform, and performing Fourier transform on the abnormal frequency waveform with the greatest degree of distinction to obtain a frequency fluctuation graph.

[0013] Further, determining the frequency type data of the abnormal audio device includes: Obtain the frequency fluctuation and audio frequency position of abnormal audio equipment; Inputting the frequency fluctuation and the audio frequency position into a frequency type recognition model to determine the frequency type data of the abnormal audio device; the frequency type data includes a frequent frequency change type or a stable frequency change type; The generation logic of the frequency type recognition model is as follows: obtaining the second historical sample data of frequency fluctuations and audio frequency positions, and labeling the corresponding frequency type data for each sample, dividing the sample data into a training set and a test set, using a deep neural network classification model, using the frequency fluctuation map and audio frequency position of the training set as input, and the frequency type data label as output, using the cross entropy loss function to optimize the model parameters, using the test set for testing, and screening the model whose test accuracy reaches a preset value as the frequency type recognition model.

[0014] Further, the audio spectrum of the abnormal audio device is optimized based on the frequency data, including: s401: performing spectrum analysis on the audio spectrum to obtain frequency data of the audio spectrum within a preset time range, wherein the frequency data includes a frequency peak value and a frequency average value; s402: set the compression ratio of the audio peak value and the average value greater than the preset difference threshold to the range of 3:1-5:1; and adjust the level point threshold of the compressor to the level position corresponding to the frequency peak control; s403: determining a regulation strategy according to the frequency type data, if the frequency changes frequently, extending the start time to a first extension time threshold, and setting the release time to a first release time threshold; If the frequency changes smoothly, the start time is extended to the second extension time threshold, and the release time is set at the second release time threshold; s404: Play the adjusted abnormal audio device, and re-determine whether it is an abnormal audio device again based on the audio evaluation coefficient. If not, mark the abnormal audio device as a normal audio device to complete the audio processing; If yes, repeat s402-s404 until the abnormal audio device is marked as a normal audio device.

[0015] An electronic audio data processing device based on deep learning, used to implement an electronic audio data processing method based on deep learning, comprising: An abnormality analysis module, used to obtain an audio evaluation coefficient of each electronic audio device, and determine abnormal audio devices within a preset playback time period according to the audio evaluation coefficient; A data acquisition module, used to acquire audio adjustment data of the abnormal audio device, determine frequency type data of the abnormal audio device according to the audio adjustment data, and obtain corresponding sound quality abnormality data according to the frequency type data; The audio optimization module is used to collect frequency data of audio frequencies within a preset time range according to the audio frequency position of the abnormal sound quality data, and optimize the audio spectrum of the abnormal audio device based on the frequency data.

[0016] A computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute an electronic audio data processing method based on deep learning.

[0017] Technical effects and advantages of the present invention: The present invention is applied to the debugging of electronic audio equipment for large-scale concerts, and has significant beneficial effects. First, by accurately obtaining the audio evaluation coefficient of the electronic audio equipment, it is possible to efficiently identify abnormal audio equipment within the preset playback time period, provide a clear target for subsequent debugging, and avoid wasting energy on normal equipment. Secondly, by obtaining the audio adjustment data of the abnormal audio equipment and determining the frequency type and abnormal sound quality data, the sound quality problem can be solved in a targeted manner and the audio quality can be improved. Furthermore, the audio spectrum is optimized by collecting frequency data according to the audio frequency position, and an adjustment strategy is formulated in combination with different frequency types to make the audio spectrum more reasonable. Finally, through multiple adjustments and judgments, it is ensured that the abnormal audio equipment returns to normal and the audio effect of the concert is guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a flow chart of the electronic audio data processing method based on deep learning of Example 1; Figure 2 This is a schematic diagram of the structure of an electronic audio data processing device based on deep learning in Example 2. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] Example 1 like Figure 1As shown, the present application discloses a method for processing electronic audio data based on deep learning, the method comprising: Step 1: obtaining an audio evaluation coefficient of each electronic audio device, and determining abnormal audio devices within a preset playback time period according to the audio evaluation coefficient; It should be additionally explained here that the present invention is applied to the electronic audio equipment debugging process before a large-scale concert. There are D electronic equipment audios in the large-scale concert, where D is an integer greater than zero, and each electronic audio device performs audio playback within its own preset playback time period.

[0021] In a specific embodiment, obtaining the audio evaluation coefficient of each electronic audio device includes: Acquire audio recording data of the electronic audio device; the audio recording data includes the total number of audios within a preset time period and the audio fluctuation coefficient of each audio; Among them, the generation logic of the audio fluctuation coefficient of the i-th audio is as follows: S101. Extract the audio waveform data of the i-th audio from the audio file, perform time-frequency analysis on it, and generate a time series containing audio frequency components ,in, is the frequency amplitude of the sampling point, r is the total number of sample points; For example: a 3-minute audio (sampling rate is 44.1kHz) (3 minutes of audio has approximately 44.1kHz x 180 seconds = 7,938,000 sampling points) S102. Normalize the extracted audio waveform data and scale it to a standardized interval , in order to eliminate the influence of absolute value on fluctuation calculation, the normalized formula is: ; In the formula, For time point After normalization, all frequency amplitudes are within the range Inside.

[0022] S103. Segment the audio waveform data according to the preset time window parameters and divide it into K sub-signals. Calculate the audio feature data for each signal segment. The audio feature data includes a local fluctuation mean and a spectrum fluctuation mean, which respectively characterize the time domain and frequency domain fluctuation characteristics of the audio signal.

[0023] The energy value is calculated as follows: , where is the energy value of the kth audio segment, R is the number of sampling points in each audio segment, is the frequency amplitude of the normalized audio signal at the rth sampling point.

[0024] It should be additionally noted here that the larger the frequency amplitude, the higher the energy value, the larger the time window, the more sampling points, and the higher the calculated total energy.

[0025] The calculation formula of the local fluctuation mean is: , where is the kth segment relative to the The energy fluctuation value of the segment, is the energy value of the kth audio segment, is the energy value of the k-1th audio segment; , where is the mean of all local fluctuations, K is the total number of segments, is the local fluctuation value of each segment.

[0026] The calculation formula of the spectrum fluctuation mean is: , where is the mean value of spectrum fluctuation, K is the total number of segments, is the fluctuation value of each spectrum segment, is the main frequency amplitude of the kth spectrum, is the main frequency amplitude of the k-1th spectrum.

[0027] S104. Calculation formula of audio fluctuation coefficient: ; In the formula, is the audio fluctuation coefficient of the i-th audio, and is a weight parameter used to balance the effects of energy fluctuation and spectrum fluctuation. is the local fluctuation mean, is the mean value of spectrum fluctuation; and Learned through training data or set empirically.

[0028] For example, the average energy fluctuation , average spectrum fluctuation , , , the audio fluctuation coefficient is 0.026.

[0029] Example description: Suppose we calculate the fluctuation coefficient of a rock audio and a classical audio: Rock audio: The spectrum and energy vary dramatically, and the final fluctuation coefficient may be 0.08.

[0030] Classical audio: The spectrum is smooth, the energy changes little, and the final fluctuation coefficient may be 0.02.

[0031] The higher the fluctuation coefficient, the more dramatic the audio dynamic changes.

[0032] By segmenting, extracting features and analyzing the spectrum of audio signals, the fluctuation characteristics of the audio are quantified, and the generated audio fluctuation coefficient is used in scenarios such as audio quality assessment, audio style analysis or device abnormality monitoring.

[0033] In a specific embodiment, obtaining the main frequency amplitude of the kth frequency spectrum includes: S201. Input discrete frequency amplitudes and convert the discrete frequency amplitudes from the time domain to the frequency domain through Fourier transform: ; The audio signal frequency components, is the discrete frequency, ,in is the sampling rate, is the frequency amplitude of the input signal at the rth sampling point in the time domain, is the rth point of the discrete time series, is the number of sampling points in each audio segment, is a complex exponential function, which is used to decompose the time domain signal into frequency components. is an imaginary unit, is an angle representing the contribution of frequency p to discrete time r.

[0034] S202. Calculate the The amplitude of the frequency components: ; is the frequency domain signal The amplitude of the frequency component (complex number), is the real part, is the imaginary part, for The modulus of , indicating the amplitude.

[0035] S203. Obtain the main frequency amplitude, including: Traverse all frequency components and find the component with the largest amplitude: ; is the main frequency amplitude, that is, the largest amplitude in the spectrum, Find the maximum value among the magnitudes of all frequency components.

[0036] S204. Obtain the frequency corresponding to the main frequency amplitude, and the calculation formula is: ; is the frequency corresponding to the main frequency amplitude, Main frequency The corresponding frequency index; is the number of sampling points per second, indicating the sampling frequency of the audio signal. The actual frequency value calculated for the frequency index; The frequency index corresponding to the main frequency amplitude.

[0037] Taking a specific implementation scenario as an example, the generation logic of the audio evaluation coefficient is as follows: The total number of audios in the preset time period and the audio fluctuation coefficient of each audio are calculated to obtain the audio evaluation coefficient, and the calculation formula is: , is the audio evaluation coefficient, is the audio fluctuation coefficient of the i-th audio, It is the total number of audios in the preset time period.

[0038] In a specific implementation, determining abnormal audio devices within a preset playback time period includes: Extract audio evaluation standard coefficients of normal audio equipment; It should be additionally explained here that: the calculation process of the audio evaluation standard coefficient is the same as the generation process of the above-mentioned audio evaluation coefficient. Specifically, the audio evaluation standard coefficient is obtained by a technician in this field when two normal audio devices are in a historical normal working stage, respectively, multiple audio waveform data are obtained, that is, multiple sets of audio waveform data are obtained, and the audio waveform data sets include two sets of audio waveform data, and the two sets of audio waveform data are monitored and obtained from two normal audio devices respectively; the signal fluctuation coefficients of the multiple sets of audio waveform data are calculated, and the average of the multiple signal fluctuation coefficients is used as the audio evaluation standard coefficient.

[0039] Comparing the audio evaluation standard coefficient with the audio evaluation coefficient, if the audio evaluation coefficient is greater than or equal to the audio evaluation standard coefficient, determining that the corresponding electronic audio device is a normal electronic audio device; If the audio evaluation coefficient is less than the audio evaluation standard coefficient, the corresponding electronic audio device is determined to be an abnormal audio device.

[0040] Step 2: Obtain a frequency fluctuation graph of the abnormal audio device, determine the frequency type data of the abnormal audio device according to the frequency fluctuation graph, and obtain corresponding sound quality abnormality data according to the frequency type data; It is to be further explained that: according to the preset relationship between the frequency type data (including the audio frequency position) and the sound quality abnormality data, the sound quality abnormality data corresponding to the frequency type data is determined. The sound quality abnormality data covers multiple reasons for sound quality abnormality and the standard audio fluctuation range associated with each reason.

[0041] Taking a specific implementation scenario as an example, obtaining a frequency fluctuation diagram of an abnormal audio device includes: S301: Obtain frequency data of an abnormal audio device; construct a frequency fluctuation graph with time in the frequency data as the horizontal axis and frequency value in the frequency data as the vertical axis; S302: dividing the frequency fluctuation graph into equal parts according to H audio rhythms to obtain a frequency waveform set, wherein the frequency waveform set includes G frequency sub-waveforms, where G is an integer greater than zero; S303: extracting the g-th frequency sub-waveform in the frequency waveform set, where g is an integer greater than zero and the initial value of g is 1; S304: retrieve the standard frequency waveform, calculate the difference between the frequency sub-waveform and the standard frequency waveform, and if the difference between the frequency sub-waveform and the standard frequency waveform is less than the preset frequency difference threshold, transfer to S305; if the difference between the frequency sub-waveform and the standard frequency waveform is greater than or equal to the preset frequency difference threshold, mark the frequency sub-waveform as an abnormal frequency waveform, and jump to S305; It should be additionally explained here that: a plurality of audio rhythms are pre-stored in the system database, each of which is associated with a standard frequency waveform, which reflects the normal frequency waveform of the electronic audio device without abnormal sound quality; The degree of distinction is obtained by calculating the Euclidean distance, which can intuitively measure the absolute difference between the frequency sub-waveform and the standard frequency waveform at each frequency point. If the Euclidean distance between a frequency sub-waveform and the standard waveform is large, it means that the sub-waveform deviates from the standard in frequency characteristics and is likely to be an abnormal waveform. For example, in normal audio playback, the frequency fluctuations of a specific frequency type have certain regularities. When a device fails or the audio file is damaged, the frequency waveform will change. These abnormal changes can be quickly located through the Euclidean distance.

[0042] It should be noted that before constructing the frequency fluctuation graph, the start and end time of each audio measure is recorded, and the time range is determined by the beat information of the audio and the timestamp of the audio file. For example, it is known that an audio is in 4 / 4 beats, 120 beats per minute, each beat is 0.5 seconds long, and a measure has 4 beats, that is, each measure is 2 seconds long. Combined with the start time of the audio, the time range of each measure is calculated. When the frequency fluctuation graph is divided into equal parts according to H audio measures in step 2, the audio measure number corresponding to each frequency sub-waveform is recorded at the same time. When extracting the g-th frequency sub-waveform in step 4, the standard frequency waveform associated with the audio measure is retrieved from the system database according to the recorded audio measure number.

[0043] S305: Set g=g+1 and jump back to S303; S306: Repeat S303 to S305 until g=G, then end the cycle and obtain multiple abnormal frequency waveforms; S307: extract the degree of distinction corresponding to each abnormal frequency waveform, and perform Fourier transform on the abnormal frequency waveform with the greatest degree of distinction to obtain a frequency fluctuation graph. The Fourier transform is specifically a fast Fourier transform or a short-time Fourier transform.

[0044] For example, the code: #Assume that abnormal_waveform is the abnormal frequency waveform with the greatest distinction abnormal_waveform=np.array([1,2,3,...]) #Perform Fourier transform freq_domain=np.fft.fft(abnormal_waveform) #Analyze frequency domain characteristics and find abnormal frequency components #Assume that the frequency components whose amplitude exceeds a certain threshold are abnormal threshold=100 abnormal_indices=np.where(np.abs(freq_domain)>threshold)[0] #Process abnormal frequency components and reduce amplitude

[0045] #Perform inverse Fourier transform optimized_waveform = np.fft.ifft(freq_domain) #Replace the original abnormal waveform with the optimized waveform and update the frequency fluctuation graph #frequency_waveforms is a list storing all frequency sub-waveforms g=1 frequency_waveforms[g-1]=optimized_waveform.real. In a specific implementation, determining the frequency type data of the abnormal audio device includes: Obtain the frequency fluctuation and audio frequency position of abnormal audio equipment; Inputting the frequency fluctuation and the audio frequency position into a frequency type recognition model to determine the frequency type data of the abnormal audio device; the frequency type data includes a frequent frequency change type or a stable frequency change type; The audio frequency region refers to different types of audio with different energy distribution in each frequency region. Popular audio generally has stronger energy in the mid-frequency part (such as 200Hz-2kHz) to highlight the human voice, while heavy metal audio generally has higher energy in the low-frequency (20Hz-200Hz) and high-frequency (5kHz-20kHz) parts to create a strong sense of rhythm and sharp sound effects.

[0046] Taking a specific implementation scenario as an example, the generation logic of the frequency type recognition model is: obtain the second historical sample data of frequency fluctuations and audio frequency positions, and label the corresponding frequency type data for each sample, divide the sample data into a training set and a test set, and use a deep neural network classification model (such as CNN, LSTM or Transformer), with the frequency fluctuation map and audio frequency position of the training set as input, and the frequency type data label as output, and use the cross-entropy loss function to optimize the model parameters so that it can effectively distinguish between categories with frequent frequency changes and smooth frequency changes, and use the test set for testing, and screen the model with a test accuracy that reaches a preset value as the frequency type recognition model.

[0047] It should be additionally noted here that the cross entropy loss function is used to optimize the model parameters so that it can effectively distinguish between categories with frequent frequency changes and categories with stable frequency changes.

[0048] Step 3: Collect frequency data of the audio frequency within a preset time range according to the audio frequency location of the abnormal sound quality data, and optimize the audio spectrum of the abnormal audio device based on the frequency data.

[0049] It should be additionally explained here that: the frequency data includes a plurality of frequency data; In a specific embodiment, optimizing the audio spectrum of the abnormal audio device based on the frequency data includes: s401: performing spectrum analysis on the audio spectrum to obtain frequency data of the audio spectrum within a preset time range, wherein the frequency data includes a frequency peak value and a frequency average value; s402: Set the compression ratio of the audio peak value to a range of 3:1-5:1 when the difference between the audio peak value and the average value is greater than the preset difference threshold; and adjust the level point threshold of the compressor to a level position corresponding to the frequency peak control. The level position refers to the specific position of the audio spectrum in the numerical system of the frequency peak value in decibels.

[0050] s403: Determine the adjustment strategy according to the frequency type data. If the frequency changes frequently, the start time is extended to the first extension time threshold, and the release time is set to the first release time threshold; the first time threshold is 5-10 milliseconds, and the first release time threshold is 50-100 milliseconds.

[0051] If the frequency changes smoothly, the start time is extended to the second extension time threshold, and the release time is set at the second release time threshold; the second time extension threshold is 10-20 milliseconds, and the second release time threshold is 100-200 milliseconds.

[0052] s404: Play the adjusted abnormal audio device, and re-determine whether it is an abnormal audio device again based on the audio evaluation coefficient. If not, mark the abnormal audio device as a normal audio device to complete the audio processing; If yes, repeat s402-s404 until the abnormal audio device is marked as a normal audio device.

[0053] In summary, the electronic audio data processing method based on deep learning is applied to the debugging of electronic audio equipment for large-scale concerts, and has significant beneficial effects. First, by accurately obtaining the audio evaluation coefficient of the electronic audio equipment, the abnormal audio equipment within the preset playback time period can be efficiently identified, providing a clear goal for subsequent debugging and avoiding wasting energy on normal equipment. Secondly, the audio adjustment data of the abnormal audio equipment is obtained and the frequency type and sound quality abnormal data are determined, which can solve the sound quality problem in a targeted manner and improve the audio quality. Furthermore, the audio spectrum is optimized by collecting frequency data according to the audio frequency position, and an adjustment strategy is formulated in combination with different frequency types to make the audio spectrum more reasonable. Finally, through multiple adjustments and judgments, it is ensured that the abnormal audio equipment returns to normal and the audio effect of the concert is guaranteed.

[0054] Example 2

[0055] like Figure 2 As shown, the present application provides an electronic audio data processing device 500 based on deep learning, comprising: An abnormality analysis module 510, used to obtain an audio evaluation coefficient of each electronic audio device, and determine abnormal audio devices within a preset playback time period according to the audio evaluation coefficient; The data acquisition module 520 is used to acquire audio adjustment data of the abnormal audio device, determine the frequency type data of the abnormal audio device according to the audio adjustment data, and obtain corresponding sound quality abnormality data according to the frequency type data; The audio optimization module 530 is used to collect frequency data of audio frequencies within a preset time range according to the audio frequency position of the abnormal sound quality data, and optimize the audio spectrum of the abnormal audio device based on the frequency data.

[0056] Example 3 The present application provides a computer-readable storage medium storing instructions. When the instructions are executed on a computer, the computer executes the electronic audio data processing method based on deep learning of Example 1.

[0057] The specific implementation methods described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An electronic audio data processing method based on deep learning, characterized in that: include: Step 1: obtaining an audio evaluation coefficient of each electronic audio device, and determining abnormal audio devices within a preset playback time period according to the audio evaluation coefficient; Step 2: Obtain a frequency fluctuation graph of the abnormal audio device, determine the frequency type data of the abnormal audio device according to the frequency fluctuation graph, and obtain corresponding sound quality abnormality data according to the frequency type data; Step 3: Collect frequency data of the audio frequency within a preset time range according to the audio frequency location of the abnormal sound quality data, and optimize the audio spectrum of the abnormal audio device based on the frequency data.

2. The electronic audio data processing method based on deep learning according to claim 1, characterized in that: The generation logic of the audio evaluation coefficient is as follows: Acquire audio recording data of the electronic audio device; the audio recording data includes the total number of audios within a preset time period and the audio fluctuation coefficient of each audio; The total number of audios in the preset time period and the audio fluctuation coefficient of each audio are calculated to obtain the audio evaluation coefficient, and the calculation formula is: , is the audio evaluation coefficient, is the audio fluctuation coefficient of the i-th audio, It is the total number of audios in the preset time period.

3. The electronic audio data processing method based on deep learning according to claim 2 is characterized in that: The generation logic of the audio fluctuation coefficient is as follows: S101. Extracting the audio waveform data of the i-th audio from the audio file, and performing time-frequency analysis on it to generate a time series containing frequency components; S102. Normalize the extracted audio waveform data and scale it to a standardized interval ; S103. Segment the audio waveform data according to the preset time window parameters into K segments of sub-signals, and calculate the audio feature data of each segment of the signal, wherein the audio feature data includes the local fluctuation mean and the spectrum fluctuation mean; S104. Calculate the audio fluctuation coefficient based on the audio feature data.

4. The electronic audio data processing method based on deep learning according to claim 3 is characterized in that: The calculation formula of the local fluctuation mean is: , where is the kth segment relative to the The energy fluctuation value of the segment, is the energy value of the kth audio segment, is the energy value of the k-1th audio segment; , where is the mean of all local fluctuations, K is the total number of segments, is the local fluctuation value of each segment; The calculation formula of the spectrum fluctuation mean is: , where is the mean value of spectrum fluctuation, K is the total number of segments, is the fluctuation value of each spectrum segment, is the main frequency amplitude of the kth spectrum, is the main frequency amplitude of the k-1th spectrum.

5. The electronic audio data processing method based on deep learning according to claim 4 is characterized in that: Obtain the main frequency amplitude of the kth spectrum, including: S201. Input a discrete frequency amplitude, and convert the discrete frequency amplitude from the time domain to the frequency domain through Fourier transform; S202. Calculate the The amplitude of the frequency component; S203. Get the main frequency amplitude: Traverse all frequency components and find the component with the largest amplitude; S204. Obtain the frequency corresponding to the main frequency amplitude; Identify abnormal audio devices within a preset playback time period, including: Extract audio evaluation standard coefficients of normal audio equipment; Comparing the audio evaluation standard coefficient with the audio evaluation coefficient, if the audio evaluation coefficient is greater than or equal to the audio evaluation standard coefficient, determining that the corresponding electronic audio device is a normal electronic audio device; If the audio evaluation coefficient is less than the audio evaluation standard coefficient, the corresponding electronic audio device is determined to be an abnormal audio device; According to the preset relationship between the frequency type data and the abnormal sound quality data, the abnormal sound quality data corresponding to the frequency type data is determined, and the abnormal sound quality data covers multiple abnormal sound quality reasons and the standard audio fluctuation range associated with each reason.

6. The electronic audio data processing method based on deep learning according to claim 5, characterized in that: Get frequency fluctuation graphs of abnormal audio devices, including: S301: Obtain frequency data of an abnormal audio device; construct a frequency fluctuation graph with time in the frequency data as the horizontal axis and frequency value in the frequency data as the vertical axis; S302: dividing the frequency fluctuation graph into equal parts according to H audio rhythms to obtain a frequency waveform set, wherein the frequency waveform set includes G frequency sub-waveforms, where G is an integer greater than zero; S303: extracting the g-th frequency sub-waveform in the frequency waveform set, where g is an integer greater than zero and the initial value of g is 1; S304: retrieve the standard frequency waveform, calculate the difference between the frequency sub-waveform and the standard frequency waveform, and if the difference between the frequency sub-waveform and the standard frequency waveform is less than the preset frequency difference threshold, transfer to S305; if the difference between the frequency sub-waveform and the standard frequency waveform is greater than or equal to the preset frequency difference threshold, mark the frequency sub-waveform as an abnormal frequency waveform, and jump to S305; S305: Set g=g+1 and jump back to S303; S306: Repeat S303 to S305 until g=G, then end the cycle and obtain multiple abnormal frequency waveforms; S307: extracting the degree of distinction corresponding to each abnormal frequency waveform, and performing Fourier transform on the abnormal frequency waveform with the greatest degree of distinction to obtain a frequency fluctuation graph.

7. The electronic audio data processing method based on deep learning according to claim 6, characterized in that: Identify frequency type data of abnormal audio devices, including: Obtain the frequency fluctuation and audio frequency position of abnormal audio equipment; Inputting the frequency fluctuation and the audio frequency position into a frequency type recognition model to determine the frequency type data of the abnormal audio device; the frequency type data includes a frequent frequency change type or a stable frequency change type; The generation logic of the frequency type recognition model is as follows: obtaining the second historical sample data of frequency fluctuations and audio frequency positions, and labeling the corresponding frequency type data for each sample, dividing the sample data into a training set and a test set, using a deep neural network classification model, with the frequency fluctuation map and audio frequency position of the training set as input, and the frequency type data label as output, using the cross entropy loss function to optimize the model parameters, and screening the model whose test accuracy reaches a preset value as the frequency type recognition model.

8. The electronic audio data processing method based on deep learning according to claim 7, characterized in that: Optimize the audio spectrum of abnormal audio devices based on frequency data, including: s401: performing spectrum analysis on the audio spectrum to obtain frequency data of the audio spectrum within a preset time range, wherein the frequency data includes a frequency peak value and a frequency average value; s402: set the compression ratio of the audio peak value and the average value greater than the preset difference threshold to the range of 3:1-5:1; and adjust the level point threshold of the compressor to the level position corresponding to the frequency peak control; s403: determining a regulation strategy according to the frequency type data, if the frequency changes frequently, extending the start time to a first extension time threshold, and setting the release time to a first release time threshold; If the frequency changes smoothly, the start time is extended to the second extension time threshold, and the release time is set at the second release time threshold; s404: Play the adjusted abnormal audio device, and re-determine whether it is an abnormal audio device again based on the audio evaluation coefficient. If not, mark the abnormal audio device as a normal audio device to complete the audio processing; If yes, repeat s402-s404 until the abnormal audio device is marked as a normal audio device.

9. An electronic audio data processing device based on deep learning, used to implement the electronic audio data processing method based on deep learning as described in any one of claims 1 to 8, characterized in that: include: An abnormality analysis module, used to obtain an audio evaluation coefficient of each electronic audio device, and determine abnormal audio devices within a preset playback time period according to the audio evaluation coefficient; A data acquisition module, used to acquire audio adjustment data of the abnormal audio device, determine frequency type data of the abnormal audio device according to the audio adjustment data, and obtain corresponding sound quality abnormality data according to the frequency type data; The audio optimization module is used to collect frequency data of audio frequencies within a preset time range according to the audio frequency position of the abnormal sound quality data, and optimize the audio spectrum of the abnormal audio device based on the frequency data.

10. A computer-readable storage medium, characterized in that: Instructions are stored, and when the instructions are executed on a computer, the computer is caused to execute the electronic audio data processing method based on deep learning as described in any one of claims 1 to 8.

Images