A method, system and device for voice data enhancement for AI voice communication

Abstract

The present application relates to the field of intelligent control of devices, and specifically to a method, system and device for enhancing voice data for AI voice communication, including: obtaining a voice signal, converting the voice signal into a voice digital signal, using the voice digital signal to obtain its acoustic wave graph and spectrogram, using the cross-modal idea to convert the one-dimensional acoustic wave graph into a two-dimensional spectrogram grayscale graph, obtaining the changing time period of the acoustic wave graph and the spectrogram grayscale graph, obtaining the union of the changing time periods of the two to obtain a combined time period, dividing the spectrogram grayscale graph into a plurality of spectrogram grayscale segments during the combined time period, calculating the adaptive window and attenuation coefficient of each pixel point in each spectrogram grayscale segment, using the adaptive window and attenuation coefficient of each pixel point in each spectrogram grayscale segment to obtain the denoised grayscale value, and then obtaining the defogging spectrogram grayscale graph, mapping and transforming the defogging spectrogram grayscale graph to obtain an enhanced voice signal, and the present invention realizes the enhancement of the voice signal.

Description

Technical Field

[0001] The present application relates to the field of intelligent control of devices, and specifically to a voice data enhancement method, system and device for AI voice communication. Background Art

[0002] In recent years, voice communication technology has developed rapidly. However, during voice communication, voice is often affected by environmental noise. Therefore, voice data enhancement is extremely important for the development of voice communication technology.

[0003] Speech enhancement is a data processing technology for obtaining clear speech in a complex background environment. Traditional speech data enhancement methods are often spectral subtraction, Wiener filtering, and wavelet denoising. When using spectral subtraction, since the noise of the silent segment is used as the noise of the current frame, large residual noise will appear in some speech areas; and the noise power spectrum of the Wiener filter is obtained by estimating the silent segment, so the filtering effect is not good; the threshold setting of the wavelet denoising change is improper, which affects the ideal noise reduction effect; therefore, a speech data enhancement method, system and device for AI voice communication are urgently needed to solve the problem that the existing denoising methods have large residual noise and poor noise reduction effect. Summary of the invention

[0004] The present invention provides an intelligent image compression method based on big data to solve the problems of large residual noise and poor noise reduction effect in existing denoising methods.

[0005] A voice data enhancement method for AI voice communication of the present invention adopts the following technical solution:

[0006] S1, collecting a speech signal, and obtaining a spectrogram of the speech signal, and converting the spectrogram into a spectrogram grayscale image, wherein the energy value in the spectrogram is used as the grayscale value of the spectrogram grayscale image;

[0007] S2, obtaining the segmentation time interval of the spectrogram grayscale image, segmenting the spectrogram grayscale image into multiple spectrogram grayscale image segments using the segmentation time interval, and dividing all the spectrogram grayscale image segments into normal speech spectrogram grayscale image segments and noise spectrogram grayscale image segments;

[0008] S3, respectively set initial windows for the normal speech spectrum grayscale segment and the noise spectrum grayscale segment, respectively use each pixel in the normal speech spectrum grayscale segment and the noise spectrum grayscale segment as the central pixel of the set initial window, and use the mean and variance of the grayscale values ​​of all pixels in each initial window to obtain the initial window distribution of each pixel;

[0009] S4, obtaining an adaptive window for each pixel using the initial window distribution of each pixel and the frequency of all pixels in the initial window;

[0010] S5. Obtain the attenuation coefficient of each pixel according to the grayscale value of all pixels in the adaptive window where each pixel is located in the spectrogram grayscale image, obtain the grayscale value of each pixel after defogging using the attenuation coefficient and grayscale value of each pixel, obtain the defogged spectrogram grayscale image using the defogged grayscale value of each pixel, and perform mapping transformation on the defogged spectrogram grayscale image to obtain the denoised speech signal.

[0011] Furthermore, the method for obtaining the segmented time interval of the spectrogram grayscale image is:

[0012] Obtain the maximum grayscale value difference between every two adjacent time frames of the spectrogram grayscale image within the range of the speech fundamental frequency;

[0013] Set a threshold and obtain the time frame set corresponding to the maximum gray value difference greater than the threshold;

[0014] The segmented time interval of the spectrogram grayscale image is obtained by using the time frame set corresponding to the maximum grayscale value difference.

[0015] Furthermore, the method for obtaining the segmented time interval of the spectrogram grayscale image is:

[0016] Obtain the amplitude difference between every two adjacent time frames in the sound wave graph, and obtain the maximum grayscale value difference between every two adjacent time frames in the speech spectrum grayscale graph within the range of the speech fundamental frequency;

[0017] Setting difference thresholds for the amplitude difference and the maximum gray value difference respectively, and obtaining a set of time frames corresponding to the amplitude difference and the maximum gray value difference that are greater than the corresponding difference thresholds;

[0018] The time frame sets corresponding to the amplitude difference and the maximum gray value difference are used to obtain the time period sets of the acoustic wave graph and the grayscale graph of the speech spectrum respectively;

[0019] The combination time period is obtained by obtaining the union of the change time periods of the acoustic wave graph and the spectrogram grayscale graph, and the spectrogram grayscale graph is segmented using the combined time period to obtain the segmented time interval segments of the spectrogram grayscale graph.

[0020] Furthermore, the method of dividing all the spectrogram grayscale segments into normal speech spectrogram grayscale segments and noise spectrogram grayscale segments is:

[0021] When the maximum grayscale value in the spectrogram grayscale segment is greater than the threshold, the spectrogram grayscale segment is a normal speech spectrogram grayscale segment;

[0022] When the maximum grayscale value in the spectrogram grayscale segment is less than the threshold, the spectrogram grayscale segment is a noise spectrogram grayscale segment.

[0023] Furthermore, the specific expression of the distribution degree of the initial window of each pixel point is:

[0024]

[0025] Where: Indicates The grayscale image of the spectral graph in the segment The distribution of the initial window of pixels, Indicates The grayscale image of the spectral graph in the segment The grayscale mean of the initial window of pixels, Indicates The grayscale image of the spectral graph in the segment The variance of the grayscale values ​​within the initial window of pixels.

[0026] Furthermore, the specific expression for obtaining the adaptive window of each pixel is:

[0027] If the spectrogram grayscale segment is a normal speech spectrogram grayscale segment, the expression of the adaptive window of each pixel in the spectrogram grayscale segment is:

[0028]

[0029] Where: Indicates The grayscale image of the spectral graph in the segment An adaptive window of pixels, Indicates The grayscale image of the spectral graph in the segment The distribution of the initial window of pixels, Indicates The grayscale image of the spectral graph in the segment The frequency of pixels, Indicates The grayscale image of the spectral graph in the segment The hyperparameter of the initial window length of pixels, Indicates The grayscale image of the spectral graph in the segment The hyperparameter of the initial window width of pixels is 10, and 10 represents the initial window size of pixels in the grayscale segment of normal speech spectrum.

[0030] If the spectrogram grayscale segment is a noise spectrogram grayscale segment, the expression of the adaptive window of each pixel in the spectrogram grayscale segment is:

[0031]

[0032] Where: Indicates The grayscale image of the spectral graph in the segment An adaptive window of pixels, Indicates The grayscale image of the spectral graph in the segment The distribution of the initial window of pixels, Indicates The grayscale image of the spectral graph in the segment The frequency of pixels, Indicates The grayscale image of the spectral graph in the segment The hyperparameter of the initial window length of pixels, Indicates The grayscale image of the spectral graph in the segment The hyperparameter of the initial window width of pixels is 3, and 3 represents the initial window size of the noisy digital signal pixels.

[0033] Furthermore, the expression of the attenuation coefficient of each pixel is:

[0034]

[0035] Where: Indicates the "foggy" spectrum grayscale image with pixels The adaptive window centered at Indicates the "foggy" spectrum grayscale image with pixels In the adaptive window centered The gray value of the point, The overall representation is the dark channel image, Represents a global information variable.

[0036] A voice data enhancement system for AI voice communication, comprising:

[0037] The speech acquisition module acquires speech signals and obtains a spectrogram of the speech signals, and converts the spectrogram into a spectrogram grayscale image, wherein the energy value in the spectrogram is used as the grayscale value of the spectrogram grayscale image;

[0038] The signal determination module obtains the segmentation time interval of the spectrogram grayscale image, divides the spectrogram grayscale image into multiple spectrogram grayscale image segments by using the segmentation time interval, and divides all the spectrogram grayscale image segments into normal speech spectrogram grayscale image segments and noise spectrogram grayscale image segments;

[0039] Acquire an adaptive window module, set initial windows for normal speech spectrum grayscale segments and noise spectrum grayscale segments respectively, use each pixel in the normal speech spectrum grayscale segments and noise spectrum grayscale segments as the central pixel of the set initial window respectively, and use the mean and variance of the grayscale values ​​of all pixels in each initial window to obtain the initial window distribution of each pixel;

[0040] The adaptive window of each pixel is obtained by using the initial window distribution of each pixel and the frequency of all pixels in the initial window;

[0041] The speech denoising module obtains the attenuation coefficient of each pixel according to the grayscale value of all pixels in the adaptive window where each pixel is located in the spectrum grayscale image, uses the attenuation coefficient and grayscale value of each pixel to obtain the grayscale value of each pixel after defogging, uses the grayscale value of each pixel after defogging to obtain the defogged spectrum grayscale image, and performs mapping transformation on the defogged spectrum grayscale image to obtain the denoised speech signal.

[0042] A voice data enhancement device for AI voice communication, comprising: a memory, a processor, and a voice data enhancement program stored in the memory and executable on the processor, wherein when the voice data enhancement program is executed by the processor, the steps of the voice data enhancement method for AI voice communication as described in any one of S1 to S5 in the specification are implemented.

[0043] The beneficial effects of the present invention are as follows: a one-dimensional speech digital signal having only amplitude is converted into a two-dimensional spectrogram grayscale image having both frequency and amplitude, and the spectrogram grayscale image can quantify the speech digital signal for easy analysis and calculation; a dark channel priori defogging algorithm in an image is introduced into the spectrogram grayscale image denoising process, and the characteristics of an acoustic wave image and a spectrogram grayscale image are combined to adaptively obtain a statistical window in the dark channel priori algorithm, i.e., an adaptive window, for defogging operation to obtain a defogged spectrogram grayscale image, and the adaptive window combines the characteristics of the spectrogram grayscale image itself to make the defogging result more accurate; a defogging spectrogram grayscale image is mapped and transformed to obtain a denoised speech signal, thereby achieving enhancement of speech data. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 A flow chart of a method for enhancing voice data for AI voice communication according to the present invention;

[0046] Figure 2 The present invention is a structural block diagram of a voice data enhancement system for AI voice communication. DETAILED DESCRIPTION

[0047] The present invention introduces the idea of ​​cross-modal data processing, converting a one-dimensional speech signal with only amplitude into a two-dimensional spectrogram with both frequency and amplitude. The dark channel prior defogging algorithm in the image is introduced into the spectrogram denoising process, and the statistical window in the dark channel prior algorithm is adaptively obtained to perform defogging operations based on the characteristics of the acoustic wave graph and the spectrogram, thereby achieving speech data enhancement.

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] An embodiment of a voice data enhancement method for AI voice communication of the present invention is as follows: Figure 1 As shown, including:

[0050] S1. Collect a speech signal, obtain a spectrogram of the speech signal, and convert the spectrogram into a spectrogram grayscale image, wherein the energy value in the spectrogram is used as the grayscale value of the spectrogram grayscale image.

[0051] The specific steps for collecting a voice signal are: setting up a corresponding platform, including a microphone, a sound card, a data transmission system, a voice data enhancement system, a data storage module and a supporting bracket, and using the microphone device in the platform to collect voice signals for a certain period of time.

[0052] The voice signal is converted into a voice digital signal using the analog-to-digital conversion function of the sound card device, a sound wave diagram of the voice digital signal is drawn according to the time and amplitude in the voice digital signal, and a spectrogram of the voice signal is obtained by Fourier transforming the voice digital signal.

[0053] Among them, the one-dimensional speech signal with only amplitude is converted into a two-dimensional spectrogram with both frequency and amplitude. The color value in the spectrogram represents the strength of the signal energy. The darker the color, the greater the energy. The color only has semantic meaning. The specific color value is the size of the three channels. It has no reference meaning. Therefore, in order to be more conducive to quantifying the size of energy, the energy value is normalized to the range of 0~255 in the present invention.

[0054]

[0055] In the formula, Indicates Frame voice digital signal The frequency-normalized energy value, Indicates the maximum energy value in the digital voice signal. It represents the minimum energy value in the digital speech signal, and 255 represents the gray value distribution range, so that the normalized result is in the range of 0~255.

[0056] At this point, the colored spectrogram can be converted into a grayscale spectrogram, and the grayscale value of each pixel can represent the energy value; the larger the grayscale value of the pixel, the brighter the brightness in the image, which represents greater energy.

[0057] S2. Obtain the segmentation time interval of the spectrogram grayscale image, use the segmentation time interval to segment the spectrogram grayscale image into multiple spectrogram grayscale segments, and divide all the spectrogram grayscale segments into normal speech spectrogram grayscale segments and noise spectrogram grayscale segments.

[0058] Among them, the first method of obtaining the segmentation time interval of the spectrogram grayscale image is:

[0059] Obtain the maximum grayscale value difference between every two adjacent time frames of the spectrogram grayscale image within the range of the speech fundamental frequency; set a threshold to obtain the time frame set corresponding to the maximum grayscale value difference greater than the threshold; use the time frame set corresponding to the maximum grayscale value difference to obtain the segmented time interval of the spectrogram grayscale image.

[0060] The second method for obtaining the segmentation time interval of the spectrogram grayscale image is: obtaining the amplitude difference between every two adjacent time frames in the sound wave image, and obtaining the maximum grayscale value difference between every two adjacent time frames in the spectrogram grayscale image within the fundamental frequency range of speech; setting difference thresholds for the amplitude difference and the maximum grayscale value difference respectively, and obtaining the set of time frames corresponding to the amplitude difference and the maximum grayscale value difference that are greater than the corresponding difference thresholds; using the set of time frames corresponding to the amplitude difference and the maximum grayscale value difference to obtain the set of changing time periods of the sound wave image and the spectrogram grayscale image respectively; finding the union of the set of changing time periods of the sound wave image and the spectrogram grayscale image to obtain the combined time period, and using the combined time period to obtain the segmentation time interval of the spectrogram grayscale image.

[0061] The present invention adopts the second method, specifically: the specific steps of obtaining the change time period of the sound wave graph are: according to the prior art, it is known that the amplitude range of noise is greatly different from the amplitude range of normal clean speech signals, so the current frame in the sound wave graph is obtained. The amplitude With the previous frame The amplitude Differences , by setting the amplitude threshold , determine the time period Greater than the amplitude threshold The corresponding time frames are recorded as . Will gather Each time frame in the sequence of the previous time frame With the next time frame Combined into variable time periods ,in, is frame 0 and The frames are combined to obtain, for example: and The two time frames are combined into , and The two time frames are combined into . So far, each change time period in the acoustic wave graph is obtained. Among them, the amplitude threshold It can be determined according to the specific implementation of the implementer. The present invention provides an empirical reference value. .

[0062] The specific steps of obtaining the time period of the spectrogram grayscale image are as follows: According to the prior art, the energy of the noise is evenly distributed in the spectrogram grayscale image, and the energy value is small; while the speech signal has regular changes in the spectrogram grayscale image, and the energy value is large. Therefore, the place with greater energy is more likely to be the time period of the speech signal. According to the fundamental frequency attribute of the speech signal, through the current frame The maximum energy value in the range of 100HZ to 400HZ under the same frame The energy difference is calculated based on the maximum energy value in the range of 100HZ to 400HZ under ,in Indicates the fundamental frequency range , by setting the energy threshold Determine the time period Greater than the amplitude threshold Each time frame is denoted as . Will gather Each time frame in the sequence, that is, the previous time frame With the next time frame Combined into variable time periods ,in, is frame 0 and The frames are combined to obtain, for example: and The two time frames are combined into , and The two time frames are combined into . So far, each change time period in the grayscale graph of the spectrum is obtained. The energy threshold It can be determined according to the specific implementation of the implementer. The present invention provides an empirical reference value. .

[0063] Since the acoustic wave graph and the spectrogram grayscale graph are two different modes of the same speech signal, the set of time periods of the acoustic wave graph is The set of changing time periods in the spectrogram Perform a union operation on each change time period in the set, that is, combine the corresponding change time periods in pairs to form a larger change time period. , and the next combined time period starts from the combined time period Calculation starts from the maximum range and continues in sequence to obtain all combined time periods.

[0064] The method to obtain the combined time period is: The first change period in scope , and the collection The first change period scope , the intersection operation of the two forms a larger combination time range ;gather The second change period in scope , and the collection The second change period scope , since the maximum range of the first combined time period range is 200, the second combined time period range is .

[0065] At this point, a combined time period is obtained, and the combined time period, as a segmentation time interval of the spectrogram grayscale image, can divide the entire horizontal axis of the spectrogram grayscale image into multiple spectrogram grayscale image segments.

[0066] The maximum grayscale value of the pixel points of the speech digital signal in the spectrogram grayscale map is counted, that is, the maximum grayscale value mean in multiple segments of clear speech digital signal data is obtained, a threshold is set according to the maximum grayscale value mean, and the specific steps of distinguishing the normal digital signal and the noise digital signal in each spectrogram grayscale segment according to the set threshold are as follows: according to the characteristic of the larger energy value in the normal digital signal, the grayscale value of the normal digital signal in the spectrogram grayscale map is larger, and the grayscale value of the noise digital signal is smaller. By counting the size of the grayscale value corresponding to the signal in each spectrogram grayscale segment in the spectrogram grayscale map, it is determined whether each spectrogram grayscale segment is mainly a normal speech spectrogram grayscale segment or a noise spectrogram grayscale segment, and the threshold is set according to the empirical value , if the maximum grayscale value of the speech digital signal in the spectrum grayscale segment is greater than the threshold , then the spectrogram grayscale segment is set as a normal speech spectrogram grayscale segment. If the maximum grayscale value of the speech digital signal in the spectrogram grayscale segment is less than the threshold , then the spectrum grayscale segment is set as the noise spectrum grayscale segment.

[0067] S3. Set initial windows for normal speech spectrum grayscale segments and noise spectrum grayscale segments respectively, use each pixel in the normal speech spectrum grayscale segments and noise spectrum grayscale segments as the central pixel of the set initial window respectively, and use the mean and variance of the grayscale values ​​of all pixels in each initial window to obtain the initial window distribution of each pixel.

[0068] S4. Obtain an adaptive window for each pixel using the initial window distribution of each pixel and the frequency of all pixels in the initial window.

[0069] Among all the spectrum grayscale segments, the first The maximum value of the horizontal coordinate of the grayscale segment of the spectrum , the maximum value of the vertical axis range , where the range of the ordinate is the same for each spectrum grayscale segment, i.e. the maximum value of the frequency in the spectrogram; the range of the abscissa and ordinate is the Adaptive window of pixels within the grayscale image segment The maximum value of the grayscale image of the spectrum is known from small to large in frequency, and the grayscale value of the pixel with low frequency is greater than the grayscale value of the pixel with high frequency. Therefore, considering the amount of calculation and the accuracy of the result, in order to obtain the adaptive window of each pixel in the grayscale image segment of the spectrum, For the grayscale segment of the noise spectrum, since the grayscale value of the noise pixel is low, the smaller For normal speech spectrum grayscale segments, since the grayscale value of speech pixels is high, a larger The optimal dark channel can be obtained only when the value is 0.05. Therefore, the size of the initial window is 3×3 for the normal speech spectrum grayscale segment, and the size of the initial window is 10×10 for the noise spectrum grayscale segment. The distribution of the grayscale value of each pixel in the corresponding window is calculated in different spectrum grayscale segments. , thus obtaining an adaptive window. The grayscale image of the spectral graph in the segment The distribution of the initial window of pixels The specific expression is:

[0070]

[0071] Where: Indicates The grayscale image of the spectral graph in the segment The distribution of the initial window of pixels, Indicates The grayscale image of the spectral graph in the segment The mean gray value of the initial window of pixels, Indicates The grayscale image of the spectral graph in the segment The variance of the grayscale value in the initial window of pixels, is the hyperbolic tangent function.

[0072] The logic of this formula is: the smaller the mean of the grayscale value in the initial window of each pixel point in each spectrogram grayscale segment of the spectrogram grayscale image, the greater the probability that the window contains a better dark channel, but the smaller the mean, the greater the probability that the window may contain a larger grayscale value. Therefore, the window that satisfies both the small mean and the small variance has a greater probability of containing a dark channel. In the present invention, the weight of the mean in the window of the window distribution calculation is greater than the weight of the variance, so the weights of the mean and variance are set to 0.6 and 0.4 respectively.

[0073] According to The grayscale image of the spectral graph in the segment The distribution of the initial window of pixels is obtained The grayscale image of the spectral graph in the segment The adaptive window of pixels. The first spectrum grayscale segment is a normal speech spectrum grayscale segment, then The grayscale image of the spectral graph in the segment The expression of the adaptive window of pixels is:

[0074]

[0075] Where: Indicates The grayscale image of the spectral graph in the segment An adaptive window of pixels, Indicates The grayscale image of the spectral graph in the segment The distribution of the initial window of pixels, Indicates The grayscale image of the spectral graph in the segment The frequency of pixels, Indicates The grayscale image of the spectral graph in the segment The hyperparameter of the length of the initial window of pixels, Indicates The grayscale image of the spectral graph in the segment The width of the initial window is a hyperparameter of pixels, and 10 represents the initial window size of the grayscale spectrum segment of normal speech.

[0076] Jordi The first spectrum grayscale segment is a noise spectrum grayscale segment. The grayscale image of the spectral graph in the segment The expression of the adaptive window of pixels is:

[0077]

[0078] Where: Indicates The grayscale image of the spectral graph in the segment An adaptive window of pixels, Indicates The grayscale image of the spectral graph in the segment The distribution of the initial window of pixels, Indicates The grayscale image of the spectral graph in the segment The frequency of pixels, Indicates The grayscale image of the spectral graph in the segment The hyperparameter of the length of the initial window of pixels, Indicates The grayscale image of the spectral graph in the segment The hyperparameter of the width of the initial window of pixels is 3, and 3 represents the initial window size of the noise spectrum grayscale segment.

[0079] The formula logic is: within the window The smaller it is, the smaller the mean and variance in the window are, the greater the probability that the window contains a dark channel, and the smaller the step size that needs to be adjusted to reduce the amount of calculation; The larger the pixel The greater the frequency, the smaller the gray value of the pixel, the greater the probability of the dark channel near the window, and the smaller the step size that needs to be adjusted. and Used to adjust the step value, which can be determined according to the specific implementation of the implementer. Among them, for the noise spectrum grayscale segment: , , for the speech spectrogram grayscale segment: , ,in, and For the The maximum value of the horizontal axis and the maximum value of the vertical axis in the grayscale image of the segment spectrum. If the adaptive window is recalculated , the calculated value cannot exceed The maximum value of the range of the horizontal axis and the maximum value of the range of the vertical axis within the grayscale image of the segment. If exceeded, the window size is set to , and It can be directly obtained from the spectrogram grayscale image.

[0080] At this point, the adaptive window of each pixel in each spectrogram grayscale segment, that is, the statistical window of the adaptive dark channel prior dehazing algorithm, is obtained.

[0081] S5. Obtain the attenuation coefficient of each pixel according to the grayscale value of all pixels in the adaptive window where each pixel is located in the spectrogram grayscale image, obtain the grayscale value of each pixel after defogging using the attenuation coefficient and grayscale value of each pixel, obtain the defogged spectrogram grayscale image using the defogged grayscale value of each pixel, and perform mapping transformation on the defogged spectrogram grayscale image to obtain the denoised speech signal.

[0082] The specific steps for obtaining the attenuation coefficient of each pixel point are: introducing the idea of ​​cross-modal processing, and realizing the modal conversion from digital to image by converting the one-dimensional speech signal into a two-dimensional spectrogram grayscale image. It is known from the prior art that normal digital signals have obvious "horizontal stripes" and "vertical stripes" in the spectrogram, and the distance between each "vertical stripe", that is, the sampling time point, is relatively average, and the energy value is relatively high; the noise digital signal is usually evenly distributed in the spectrogram, and the energy is relatively coherent and the energy value is low. When there is a noise digital signal in the speech digital signal, it will appear obvious fog in the spectrogram grayscale image. Through the statistical window of the adaptive dark channel prior defogging algorithm, that is, the adaptive window, the spectrogram grayscale image is subjected to an adaptive defogging algorithm to realize denoising processing, and the data modality conversion is performed to obtain enhanced speech data.

[0083] Therefore, the present invention adopts the statistical window of the adaptive dark channel prior dehazing algorithm, performs an adaptive dehazing algorithm on the spectrogram, obtains a "clearer" spectrogram by dehazing the spectrogram, and performs a mapping transformation on the spectrogram, that is, converts the image into a digital image, thereby obtaining a denoised speech signal.

[0084] The model of the grayscale graph of the spectrum describing "fog" in the present invention can be expressed as:

[0085]

[0086] Where: Pixel points in the grayscale spectrum representing "fog" The gray value of Pixel points in the grayscale spectrum representing “no fog” The gray value of Represents the global information variable, i.e., the maximum grayscale value in the spectrogram grayscale image. Represents a pixel in the grayscale image of the spectrogram The attenuation coefficient of the gray value.

[0087] Among them, the attenuation coefficient Pixel points in the grayscale spectrum representing “no fog” The gray value of the non-masked ratio displayed under the noise mask, The larger the value, the less the noise digital signal is masked. Represents the global information variable, which is used to represent the speech signal of the current frame. The energy information due to background influence is Indicates the ratio of the grayscale of this part to the grayscale of the normal digital signal of the current frame.

[0088] According to the model of the "foggy" spectral grayscale image, the existing attenuation coefficient can be obtained The calculation expression is:

[0089]

[0090] Where: Indicates the "foggy" spectrum grayscale image with pixels The adaptive window centered at Indicates the "foggy" spectrum grayscale image with pixels The gray value of point b in the adaptive window centered at The overall representation of the dark channel image is to take the minimum grayscale value in the "foggy" spectrum grayscale image and represent it as a pixel. Take the center The size of the window is used, and the minimum gray value in the window is taken to replace the central pixel The gray value of the dark channel, Represents a global information variable.

[0091] Therefore, the adaptive statistical window size of the local area is calculated, that is, the pixel point Adaptive The lower the gray value of the pixel in the window, the better the dark channel is. The window affects the construction of the dark channel, and thus affects the effect of the "defogging" dark channel priori algorithm. According to the algorithm principle, the best dark channel is pure black. Therefore, this case realizes the "defogging" processing of the spectrogram by implementing the adaptive parameters of the "defogging" dark channel priori algorithm.

[0092] According to the adaptive window of each pixel point in each spectrum grayscale segment of the spectrum grayscale image, the attenuation coefficient of the corresponding pixel point, the frequency of the corresponding pixel point and the model of the "foggy" spectrum grayscale image, the calculation expression of the grayscale value of the pixel point in the spectrum grayscale image after "defogging" is obtained:

[0093]

[0094] Where: Pixels in the grayscale image of the spectrum representing "fog" Gray value of Pixel points in the grayscale spectrum representing “no fog” Gray value of Represents global information variables, Represents pixel The attenuation coefficient of the gray value is The threshold is set to avoid the attenuation coefficient being too small and .

[0095] The grayscale value of the pixel in the dehazed spectrogram grayscale image is used to obtain the dehazed spectrogram grayscale image.

[0096] The obtained "de-fogging" spectrogram grayscale image is mapped and transformed to obtain the processed speech signal. The specific process is: The "de-fogging" spectrogram grayscale image can be represented as behavior N , listed as M The matrix P, Indicates Row and The gray value of the column point, then for The The mapping of the dehazed spectrogram grayscale image to the speech signal can be expressed as:

[0097]

[0098] Where: The mapped speech signal is indexed for time, and Respectively angular frequency and initial phase, 25 is the frame length, and the corresponding speech length is , Indicates the number of rows in the spectrum grayscale image matrix after “dehazing”.

[0099] The obtained "de-fogging" spectrogram grayscale image is mapped and transformed to obtain the denoised speech signal, that is, the denoised speech signal is obtained.

[0100] A voice data enhancement system for AI voice communication, such as Figure 2 As shown, including:

[0101] The voice acquisition module uses a microphone device to collect a voice signal, converts the collected voice signal into a voice digital signal using a sound card device, draws a sound wave graph of the voice digital signal, obtains a spectrogram of the voice digital signal, converts all energy values ​​in the spectrogram into grayscale values ​​to construct a spectrogram grayscale graph.

[0102] The signal determination module is used to obtain the segmentation time interval of the spectrogram grayscale image, segment the spectrogram grayscale image into multiple spectrogram grayscale segments using the segmentation time interval, and divide all the spectrogram grayscale segments into normal speech spectrogram grayscale segments and noise spectrogram grayscale segments.

[0103] Obtain an adaptive window module, which is used to set initial windows for normal speech spectrogram grayscale segments and noise spectrogram grayscale segments respectively, and use each pixel point in the normal speech spectrogram grayscale segment and the noise spectrogram grayscale segment as the central pixel point of the set initial window respectively, and use the mean and variance of the grayscale values ​​of all pixels in each initial window to obtain the initial window distribution degree of each pixel point;

[0104] The adaptive window of each pixel is obtained by using the initial window distribution of each pixel and the frequency of all pixels in the initial window.

[0105] The speech denoising module is used to obtain the attenuation coefficient of each pixel according to the grayscale value of all pixels in the adaptive window where each pixel is located in the speech spectrum grayscale image, use the attenuation coefficient and grayscale value of each pixel to obtain the grayscale value of each pixel after defogging, use the grayscale value of each pixel after defogging to obtain the defogged speech spectrum grayscale image, and perform mapping transformation on the defogged speech spectrum grayscale image to obtain the denoised speech signal.

[0106] A voice data enhancement device for AI voice communication, comprising: a memory, a processor, and a voice data enhancement program stored in the memory and executable on the processor, wherein when the voice data enhancement program is executed by the processor, the steps of the voice data enhancement method for AI voice communication as described in S1-S5 in the specification are implemented.

[0107] The beneficial effects of the present invention are as follows: a one-dimensional speech digital signal having only amplitude is converted into a two-dimensional spectrogram grayscale image having both frequency and amplitude, and the spectrogram grayscale image can quantify the speech digital signal for easy analysis and calculation; a dark channel priori defogging algorithm in an image is introduced into the spectrogram grayscale image denoising process, and the characteristics of an acoustic wave image and a spectrogram grayscale image are combined to adaptively obtain a statistical window in the dark channel priori algorithm, i.e., an adaptive window, for defogging operation to obtain a defogged spectrogram grayscale image, and the adaptive window combines the characteristics of the spectrogram grayscale image itself to make the defogging result more accurate; a defogging spectrogram grayscale image is mapped and transformed to obtain a denoised speech signal, thereby achieving enhancement of speech data.

[0108] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A voice data enhancement method for AI voice communication, characterized in that: include: S1, collecting a speech signal, and obtaining a spectrogram of the speech signal, and converting the spectrogram into a spectrogram grayscale image, wherein the energy value in the spectrogram is used as the grayscale value of the spectrogram grayscale image; S2, obtaining the segmentation time interval of the spectrogram grayscale image, segmenting the spectrogram grayscale image into multiple spectrogram grayscale image segments using the segmentation time interval, and dividing all the spectrogram grayscale image segments into normal speech spectrogram grayscale image segments and noise spectrogram grayscale image segments; S3, respectively set initial windows for the normal speech spectrum grayscale segment and the noise spectrum grayscale segment, respectively use each pixel in the normal speech spectrum grayscale segment and the noise spectrum grayscale segment as the central pixel of the set initial window, and use the mean and variance of the grayscale values ​​of all pixels in each initial window to obtain the initial window distribution of each pixel; S4, obtaining an adaptive window for each pixel using the initial window distribution of each pixel and the frequency of all pixels in the initial window; S5. Obtain the attenuation coefficient of each pixel according to the grayscale value of all pixels in the adaptive window where each pixel is located in the spectrogram grayscale image, obtain the grayscale value of each pixel after defogging using the attenuation coefficient and grayscale value of each pixel, obtain the defogged spectrogram grayscale image using the defogged grayscale value of each pixel, and perform mapping transformation on the defogged spectrogram grayscale image to obtain the denoised speech signal.

2. The method for enhancing voice data for AI voice communication according to claim 1, characterized in that: The method for obtaining the segmented time interval of the spectrogram grayscale image is: Obtain the maximum grayscale value difference between every two adjacent time frames of the spectrogram grayscale image within the range of the speech fundamental frequency; Set a threshold and obtain the time frame set corresponding to the maximum gray value difference greater than the threshold; The segmented time interval of the spectrogram grayscale image is obtained by using the time frame set corresponding to the maximum grayscale value difference.

3. The method for enhancing voice data for AI voice communication according to claim 1, characterized in that: The method for obtaining the segmented time interval of the spectrogram grayscale image is: Obtain the amplitude difference between every two adjacent time frames in the sound wave graph, and obtain the maximum grayscale value difference between every two adjacent time frames in the speech spectrum grayscale graph within the range of the speech fundamental frequency; Setting difference thresholds for the amplitude difference and the maximum gray value difference respectively, and obtaining a set of time frames corresponding to the amplitude difference and the maximum gray value difference that are greater than the corresponding difference thresholds; The time frame sets corresponding to the amplitude difference and the maximum gray value difference are used to obtain the time period sets of the acoustic wave graph and the grayscale graph of the speech spectrum respectively; The combination time period is obtained by obtaining the union of the change time periods of the acoustic wave graph and the spectrogram grayscale graph, and the spectrogram grayscale graph is segmented using the combined time period to obtain the segmented time interval segments of the spectrogram grayscale graph.

4. The method for enhancing voice data for AI voice communication according to claim 1, characterized in that: The method for dividing all the spectrogram grayscale segments into normal speech spectrogram grayscale segments and noise spectrogram grayscale segments is: Set a threshold. If the maximum grayscale value of the speech signal in the spectrogram grayscale segment is greater than the threshold, the spectrogram grayscale segment is used as a normal speech spectrogram grayscale segment. If the maximum grayscale value of the speech signal in the spectrogram grayscale segment is less than the threshold, the spectrogram grayscale segment is used as a noise spectrogram grayscale segment.

5. The method for enhancing voice data for AI voice communication according to claim 1, characterized in that: The specific expression of the distribution degree of the initial window of each pixel is: Where: Indicates The grayscale image of the spectral The distribution of the initial window of pixels, Indicates The grayscale image of the spectral The grayscale mean of the initial window of pixels, Indicates The grayscale image of the spectral The variance of the grayscale values ​​within the initial window of pixels.

6. The method for enhancing voice data for AI voice communication according to claim 1, characterized in that: The specific expression for obtaining the adaptive window of each pixel is: If the spectrogram grayscale segment is a normal speech spectrogram grayscale segment, the expression of the adaptive window of each pixel in the spectrogram grayscale segment is: Where: Indicates The grayscale image of the spectral An adaptive window of pixels, Indicates The grayscale image of the spectral The distribution of the initial window of pixels, Indicates The grayscale image of the spectral The frequency of pixels, Indicates The grayscale image of the spectral The hyperparameter of the initial window length of pixels, Indicates The grayscale image of the spectral The hyperparameter of the initial window width of pixels, 10 represents the initial window size of pixels in the grayscale segment of normal speech spectrogram; If the spectrogram grayscale segment is a noise spectrogram grayscale segment, the expression of the adaptive window of each pixel in the spectrogram grayscale segment is: Where: Indicates The grayscale image of the spectral An adaptive window of pixels, Indicates The grayscale image of the spectral The distribution of the initial window of pixels, Indicates The grayscale image of the spectral The frequency of pixels, Indicates The grayscale image of the spectral The hyperparameter of the initial window length of pixels, Indicates The grayscale image of the spectral The hyperparameter of the initial window width of pixels is 3, and 3 represents the initial window size of the noisy digital signal pixels.

7. The method for enhancing voice data for AI voice communication according to claim 1, characterized in that: The expression of the attenuation coefficient of each pixel is: Where: Indicates the "foggy" spectrum grayscale image with pixels The adaptive window centered at Indicates the "foggy" spectrum grayscale image with pixels In the adaptive window centered The gray value of the point, The overall representation is the dark channel image, Represents a global information variable.

8. A voice data enhancement system for AI voice communication, characterized in that: include: The speech acquisition module is used to acquire speech signals, obtain a spectrogram of the speech signals, and convert the spectrogram into a spectrogram grayscale graph, wherein the energy value in the spectrogram is used as the grayscale value of the spectrogram grayscale graph; A signal determination module is used to obtain a segmentation time interval of a spectrogram grayscale image, segment the spectrogram grayscale image into multiple spectrogram grayscale image segments using the segmentation time interval, and divide all the spectrogram grayscale image segments into normal speech spectrogram grayscale image segments and noise spectrogram grayscale image segments; Obtain an adaptive window module, which is used to set initial windows for normal speech spectrogram grayscale segments and noise spectrogram grayscale segments respectively, and use each pixel point in the normal speech spectrogram grayscale segment and the noise spectrogram grayscale segment as the central pixel point of the set initial window respectively, and use the mean and variance of the grayscale values ​​of all pixels in each initial window to obtain the initial window distribution degree of each pixel point; The adaptive window of each pixel is obtained by using the initial window distribution of each pixel and the frequency of all pixels in the initial window; The speech denoising module is used to obtain the attenuation coefficient of each pixel according to the grayscale value of all pixels in the adaptive window where each pixel is located in the speech spectrum grayscale image, use the attenuation coefficient and grayscale value of each pixel to obtain the grayscale value of each pixel after defogging, use the grayscale value of each pixel after defogging to obtain the defogged speech spectrum grayscale image, and perform mapping transformation on the defogged speech spectrum grayscale image to obtain the denoised speech signal.

9. A voice data enhancement device for AI voice communication, characterized in that: include: A memory, a processor, and a voice data enhancement program stored in the memory and executable on the processor, wherein the voice data enhancement program, when executed by the processor, implements the steps of the voice data enhancement method for AI voice communication as described in any one of claims 1 to 7.

Images