A mobile operating device with audio and video decomposition function

Abstract

The present invention relates to the technical field of data denoising, and in particular to a mobile working device with an audio and video decomposition function; a noise probability value is obtained according to the data distribution within a sliding window range in a peak point sequence of an audio signal and the length of an audio signal corresponding to the sliding window range; a signal segment is obtained according to the noise probability value; a peak feature in a spectrum diagram of the signal segment is clustered to obtain an audio feature cluster and a noise feature cluster; a signal confidence is obtained according to the time interval between the signal segment and other signal segments and the peak of the audio feature cluster; a reference segment is obtained according to the signal confidence and the noise feature cluster; a noise level characterization value is obtained according to the noise feature cluster of the signal segment and the reference segment and the signal confidence of the signal segment. The present invention obtains an adaptive step size factor according to the noise level characterization value and denoises the signal segment, thereby improving the denoising efficiency and accuracy, and making the working performance of the mobile working device better.

Description

Technical Field

[0001] The present invention relates to the technical field of data denoising, and in particular to a mobile operating device with an audio and video decomposition function. Background Art

[0002] The mobile working device with audio and video decomposition function is a portable wearable device that integrates audio and video separation, processing and analysis functions. It is usually used in the fields of audio and video data recording of personnel on-site activities and remote monitoring of construction. The device first obtains external audio and video signals, and then performs a series of processing and transmission on them, such as noise elimination, echo suppression and image enhancement. When collecting audio on site, since interference factors such as random noise will affect the audio quality, denoising is required to improve the working performance of the mobile working device.

[0003] In the existing denoising algorithms, the least mean square algorithm in the adaptive filter algorithm is usually used to remove the random noise appearing in the audio signal; however, since the mobile working device is usually in a moving state during the working process, the noise distribution intensity and distribution position in the collected audio signal are usually random. When the algorithm is used to remove the random noise in the collected audio signal, problems such as slow convergence speed or inaccurate denoising may occur. The value of the step size factor in the algorithm is one of the main influencing factors. If the random noise distribution is small and the step size factor is too small, the convergence speed of the algorithm will be slow. If the step size factor is too large, the denoising effect will be poor and the accuracy of the denoising will be affected. Therefore, when denoising the audio collected by the mobile working device, the value of the step size factor in the least mean square algorithm will affect the denoising speed and accuracy. Summary of the invention

[0004] In order to solve the technical problem that the value of the step size factor in the least mean square algorithm will affect the denoising speed and accuracy when denoising the audio collected by the mobile working device, the purpose of the present invention is to provide a mobile working device with audio and video decomposition function, and the technical solution adopted is as follows:

[0005] An audio acquisition module, used to acquire audio signals;

[0006] An audio processing module, configured to obtain a peak point sequence according to the fluctuation characteristics of the audio signal; obtain a noise probability value according to the difference characteristics of data distribution in different sliding window ranges in the peak point sequence and the difference characteristics of audio signal lengths corresponding to different sliding window ranges; and segment the audio signal according to the noise probability value to obtain different signal segments;

[0007] Noise analysis module; used to obtain the spectrum of the signal segment; cluster according to the peak features in the spectrum to obtain audio feature clusters and noise feature clusters; obtain signal confidence according to the time interval between the signal segment and other signal segments, the peak difference features of the audio feature cluster, and the peak number difference features within the audio feature cluster; obtain a reference segment in the signal segment according to the signal confidence and the noise feature cluster; obtain a noise level characterization value according to the peak difference features of the noise feature cluster of the signal segment and the reference segment and the signal confidence of the signal segment;

[0008] The audio denoising module is used to adjust the step size factor of the least mean square algorithm according to the noise level characterization value to obtain an adaptive step size factor; and denoise the signal segment by using the least mean square algorithm according to the adaptive step size factor.

[0009] Furthermore, the step of acquiring a peak point sequence according to the fluctuation characteristics of the audio signal includes:

[0010] The peak points in the audio signal are sorted in order to obtain the peak point sequence.

[0011] Furthermore, the step of obtaining the noise probability value according to the difference characteristics of data distribution in different sliding window ranges in the peak point sequence and the difference characteristics of audio signal lengths corresponding to different sliding window ranges includes:

[0012] Sliding on the peak point sequence according to a preset window and a preset sliding step size, each time sliding, obtaining a sliding window range; calculating the absolute value of the difference between the numerical standard deviation in the sliding window range and the next adjacent sliding window range, and obtaining the first noise interference characterization value of the sliding window range; calculating the interval length between the first peak point and the last peak point in any sliding window range in the audio signal, and obtaining the length characteristic value of the arbitrary sliding window range; calculating the absolute value of the difference between the length characteristic value of the sliding window range and the next adjacent sliding window range, and obtaining the second noise interference characterization value of the sliding window range; calculating the average value of the product of the first noise interference characterization value and the second noise interference characterization value of the sliding window range and normalizing them, and obtaining the noise probability value of the audio signal.

[0013] Furthermore, the step of segmenting the audio signal according to the noise probability value to obtain different signal segments includes:

[0014] The product of the noise probability value of the audio signal and a preset segmentation reference is calculated and rounded up to obtain the number of segments; and the audio signal is evenly divided into the number of different signal segments.

[0015] Furthermore, the step of clustering according to the peak features in the spectrogram to obtain audio feature clusters and noise feature clusters includes:

[0016] The peak points in the spectrum graph are sorted from large to small to obtain a peak feature sequence, and the peak feature sequence is clustered according to the K-means clustering algorithm to obtain two clusters; the cluster with the largest peak average value within the cluster is used as the audio feature cluster, and the other cluster is used as the noise feature cluster.

[0017] Furthermore, the step of obtaining signal confidence according to the time interval between the signal segment and other signal segments, the peak difference feature of the audio feature cluster, and the peak quantity difference feature within the audio feature cluster includes:

[0018] Calculate the absolute value of the difference between the average values ​​of peak values ​​in the audio feature clusters of the signal segment and other signal segments to obtain a signal energy difference characterization value; calculate the absolute value of the difference in the number of peak values ​​in the audio feature clusters of the signal segment and other signal segments to obtain a signal frequency difference characterization value; calculate the product of the inverse of the time interval between the signal segment and other signal segments, the signal energy difference characterization value, and the signal frequency difference characterization value to obtain a difference characteristic value between the signal segment and other signal segments; calculate the average value of the difference characteristic values ​​between the signal segment and all other signal segments and perform negative correlation mapping to obtain the signal confidence of the signal segment.

[0019] Further, the step of obtaining a reference segment in the signal segmentation according to the signal confidence and the noise feature cluster comprises:

[0020] When the signal segment corresponding to the maximum value of the signal confidence is used as the preliminary segment, the minimum value of the peak value average value in the noise feature cluster in the preliminary segment is used as the reference segment.

[0021] Furthermore, the step of obtaining a noise level characterization value according to the peak difference characteristics of the noise feature clusters of the signal segment and the reference segment and the signal confidence of the signal segment comprises:

[0022] Calculate the sum of the peak value average value within the noise feature cluster of the reference segment and a preset minimum positive number to obtain an evaluation benchmark; calculate the ratio of the peak value average value within the noise feature cluster of the signal segment to the evaluation benchmark to obtain a noise evaluation characteristic value; calculate the difference between a constant 1 and the signal confidence of the signal segment to obtain a noise influence degree; calculate the product of the noise evaluation characteristic value and the noise influence degree to obtain a noise degree characterization value of the signal segment.

[0023] Furthermore, the step of adjusting the step size factor of the least mean square algorithm according to the noise level characterization value to obtain an adaptive step size factor includes:

[0024] Calculate the product of the noise level characterization value and the preset step size factor to obtain the adjustment characteristic value; calculate the sum of the adjustment characteristic value and a preset minimum positive number to obtain the adjustment benchmark; map the inverse positive correlation of the adjustment benchmark to the range of the step size interval to obtain the adaptive step size factor corresponding to the signal segment.

[0025] The present invention has the following beneficial effects:

[0026] In the present invention, obtaining a peak point sequence can be used to analyze the noise interference of an audio signal at different times, and obtaining a noise probability value can determine whether random noise appears in an audio signal and the degree of random noise, so that the audio signal is segmented to different degrees according to the noise probability value, so that the number of signal segments can balance the denoising efficiency and denoising accuracy; obtaining different signal segments can make the denoising process of each signal segment more independent, thereby improving the denoising accuracy and efficiency. Obtaining a frequency graph can reflect the frequency distribution and energy intensity of the main signal and noise signal in the signal segmentation, and obtaining audio feature clusters and noise feature clusters can be used to analyze the degree of noise influence on different signal segments. Obtaining signal confidence can characterize the noise intensity of different signal segments, making the subsequent denoising process more accurate; obtaining a reference segmentation can determine the signal segment with the least noise influence in the signal segmentation, thereby quantifying the noise intensity of other signal segments, and further improving the denoising efficiency and accuracy. Obtaining the noise level characterization value can quantify the noise conditions of different signal segments, so that the value of the adaptive step factor is more in line with the noise characteristics of the signal segment, thereby improving the denoising efficiency and accuracy of the least mean square algorithm for the signal segment, and improving the working performance of the mobile working device. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0028] Figure 1 A block diagram of a mobile operating device with audio and video decomposition function provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the specific implementation, structure, features and effects of a mobile operation device with audio and video decomposition function proposed by the present invention in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0030] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0031] The following is a detailed description of a specific solution of a mobile operating device with audio and video decomposition function provided by the present invention in conjunction with the accompanying drawings.

[0032] See also Figure 1 , which shows a block diagram of a mobile operation device with audio and video decomposition function provided by an embodiment of the present invention, the device includes the following modules:

[0033] The audio acquisition module S1 is used to acquire audio signals.

[0034] In an embodiment of the present invention, the implementation scenario is to perform more rapid and accurate denoising on the audio signal collected by the mobile working device, thereby improving the working performance of the mobile working device. First, an audio signal is acquired, and the audio signal within a period of time is acquired through the microphone in the mobile working device. The implementer can determine the duration of the audio signal according to the implementation scenario. However, since the mobile working device is usually in a moving state during operation, the noise distribution intensity and distribution position in the collected audio signal are usually random. When using the algorithm to remove random noise in the collected audio signal, problems such as slow convergence speed or inaccurate denoising may occur. The value of the step size factor in the algorithm is one of the main influencing factors; therefore, it is necessary to adaptively improve the step size factor according to the noise characteristics in the audio signal.

[0035] The audio processing module S2 is used to obtain a peak point sequence according to the fluctuation characteristics of the audio signal; obtain a noise probability value according to the difference characteristics of data distribution in different sliding window ranges in the peak point sequence and the difference characteristics of audio signal lengths corresponding to different sliding window ranges; and segment the audio signal according to the noise probability value to obtain different signal segments.

[0036] Since the noise distribution in the collected audio signal is relatively random, there may be more noise in some time periods and less noise in some time periods. Therefore, in order to improve the accuracy of denoising, the audio signal can be segmented and different audio segments can be denoised to different degrees; therefore, the number of audio signal segments is first determined. When the audio signal is more likely to have noise interference, more segments are required to improve the accuracy of denoising; when the noise interference in the audio signal is less, fewer segments may be required to reduce the amount of calculation. First, the peak point sequence is obtained according to the fluctuation characteristics of the audio signal: the peak points in the audio signal are sorted in order to obtain the peak point sequence; in the embodiment of the present invention, the peak points in the audio signal are obtained by the AMPD peak detection algorithm. It should be noted that the algorithm belongs to the prior art and is applicable to signals with amplitude modulation. The specific steps are not repeated. Since the noise signal is randomly superimposed on the main frequency signal of the collected audio signal, when noise appears in the audio signal, it will cause the peak value of the peak point in the audio signal to change. Therefore, the noise probability value is obtained according to the difference characteristics of the data distribution in different sliding window ranges in the peak point sequence and the difference characteristics of the audio signal length corresponding to different sliding window ranges.

[0037] Preferably, in an embodiment of the present invention, the step of obtaining the noise probability value includes: sliding on the peak point sequence according to a preset window and a preset sliding step, and obtaining a sliding window range each time; in an embodiment of the present invention, the length of the preset window is one tenth of the length of the peak point sequence and rounded down, and the preset sliding step is three quarters of the length of the preset window and rounded down, which can be determined by the implementer according to the implementation scenario; sliding from the starting position of the peak point sequence, sliding the length of the preset step each time until sliding to the last position, and obtaining different sliding window ranges. Calculate the absolute value of the difference between the numerical standard deviation in the sliding window range and the next adjacent sliding window range to obtain the first noise interference characterization value of the sliding window range; the greater the difference in the numerical standard deviation in the two sliding window ranges, the greater the first noise interference characterization value, which means that the greater the difference in the peak value of the peak points in the two adjacent sliding window ranges, the worse the stability of the audio signal, and the more likely it is that there is noise interference in some time periods in the audio signal. Calculate the interval length between the first peak point and the last peak point in any sliding window range in the audio signal to obtain the length characteristic value of the arbitrary sliding window range; the interval length is the time distance between the first peak point and the last peak point, and the longer the time distance, the larger the length characteristic value. Calculate the absolute value of the difference between the length characteristic values ​​of the sliding window range and the next adjacent sliding window range to obtain the second noise interference characterization value of the sliding window range; the greater the difference in the length characteristic values ​​of the two adjacent sliding window ranges, the greater the second noise interference characterization value. When the sound in the environment is relatively stable and regular, the interval lengths between adjacent peak points in the corresponding audio signal are relatively similar; when the difference in the length characteristic values ​​is greater, it means that the greater the difference in the interval length, the more likely there is noise to cause abnormal peak points to appear, thereby affecting the distribution of the peak points. Calculate the average value of the product of the first noise interference characterization value and the second noise interference characterization value of the sliding window range and normalize them to obtain the noise probability value of the audio signal; it should be noted that the last sliding window range in the audio signal does not participate in the calculation. When the noise probability value is larger, it means that the audio signal is more seriously interfered by noise. The formula for obtaining the noise probability value includes:

[0038]

[0039] Where W represents the noise probability value of the audio signal, represents the activation function for normalization, M represents the number of sliding window ranges in the peak point sequence, Represents the standard deviation of the values ​​within the mth sliding window. Indicates The standard deviation of the values ​​within the sliding window, represents the first noise interference characterization value, Represents the length characteristic value of the mth sliding window range, Indicates The length characteristic value of the sliding window range, represents the second noise interference characterization value.

[0040] When the noise probability value of the audio signal is larger, more segmentation is needed to improve the denoising efficiency and accuracy, so the audio signal is segmented according to the noise probability value to obtain different signal segments; preferably, in the embodiment of the present invention, the step of obtaining different signal segments includes: calculating the product of the noise probability value of the audio signal and the preset segmentation benchmark and rounding up to obtain the number of segments; dividing the audio signal into the number of different signal segments. In the embodiment of the present invention, the preset segmentation benchmark is 20, which can be determined by the implementer according to the implementation scenario. If the number of segments is 12, the audio signal is evenly divided into 12 different signal segments.

[0041] Noise analysis module S3; used to obtain a spectrum diagram of a signal segment; clustering is performed according to the peak features in the spectrum diagram to obtain an audio feature cluster and a noise feature cluster; signal confidence is obtained according to the time interval between the signal segment and other signal segments, the peak difference features of the audio feature cluster, and the peak number difference features within the audio feature cluster; a reference segment in the signal segment is obtained according to the signal confidence and the noise feature cluster; a noise degree characterization value is obtained according to the peak difference features of the noise feature cluster of the signal segment and the reference segment, and the signal confidence of the signal segment.

[0042] After the audio signal is divided into different signal segments, when random noise appears in some time periods, it will be superimposed with the original signal in that time period. Therefore, it is necessary to accurately analyze the noise intensity in each signal segment to improve the denoising efficiency and accuracy. First, the spectrum of the signal segment is obtained. The spectrum can be obtained through the existing Fourier transform, and the specific steps are not repeated; when random noise appears in a certain signal segment, the spectrum of the signal segment contains the spectrum of the audio signal and the spectrum of the noise signal. If the noise intensity in the signal segment is large, the spectrum of the audio signal is removed, and the spectrum distribution range of the remaining spectrum and the intensity of the corresponding frequency signal are large. Then, clustering is performed according to the peak features in the spectrum to obtain audio feature clusters and noise feature clusters; preferably, in an embodiment of the present invention, the step of obtaining audio feature clusters and noise feature clusters includes: sorting the peak points in the spectrum from large to small to obtain a peak feature sequence, clustering the peak feature sequence according to the K-means clustering algorithm to obtain two clustering clusters; it should be noted that the K-means clustering algorithm belongs to the prior art, and the specific steps are not repeated. Since the audio signal that is not noise occupies the main part in the signal segmentation and has a large amplitude in the spectrum diagram, the cluster with the largest peak average value in the cluster is used as the audio feature cluster, and the other cluster is used as the noise feature cluster; the audio feature cluster represents the energy intensity of the main signal, and the noise feature cluster represents the energy intensity of the noise signal. After obtaining the audio feature clusters and noise feature clusters corresponding to different signal segments, the noise intensity in each signal segment can be judged based on the characteristics of the audio feature cluster of each signal segment. Therefore, the signal confidence is obtained based on the time interval between the signal segmentation and other signal segments, the peak difference characteristics of the audio feature cluster, and the peak number difference characteristics within the audio feature cluster.

[0043] Preferably, in an embodiment of the present invention, the step of obtaining signal confidence includes: calculating the absolute value of the difference between the average values ​​of the peak values ​​in the audio feature clusters of the signal segment and other signal segments to obtain a signal energy difference characterization value; the average value of the peak value in the audio feature cluster characterizes the energy intensity of the main signal in the signal segment, and the larger the signal energy difference characterization value is, the greater the difference in the energy intensity of the main signals of the two signal segments is, and the greater the degree of noise influence on one of the signal segments is. Calculating the absolute value of the difference between the number of peak values ​​in the audio feature clusters of the signal segment and other signal segments to obtain a signal frequency difference characterization value; because during the audio signal acquisition process, the closer the time is, the more similar the frequencies of the main signals are between the signal segments, the greater the difference in the number of peak values ​​in the audio feature clusters of the two signal segments is, and the greater the signal frequency difference characterization value is, it means that the greater the difference in the frequency components of the main signals in the two signal segments is, the greater the degree of noise influence on one of the signal segments is. Calculate the product of the inverse of the time interval between the signal segment and other signal segments, the signal energy difference characterization value, and the signal frequency difference characterization value to obtain the difference characteristic value between the signal segment and other signal segments; the time interval is used as a weight, and the closer the time interval is, the more reliable the results of the signal energy difference characterization value and the signal frequency difference characterization value. The larger the difference characteristic value between the two signal segments, the greater the difference in the main signals between the two signal segments, and the more likely one of the signal segments is to be affected by noise. Calculate the average value of the difference characteristic value between the signal segment and all other signal segments and negatively correlate them to obtain the signal confidence of the signal segment. The larger the signal confidence of the signal segment, the less the signal segment is affected by noise. The formula for obtaining signal confidence includes:

[0044]

[0045] Where D represents the confidence of the signal segmentation, represents an exponential function with a natural constant as the base, H represents the number of signal segments, represents the time interval between this signal segment and other h-th signal segments, It represents the signal energy difference between this signal segment and other h-th signal segments. Represents the signal frequency difference between this signal segment and other h-th signal segments. Represents the difference feature value between this signal segment and other h-th signal segments.

[0046] Furthermore, after obtaining the signal confidence of different signal segments, the greater the signal confidence, the smaller the noise intensity in the signal segment, so the reference segment in the signal segment can be obtained according to the signal confidence and the noise feature cluster; preferably, in an embodiment of the present invention, the step of obtaining the reference segment includes: when the signal segment corresponding to the maximum value of the signal confidence is used as a preliminary segment, since there may be multiple preliminary segments, the minimum value of the peak value average value in the noise feature cluster in the preliminary segment is used as the reference segment, and the smaller the peak value average value in the noise feature cluster, the smaller the impact of the noise on the signal segment, and the more likely the noise is to be divided into other lower spectrum peaks. The reference segment is the signal segment with the lowest noise intensity among all signal segments, so the noise intensity of other signal segments can be quantified according to the reference segment to improve the denoising efficiency and accuracy of the audio signal; and then the noise level characterization value is obtained according to the peak difference characteristics of the noise feature cluster of the signal segment and the reference segment, and the signal confidence of the signal segment.

[0047] Preferably, in an embodiment of the present invention, the step of obtaining a noise level characterization value includes: calculating the sum of the peak value average value within the noise feature cluster of the reference segment and a preset minimum positive number to obtain an evaluation benchmark; in an embodiment of the present invention, the preset minimum positive number is 0.1, the purpose is to avoid the denominator being 0, and the evaluation benchmark reflects the energy intensity of the noise signal in the reference segment. Calculate the ratio of the peak value average value within the noise feature cluster of the signal segment to the evaluation benchmark to obtain a noise evaluation characteristic value; the larger the noise evaluation characteristic value, the greater the degree of noise influence on the signal segment. Calculate the difference between the constant 1 and the signal confidence of the signal segment to obtain the noise influence degree; the greater the noise influence degree, the greater the degree of noise influence on the signal segment. Calculate the product of the noise evaluation characteristic value and the noise influence degree to obtain the noise level characterization value of the signal segment; the larger the noise level characterization value, the more obvious the noise signal in the signal segment.

[0048] The audio denoising module S4 is used to adjust the step size factor of the least mean square algorithm according to the noise level characterization value to obtain an adaptive step size factor; and denoise the signal segment by segment using the least mean square algorithm according to the adaptive step size factor.

[0049] According to the existing least mean square algorithm, when the noise signal is obvious and the noise level is high, a smaller step size factor is required to improve the accuracy of denoising when performing segmented denoising. On the contrary, a larger step size factor is used to improve the denoising efficiency. Therefore, the step size factor of the least mean square algorithm is adjusted according to the noise level characterization value to obtain an adaptive step size factor. Preferably, in an embodiment of the present invention, the step of obtaining the adaptive step size factor includes: calculating the product of the noise level characterization value and the preset step size factor to obtain an adjusted eigenvalue. In an embodiment of the present invention, the preset step size factor is 1.2, which can be determined by the implementer according to the implementation scenario. Calculate the sum of the adjusted eigenvalue and the preset minimum positive number to obtain an adjustment benchmark. In an embodiment of the present invention, the preset minimum positive number is 0.1. Map the inverse positive correlation of the adjustment benchmark to the range of the step size interval to obtain the adaptive step size factor corresponding to the signal segment. In the implementation of the present invention, the function of the positive correlation mapping is , where y represents the adaptive step size factor, e represents the natural constant, x represents the adjustment benchmark, and a represents the hyperparameter, which is 2.1 in the embodiment of the present invention. Represents the maximum eigenvalue of the autocorrelation matrix of the signal segment. The range of the step interval is .

[0050] After obtaining the adaptive step size factor, the signal segment can be denoised by the least mean square algorithm according to the adaptive step size factor. It should be noted that the least mean square algorithm belongs to the prior art, and the specific steps are not repeated. Denoising according to the adaptive step size factor can improve the efficiency and accuracy of denoising according to the noise characteristics in the signal segment, so that the working performance of the mobile working device is better. Finally, all signal segments can be transmitted or stored after denoising, which is not limited here.

[0051] In summary, the embodiments of the present invention provide a mobile working device with audio and video decomposition function; obtain noise probability values ​​according to data distribution in different sliding window ranges in the peak point sequence of the audio signal and the length of the audio signal corresponding to different sliding window ranges; obtain signal segmentation according to the noise probability values; cluster according to the peak characteristics in the spectrum diagram of the signal segmentation to obtain audio feature clusters and noise feature clusters; obtain signal confidence according to the time interval between the signal segmentation and other signal segmentations and the peak of the audio feature cluster; obtain reference segmentation according to the signal confidence and the noise feature cluster; obtain noise level characterization values ​​according to the noise feature clusters of the signal segmentation and the reference segmentation and the signal confidence of the signal segmentation. The present invention obtains an adaptive step size factor according to the noise level characterization value and denoises the signal segmentation, thereby improving the denoising efficiency and accuracy, and making the working performance of the mobile working device better.

[0052] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0053] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

Claims

1. A mobile working device with audio and video decomposition function, characterized in that: The device comprises the following modules: An audio acquisition module, used to acquire audio signals; An audio processing module, configured to obtain a peak point sequence according to the fluctuation characteristics of the audio signal; obtain a noise probability value according to the difference characteristics of data distribution in different sliding window ranges in the peak point sequence and the difference characteristics of audio signal lengths corresponding to different sliding window ranges; and segment the audio signal according to the noise probability value to obtain different signal segments; Noise analysis module; Used to obtain the spectrum of the signal segment; clustering according to the peak features in the spectrum to obtain audio feature clusters and noise feature clusters; Obtain signal confidence according to the time interval between the signal segment and other signal segments, the peak difference characteristics of the audio feature cluster, and the peak quantity difference characteristics within the audio feature cluster; obtain a reference segment in the signal segment according to the signal confidence and the noise feature cluster; obtain a noise level characterization value according to the peak difference characteristics of the noise feature cluster of the signal segment and the reference segment, and the signal confidence of the signal segment; An audio denoising module, used to adjust the step size factor of the least mean square algorithm according to the noise level characterization value to obtain an adaptive step size factor; De-noising the signal segments using a least mean square algorithm according to the adaptive step size factor; The step of acquiring a peak point sequence according to the fluctuation characteristics of the audio signal comprises: Sorting the peak points in the audio signal in order to obtain the peak point sequence; The step of obtaining the noise probability value according to the difference characteristics of data distribution in different sliding window ranges in the peak point sequence and the difference characteristics of audio signal lengths corresponding to different sliding window ranges comprises: Slide on the peak point sequence according to a preset window and a preset sliding step, and obtain a sliding window range each time it slides; calculate the absolute value of the difference between the numerical standard deviation in the sliding window range and the next adjacent sliding window range, and obtain the first noise interference characterization value of the sliding window range; calculate the interval length between the first peak point and the last peak point in any sliding window range in the audio signal, and obtain the length characteristic value of the arbitrary sliding window range; calculate the absolute value of the difference between the length characteristic value of the sliding window range and the next adjacent sliding window range, and obtain the second noise interference characterization value of the sliding window range; calculate the average value of the product of the first noise interference characterization value and the second noise interference characterization value of the sliding window range and normalize them to obtain the noise probability value of the audio signal; The step of obtaining signal confidence according to the time interval between the signal segment and other signal segments, the peak difference feature of the audio feature cluster, and the peak quantity difference feature within the audio feature cluster comprises: Calculate the absolute value of the difference between the average values ​​of peak values ​​in the audio feature clusters of the signal segment and other signal segments to obtain a signal energy difference characterization value; calculate the absolute value of the difference in the number of peak values ​​in the audio feature clusters of the signal segment and other signal segments to obtain a signal frequency difference characterization value; calculate the product of the inverse of the time interval between the signal segment and other signal segments, the signal energy difference characterization value, and the signal frequency difference characterization value to obtain a difference characteristic value between the signal segment and other signal segments; calculate the average value of the difference characteristic values ​​between the signal segment and all other signal segments and perform negative correlation mapping to obtain the signal confidence of the signal segment.

2. The mobile operation device with audio and video decomposition function according to claim 1, characterized in that: The step of segmenting the audio signal according to the noise probability value to obtain different signal segments comprises: The product of the noise probability value of the audio signal and a preset segmentation reference is calculated and rounded up to obtain the number of segments; and the audio signal is evenly divided into the number of different signal segments.

3. The mobile operation device with audio and video decomposition function according to claim 1, characterized in that: The step of clustering according to the peak features in the spectrogram to obtain audio feature clusters and noise feature clusters comprises: The peak points in the spectrum graph are sorted from large to small to obtain a peak feature sequence, and the peak feature sequence is clustered according to the K-means clustering algorithm to obtain two clusters; the cluster with the largest peak average value within the cluster is used as the audio feature cluster, and the other cluster is used as the noise feature cluster.

4. The mobile operation device with audio and video decomposition function according to claim 1, characterized in that: The step of obtaining a reference segment in the signal segmentation according to the signal confidence and the noise feature cluster comprises: When the signal segment corresponding to the maximum value of the signal confidence is used as the preliminary segment, the minimum value of the peak value average value in the noise feature cluster in the preliminary segment is used as the reference segment.

5. The mobile operation device with audio and video decomposition function according to claim 1, characterized in that: The step of obtaining the noise level characterization value according to the peak difference characteristics of the noise feature clusters of the signal segment and the reference segment and the signal confidence of the signal segment comprises: Calculate the sum of the peak value average value within the noise feature cluster of the reference segment and a preset minimum positive number to obtain an evaluation benchmark; calculate the ratio of the peak value average value within the noise feature cluster of the signal segment to the evaluation benchmark to obtain a noise evaluation characteristic value; calculate the difference between a constant 1 and the signal confidence of the signal segment to obtain a noise influence degree; calculate the product of the noise evaluation characteristic value and the noise influence degree to obtain a noise degree characterization value of the signal segment.

6. The mobile operation device with audio and video decomposition function according to claim 1, characterized in that: The step of adjusting the step size factor of the least mean square algorithm according to the noise level characterization value to obtain an adaptive step size factor comprises: Calculate the product of the noise level characterization value and the preset step size factor to obtain the adjustment characteristic value; calculate the sum of the adjustment characteristic value and a preset minimum positive number to obtain the adjustment benchmark; map the inverse positive correlation of the adjustment benchmark to the range of the step size interval to obtain the adaptive step size factor corresponding to the signal segment.

Images