A smart Internet of Things multimedia teaching management method and system

By screening effective extreme points and performing empirical modal decomposition and denoising, the problem of poor denoising and denoising of multimedia teaching audio data in the prior art is solved, and the management effect of multimedia teaching data in the Internet of Things is improved.

CN118197344BActive Publication Date: 2024-11-08GUANGDONG UNIV OF SCI & TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410327345.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-03-21
Publication Date
2024-11-08
Estimated Expiration
2044-03-21

AI Technical Summary

Technical Problem

The existing technology has poor denoising effect on multimedia teaching audio data, resulting in poor management of multimedia teaching data in IoT. It is mainly because the empirical modal decomposition method is sensitive to extreme points caused by noise, resulting in the component of the eigenmodal function. The extraction is inaccurate and produces a pseudo-modal.

Method used

By analyzing the audio fluctuation amplitude distribution, the continuous similar audio intervals of strong amplitude and weak amplitude are selected, combined with the grayscale distribution characteristics of the spectrogram, the effective extreme points are selected, and the empirical modal decomposition and denoising process are carried out to improve the accuracy of denoising process. sex.

Benefits of technology

It improves the denoising effect of multimedia teaching audio data, enhances the management ability of IoT multimedia teaching data, and ensures the quality of audio data after denoising.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118197344B_ABST
    Figure CN118197344B_ABST
Patent Text Reader

Abstract

The present invention relates to the technical field of audio noise filtering, and specifically to a method and system for managing multimedia teaching in the intelligent Internet of Things. The present invention analyzes the distribution of audio fluctuation amplitudes of multimedia teaching audio data and the similarity of audio fluctuations to obtain a teaching audio interval to be tested containing human voice information; further analyzes the grayscale distribution of the spectrogram corresponding to the teaching audio interval to be tested to obtain the degree of noise interference of extreme points in each teaching audio interval to be tested; performs empirical mode decomposition denoising on multimedia teaching audio data according to the effective extreme points obtained from the noise interference degree and the distribution of extreme points to obtain multimedia teaching denoised audio data, and finally compresses and stores the multimedia teaching denoised audio data, thereby performing multimedia teaching management in the Internet of Things. The present invention has a better management effect on multimedia teaching data in the Internet of Things by obtaining effective extreme points for empirical mode decomposition denoising.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of audio noise filtering, and in particular to a smart Internet of Things multimedia teaching management method and system. Background Art

[0002] With the advent of the Internet era, teaching methods have undergone tremendous changes. Multimedia teaching is a commonly used teaching method that can help students learn through videos. However, when transmitting over the network, due to the large capacity of video files, it is not efficient to directly transmit the original video, so the video needs to be compressed. Video files include image files and audio files, of which audio files are crucial for a good teaching video, but audio files usually contain noise information. Therefore, in order to ensure the compression effect and the quality of the compressed audio, it is usually necessary to denoise the collected multimedia teaching audio data before compressing it.

[0003] The prior art method for denoising the collected multimedia teaching audio data is usually as follows: decomposing the collected multimedia teaching audio data into different intrinsic mode function components by the empirical mode decomposition method, denoising each intrinsic mode function component by wavelet denoising, reconstructing the denoised intrinsic mode function components, and obtaining the denoised multimedia teaching audio data. However, when the empirical mode decomposition decomposes the collected multimedia teaching audio data, it is necessary to perform cubic spline difference fitting on the extreme points in the multimedia teaching audio data and then find the average value, so the extreme points in the multimedia teaching audio data have a greater impact on the empirical mode decomposition. However, there may be some extreme points caused by noise in the collected multimedia teaching audio data. The existence of these extreme points caused by noise will lead to inaccurate extraction of intrinsic mode function components or generate pseudo-modes, reduce the accuracy of the empirical mode decomposition, and make the denoising effect of the multimedia teaching audio data poor, affecting the effective management of the multimedia teaching data of the Internet of Things. Summary of the invention

[0004] In order to solve the technical problem that the prior art uses the empirical mode decomposition method to perform denoising on the collected audio data, which has a poor effect and affects the effective management of the Internet of Things multimedia teaching data, the purpose of the present invention is to provide a smart Internet of Things multimedia teaching management method and system, and the technical solutions adopted are as follows:

[0005] The present invention proposes a smart Internet of Things multimedia teaching management method, the method comprising:

[0006] Obtain multimedia teaching audio data;

[0007] According to the distribution of the audio fluctuation amplitude of the multimedia teaching audio data, at least two strong amplitude continuous similar audio intervals and at least two weak amplitude continuous similar audio intervals are obtained; according to the audio fluctuation similarity of each weak amplitude continuous similar audio interval, a reference amplitude continuous similar audio interval is screened out from all the weak amplitude continuous similar audio intervals;

[0008] The strong amplitude continuous similar audio interval and the reference amplitude continuous similar audio interval are used as the teaching audio interval to be tested; according to the grayscale distribution characteristics of the spectrogram of each teaching audio interval to be tested, the noise interference degree of each extreme point in each teaching audio interval to be tested is obtained; according to the noise interference degree and the distribution of the extreme points, the effective extreme points in the multimedia teaching audio data are screened out;

[0009] Performing empirical mode decomposition denoising processing according to the effective extreme value points in the multimedia teaching audio data to obtain multimedia teaching denoised audio data;

[0010] The multimedia teaching noise-removed video data is compressed and stored.

[0011] Furthermore, the method for obtaining the continuous similar audio intervals with strong amplitude includes:

[0012] Counting all extreme value points of the multimedia teaching audio data; taking the interval between two adjacent extreme value points as the amplitude interval; taking the amplitude range in each amplitude interval as the fluctuation amplitude of each amplitude interval; arranging all the fluctuation amplitudes in the time sequence of the corresponding amplitude intervals to obtain a fluctuation amplitude sequence;

[0013] Calculate the fluctuation amplitude mean corresponding to all fluctuation amplitudes, and in the fluctuation amplitude sequence, use the fluctuation amplitude less than the fluctuation amplitude mean as the weak reference amplitude, and use the fluctuation amplitude greater than the fluctuation amplitude mean as the strong reference amplitude; use the extreme point between the strong reference amplitude and the weak reference amplitude adjacent to it in the multimedia teaching audio data as the dividing point; divide the multimedia teaching audio data into at least two initial audio intervals through the dividing point;

[0014] An initial audio interval in which the number of strong reference amplitudes is greater than or equal to a preset continuous threshold is regarded as a strong amplitude continuous similar audio interval.

[0015] Furthermore, the method for obtaining the weak-amplitude continuous similar audio intervals includes:

[0016] An initial audio interval in which the number of weak reference amplitudes is greater than or equal to a preset continuous threshold is used as a weak amplitude continuous similar audio interval.

[0017] Furthermore, the method for obtaining the reference amplitude continuous similar audio intervals includes:

[0018] Sequentially taking each weak-amplitude continuous similar audio interval as a target audio interval, taking other weak-amplitude continuous similar audio intervals outside the target audio interval as comparison audio intervals, taking the audio data curve corresponding to the target audio interval as a target data curve, and taking the audio data curve corresponding to the comparison audio interval as a comparison data curve;

[0019] For any comparison data curve: obtain the matching data pairs between the data of the target data curve and the data of the comparison data curve and the corresponding DTW distance through the DTW algorithm; construct a rectangular coordinate system with the index value of the matching data pair as the horizontal coordinate and the DTW distance corresponding to the matching data pair as the vertical coordinate; perform curve fitting according to the coordinate points corresponding to each matching data pair in the rectangular coordinate system to obtain the corresponding DTW distance fluctuation curve; obtain the fluctuation similarity corresponding to the comparison data curve according to the slope distribution of the coordinate points of each matching data pair in the DTW distance fluctuation curve;

[0020] According to the distribution of fluctuation similarities corresponding to all comparison data curves, the overall fluctuation similarity corresponding to the target audio interval is obtained;

[0021] Among all weak amplitude continuous similar audio intervals, the weak amplitude continuous similar audio intervals whose overall fluctuation similarity is less than or equal to a preset fluctuation threshold are used as reference amplitude continuous similar audio intervals, and the preset fluctuation threshold is greater than 0.

[0022] Furthermore, the method for obtaining the fluctuation similarity includes:

[0023] The tangent slope of the coordinate point corresponding to each matching data pair in the DTW distance fluctuation curve is used as the reference slope corresponding to each matching data pair; the accumulated values ​​of the absolute values ​​of all reference slopes are negatively correlated to obtain the fluctuation similarity corresponding to the comparison data curve.

[0024] Furthermore, the calculation formula of the overall fluctuation similarity degree includes:

[0025]

[0026] Among them, D m is the overall fluctuation similarity corresponding to the target audio interval, m is the number of continuous similar audio intervals with weak amplitude, X i is the fluctuation similarity corresponding to the ith contrast audio interval, X max is the maximum value of fluctuation similarity in all contrasting audio intervals, X min is the minimum value of fluctuation similarity in all compared audio intervals, and Norm() is the normalization function.

[0027] Furthermore, the method for obtaining the noise interference degree includes:

[0028] The spectrogram of each teaching audio interval to be tested is used as the target spectrogram in turn;

[0029] According to the gray value of each pixel point in the target spectrogram, a segmentation threshold is obtained by using the maximum inter-class variance method; a connected domain composed of pixels whose gray value is greater than the segmentation threshold is used as a high-energy connected domain; the total area of ​​all high-energy connected domains and the number of high-energy connected domains in the target spectrogram are calculated; the ratio between the total area of ​​all high-energy connected domains corresponding to the target spectrogram and the number of high-energy connected domains is used as the degree of audio energy concentration corresponding to the target spectrogram;

[0030] Combine any two high-energy connected domains to obtain all high-energy connected domain binary groups; use the distance between the centroids of the two high-energy connected domains in each high-energy connected domain binary group as the reference distance corresponding to each high-energy connected domain binary group; and use the cumulative value of the reference distances corresponding to all high-energy connected domain binary groups as the audio energy discreteness of the target spectrogram;

[0031] The variance of all grayscale values ​​in the target spectrogram is taken as the grayscale change severity of the target spectrogram; the noise interference degree of each extreme point in the target spectrogram is obtained according to the audio energy concentration degree, the audio energy dispersion degree and the grayscale change severity, and the audio energy concentration degree is negatively correlated with the noise interference degree, and the audio energy dispersion degree and the grayscale change severity are both positively correlated with the noise interference degree.

[0032] Furthermore, the method for obtaining the effective extreme value points includes:

[0033] The extreme value points in the teaching audio interval to be tested are used as reference extreme value points; in the multimedia teaching audio data, a preset number of extreme value points of the same extreme value point type and nearest neighbors as each reference extreme value point are used as comparison extreme value points corresponding to each reference extreme value point; the amplitude difference between each reference extreme value point and each corresponding comparison extreme value point is calculated, and the sum of all amplitude differences corresponding to each reference extreme value point is used as the reference weight corresponding to each reference extreme value point;

[0034] According to the reference weight and the noise interference degree, a decomposition contribution degree corresponding to each reference extreme value point is obtained, wherein the reference weight is positively correlated with the decomposition contribution degree, and the noise interference degree is negatively correlated with the decomposition contribution degree;

[0035] The reference extreme value point whose decomposition contribution degree is greater than a preset effective threshold is taken as an effective extreme value point, and the preset effective threshold is greater than 0.

[0036] Furthermore, the method for obtaining the noise-removing frequency data for multimedia teaching includes:

[0037] According to the effective extreme points in the multimedia teaching audio data, empirical mode decomposition is performed to obtain the corresponding intrinsic mode function components; after wavelet denoising is used for each intrinsic mode function component, the audio data is reconstructed to obtain the multimedia teaching denoised audio data.

[0038] The present invention also proposes a smart Internet of Things multimedia teaching management system, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any one of the steps of a smart Internet of Things multimedia teaching management method.

[0039] The present invention has the following beneficial effects:

[0040] Considering that the noise data in the multimedia teaching audio data in the teaching environment can be roughly divided into two types of noise, one of which is relatively random noise, such as the noise of desks and chairs colliding; the other is relatively regular mechanical environmental noise, such as the noise generated by air conditioners, fans, and computer operation; and the human voice in the teaching environment, that is, the effective audio information, usually contains a large amount of energy and the information carried by the human voice is less repeated. Therefore, according to this characteristic, combined with the fact that noise is usually an isolated fluctuation characteristic, the embodiment of the present invention obtains a strong amplitude continuous similar audio interval and a reference amplitude continuous similar audio interval in a weak amplitude continuous similar audio interval through the distribution of the audio fluctuation amplitude of multimedia teaching, so that the subsequent screening of effective extreme points based on the further obtained teaching audio interval to be tested is more accurate and improves the subsequent audio denoising effect. Further, according to the characteristic that the energy carried by the spectrogram corresponding to the human voice in the teaching environment is usually large and relatively concentrated, the effective extreme points in the multimedia teaching audio data are obtained by combining the grayscale distribution characteristics of the spectrogram of the teaching audio interval to be tested with the distribution of extreme points; thereby, the accuracy of the empirical mode decomposition denoising process is improved according to the more accurate effective extreme points, so that the effect of denoising the multimedia teaching audio data is better, thereby improving the management effect of the multimedia teaching data of the Internet of Things. In summary, the present invention has a better management effect on the multimedia teaching data of the Internet of Things by obtaining effective extreme points for the method of empirical mode decomposition denoising. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 A flow chart of a smart Internet of Things multimedia teaching management method provided by an embodiment of the present invention;

[0043] Figure 2 A flowchart of a multimedia teaching audio data denoising method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the specific implementation method, structure, features and effects of a smart Internet of Things multimedia teaching management method and system proposed by the present invention in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0045] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0046] A smart Internet of Things multimedia teaching management method and system embodiment:

[0047] The specific scheme of a smart Internet of Things multimedia teaching management method and system provided by the present invention is described in detail below with reference to the accompanying drawings.

[0048] See also Figure 1 , which shows a flow chart of a smart Internet of Things multimedia teaching management method provided by an embodiment of the present invention, the method comprising:

[0049] Step S1: Acquire multimedia teaching audio data.

[0050] The embodiment of the present invention aims to provide a method and system for managing multimedia teaching in a smart Internet of Things. The method performs audio denoising on multimedia audio data by an audio noise filtering method to obtain denoised multimedia teaching denoised audio data, and further compresses and stores the denoised multimedia teaching audio data. Therefore, it is first necessary to obtain the target of audio noise filtering, that is, multimedia teaching audio data.

[0051] The embodiment of the present invention first obtains multimedia teaching audio data. Considering that in a teaching environment, effective audio data is mainly audio data when teachers are teaching, the embodiment of the present invention collects multimedia teaching audio data through a microphone device. It should be noted that the implementer can also collect multimedia teaching audio data through other audio data collection devices, which will not be further elaborated here.

[0052] Step S2: According to the distribution of audio fluctuation amplitude of multimedia teaching audio data, at least two strong amplitude continuous similar audio intervals and at least two weak amplitude continuous similar audio intervals are obtained; according to the audio fluctuation similarity of each weak amplitude continuous similar audio interval, a reference amplitude continuous similar audio interval among all weak amplitude continuous similar audio intervals is screened out.

[0053] Taking into account the teaching environment, the energy contained in the human voice, that is, the effective audio information in the multimedia teaching audio data is usually large, and the audio information corresponding to the human voice is usually continuous. Therefore, the relatively isolated audio information in the multimedia teaching audio data is usually noise, and the audio interval with large and continuous audio amplitude is usually the effective information audio interval corresponding to the human voice.

[0054] According to the distribution of the audio fluctuation amplitude of the multimedia teaching audio data, the embodiment of the present invention obtains at least two continuous similar audio intervals with strong amplitude and at least two continuous similar audio intervals with weak amplitude. The continuous similar audio interval with strong amplitude is the audio information corresponding to the human voice with large sound energy and continuous sound; the continuous similar audio interval with weak amplitude is the audio information corresponding to the human voice with small sound energy but continuous sound.

[0055] Preferably, the method for obtaining the continuous similar audio intervals with strong amplitude includes:

[0056] Count all extreme points of multimedia teaching audio data; take the interval between two adjacent extreme points as the amplitude interval; take the amplitude range in each amplitude interval as the fluctuation amplitude of each amplitude interval; arrange all fluctuation amplitudes in the time order of the corresponding amplitude interval to obtain a fluctuation amplitude sequence. For audio data, the amplitude can represent the energy, volume and other characteristic information of the audio signal. Therefore, the fluctuation amplitude sequence can represent the corresponding audio information changes of multimedia teaching audio data, and further select the corresponding audio information according to the strong and continuous amplitude characteristics corresponding to the effective human voice information.

[0057] Calculate the mean of the fluctuation amplitudes corresponding to all fluctuation amplitudes. In the fluctuation amplitude sequence, the fluctuation amplitude less than the mean of the fluctuation amplitude is used as the weak reference amplitude, and the fluctuation amplitude greater than the mean of the fluctuation amplitude is used as the strong reference amplitude. It should be noted that the implementer can also divide the weak reference amplitude and the strong reference amplitude by setting a specific amplitude threshold, which will not be further elaborated here.

[0058] The extreme point between the strong reference amplitude and the adjacent weak reference amplitude in the multimedia teaching audio data is used as a dividing point; the multimedia teaching audio data is divided into at least two initial audio intervals through the dividing point. That is, each initial audio interval only contains a strong reference amplitude or a weak reference amplitude. Further, the continuity of the number of reference amplitudes in each initial audio interval is analyzed. In the embodiment of the present invention, the initial audio interval in which the number of strong reference amplitudes is greater than or equal to the preset continuous threshold is used as a strong amplitude continuous similar audio interval. In the embodiment of the present invention, the preset continuous threshold is set to 3, and the implementer can set it according to the specific implementation environment. Since the reference amplitude types in the initial audio interval are the same, when the number of strong reference amplitudes in the initial audio interval is greater than or equal to the preset continuous threshold, it means that there is audio information with a large and continuous audio amplitude in the corresponding initial audio interval, which corresponds to the human voice audio information required by the embodiment of the present invention.

[0059] Preferably, the method for acquiring weak-amplitude continuous similar audio intervals includes:

[0060] Since the relatively isolated audio information in multimedia teaching audio data is usually noise, and there may also be effective audio information of human voice with small energy, that is, small audio amplitude, the embodiment of the present invention takes the initial audio interval in which the number of weak reference amplitudes is greater than a preset continuous threshold as a weak amplitude continuous similar audio interval, and performs further analysis on the basis of the weak amplitude continuous similar audio interval.

[0061] As for the audio noise in multimedia teaching audio data, due to scene restrictions, the corresponding noise is usually divided into two types, one of which is relatively random noise, such as the noise caused by the collision of tables and chairs, random noise of the external environment, etc.; the other is relatively regular mechanical environmental noise, such as the sound of the machine when it is running, such as the noise caused by air conditioners, fans, computers and other machine equipment. The process of obtaining weak amplitude continuous similar audio intervals and strong amplitude continuous similar audio intervals has screened out some isolated random noises, and for the more regular mechanical environmental noise, the embodiment of the present invention screens out the reference amplitude continuous similar audio intervals in all weak amplitude continuous similar audio intervals according to the similarity of the audio fluctuations of each weak amplitude continuous similar audio interval, and analyzes the weak amplitude continuous similar audio intervals through the characteristics of the mechanical environmental noise having a small amplitude and relatively similar fluctuations, to obtain the reference amplitude continuous similar audio intervals.

[0062] Preferably, the method for obtaining the continuous similar audio intervals of reference amplitude includes:

[0063] Each weak amplitude continuous similar audio interval is sequentially used as a target audio interval, other weak amplitude continuous similar audio intervals outside the target audio interval are used as comparison audio intervals, the audio data curve corresponding to the target audio interval is used as a target data curve, and the audio data curve corresponding to the comparison audio interval is used as a comparison data curve. The target data curve and the comparison data curve are data curves corresponding to the corresponding weak amplitude continuous similar audio interval in the multimedia teaching audio data.

[0064] For any comparison data curve: obtain the matching data pair between the data of the target data curve and the data of the comparison data curve and the corresponding DTW distance through the DTW algorithm. Considering that the information carried by the human voice is less repeated, and the fluctuations of the mechanical environmental noise are relatively similar, the embodiment of the present invention calculates the similarity between the target data curve and each comparison data curve through the dynamic time warping algorithm (Dynamic Time Warping, DTW) algorithm, and further determines whether the weak amplitude continuous similar audio interval corresponding to the target data curve contains only mechanical environmental noise data. In addition, it should be noted that when the data in the matching data pair meets the one-to-many situation in the DTW algorithm, for example, the data a in the target data curve corresponds to the data b, c, d in the comparison data curve, then the corresponding matching data pairs are matching data pair ab, matching data pair ac and matching data pair ad respectively.

[0065] A rectangular coordinate system is constructed with the index value of the matching data pair as the horizontal coordinate and the DTW distance corresponding to the matching data pair as the vertical coordinate; a curve fitting is performed according to the coordinate points corresponding to each matching data pair in the rectangular coordinate system to obtain the corresponding DTW distance fluctuation curve. When the target data curve is mechanical environmental noise data, it means that the similarity between the target data curve and the other comparison data curves representing the mechanical environmental noise data is high, the corresponding DTW distance is small and relatively similar, and the corresponding slope distribution mapped to the rectangular coordinate system is relatively stable and overall small; on the contrary, when the target data curve contains some human voice data, since it does not meet the similarity characteristics between mechanical environmental noise data, the overall similarity between the corresponding target data curve and the other comparison data curves is low, the corresponding DTW distance is large and the similarity is small, and the corresponding slope distribution mapped to the rectangular coordinate system is relatively chaotic and overall large. Therefore, in order to further screen out data containing only mechanical environmental noise, the embodiment of the present invention obtains the fluctuation similarity corresponding to the comparison data curve according to the slope distribution of the coordinate points of each matching data pair in the DTW distance fluctuation curve.

[0066] Preferably, the method for obtaining the fluctuation similarity includes:

[0067] The tangent slope of the corresponding coordinate point of each matching data pair in the DTW distance fluctuation curve is used as the reference slope corresponding to each matching data pair; the cumulative value of the absolute values ​​of all reference slopes is negatively correlated to obtain the fluctuation similarity corresponding to the comparison data curve. The larger the absolute value of the slope of the corresponding coordinate point of the matching data pair is as a whole, the more drastic the change of the corresponding DTW distance fluctuation curve is, that is, the lower the similarity of the weak amplitude continuous similar audio interval between the target data curve and the comparison data curve is. Therefore, the greater the fluctuation similarity after the negative correlation mapping of the cumulative value of the absolute values ​​of all reference slopes, the more similar the weak amplitude continuous similar audio interval between the corresponding target data curve and the comparison data curve is. In the embodiment of the present invention, the negative correlation mapping is performed by an exponential function with a natural constant as the base, and the implementer can also perform negative correlation mapping by other methods according to the specific implementation environment, such as the inverse, etc., which will not be further elaborated here.

[0068] In the embodiment of the present invention, each comparison data curve is sequentially used as the ath comparison data curve, and the method for obtaining the fluctuation similarity of the ath comparison data curve is expressed in the formula as follows:

[0069]

[0070] Among them, X a is the fluctuation similarity of the a-th comparison data curve, N is the number of matching data pairs corresponding to the target data curve and the a-th comparison data curve, k j is the reference slope of the j-th matching data pair corresponding to the target data curve and the a-th comparison data curve, || is the absolute value symbol, and exp() is an exponential function with a natural constant as the base.

[0071] Since a single fluctuation similarity can only characterize the similarity between the target audio interval and one of the comparison audio intervals, in order to further characterize the corresponding to the target audio interval. According to the distribution of fluctuation similarities corresponding to all comparison data curves, the overall fluctuation similarity corresponding to the target audio interval is obtained;

[0072] Preferably, the calculation formula for the overall fluctuation similarity includes:

[0073]

[0074] Among them, D m is the overall fluctuation similarity corresponding to the target audio interval, m is the number of continuous similar audio intervals with weak amplitude, X i is the fluctuation similarity corresponding to the ith contrast audio interval, X max is the maximum value of fluctuation similarity in all compared audio intervals, Xmi nis the minimum value of fluctuation similarity in all compared audio intervals, and Norm() is a normalization function. In an embodiment of the present invention, the overall fluctuation similarity, that is, all fluctuation similarities corresponding to the target audio interval, are averaged after subtracting the maximum and minimum values, so that the extreme value range of the data set of fluctuation similarity after averaging is reduced, and the interference of outliers is reduced. In an embodiment of the present invention, all normalization methods adopt linear normalization, and the implementer can also choose the normalization method according to the specific implementation environment. And because the overall fluctuation similarity is the calculation formula of the mean, the greater the corresponding overall fluctuation similarity, the more similar the weak amplitude continuous similar audio interval corresponding to the target audio interval is to other weak amplitude continuous similar audio intervals, that is, the weak amplitude continuous similar audio interval corresponding to the target audio interval is more likely to represent the mechanical environment noise data information, and the more it should be screened out.

[0075] Since the greater the overall fluctuation similarity, the more likely the corresponding weak amplitude continuous similar audio interval is to contain only information about mechanical environmental noise data, the embodiment of the present invention uses the weak amplitude continuous similar audio interval whose overall fluctuation similarity is less than or equal to the preset fluctuation threshold among all the weak amplitude continuous similar audio intervals as the reference amplitude continuous similar audio interval, and the preset fluctuation threshold is greater than 0. In the embodiment of the present invention, the preset fluctuation threshold is set to 0.32, and the implementer can set it according to the specific implementation environment.

[0076] Step S3: taking the strong amplitude continuous similar audio interval and the reference amplitude continuous similar audio interval as the teaching audio interval to be tested; obtaining the noise interference degree of each extreme point in each teaching audio interval to be tested according to the grayscale distribution characteristics of the spectrogram of each teaching audio interval to be tested; and screening out the effective extreme points in the multimedia teaching audio data according to the noise interference degree and the distribution of the extreme points.

[0077] At this point, a strong amplitude continuous similar audio interval with a strong and continuous amplitude corresponding to the effective information of the human voice is obtained, as well as a reference amplitude continuous similar audio interval corresponding to the weak amplitude continuous similar audio interval after the pure mechanical environmental noise data information has been screened out. Further analysis is performed on the basis of these audio intervals, which is less affected by noise interference and has a smaller corresponding amount of calculation. Therefore, the embodiment of the present invention uses the strong amplitude continuous similar audio interval and the reference amplitude continuous similar audio interval as the teaching audio interval to be tested. The teaching audio interval to be tested is the audio interval after deleting part of the random noise and part of the mechanical environmental noise data information, that is, the audio interval corresponding to most of the human voice information.

[0078] Further analysis is performed on the basis of the interval of the teaching audio to be tested, considering that the spectrogram can characterize the information carried in the wave graph, wherein the abscissa of the spectrogram is time and the ordinate is frequency, and further according to the spectrogram of each interval of the teaching audio to be tested, the degree of interference of the extreme points in the corresponding interval by noise is analyzed. For the gray value in the spectrogram, the larger the gray value, the larger the energy value carried by the corresponding pixel point. The energy carried by the human voice information is relatively large relative to the noise, so the gray value corresponding to the human voice information is usually relatively large. The noise in the spectrogram usually appears as discontinuous energy distribution or irregular frequency components. When the noise in the interval of the teaching audio to be tested is strong, that is, the extreme points of the corresponding interval of the teaching audio to be tested are more strongly interfered by the noise, the audio information corresponding to the noise will be covered or mixed in large quantities between the frequency components of the human voice signal, so that the distribution of the pixels corresponding to the human voice information is relatively discrete. Therefore, according to this feature, the embodiment of the present invention obtains the degree of noise interference of each extreme point in each interval of the teaching audio to be tested according to the gray distribution characteristics of the spectrogram of each interval of the teaching audio to be tested.

[0079] Preferably, the method for obtaining the noise interference degree includes:

[0080] The spectrogram of each teaching audio interval to be tested is taken as the target spectrogram in turn, and the target spectrogram is further analyzed to complete the analysis of the spectrograms of all teaching audio intervals to be tested.

[0081] According to the gray value of each pixel in the target spectrogram, the segmentation threshold is obtained by the maximum inter-class variance method; the connected domain composed of pixels with gray values ​​greater than the segmentation threshold is used as a high-energy connected domain. In the embodiment of the present invention, all spectrograms are grayscale spectrograms, in which each pixel corresponds to a gray value. Considering that the gray values ​​corresponding to the human voice information in different teaching audio intervals to be tested are usually different, but the energy corresponding to the human voice information is usually larger than the noise, the embodiment of the present invention obtains the segmentation threshold corresponding to each spectrogram through the maximum inter-class variance method, and realizes the adaptive division of the pixels of each spectrogram through the segmentation threshold. It should be noted that the maximum inter-class variance method is a prior art well known to those skilled in the art, and will not be further described here.

[0082] Since the greater the impact of noise on the spectrogram, the more discrete the distribution of the corresponding pixel points representing the human voice information that are greater than the segmentation threshold, the greater the noise, the smaller the area of ​​the corresponding connected domain and the greater the number. In order to further characterize the degree to which the target spectrogram is affected by noise, the embodiment of the present invention calculates the total area of ​​all high-energy connected domains and the number of high-energy connected domains in the target spectrogram; the ratio between the total area of ​​all high-energy connected domains corresponding to the target spectrogram and the number of high-energy connected domains is used as the degree of audio energy concentration corresponding to the target spectrogram. That is, the greater the corresponding audio energy concentration, the greater the total area of ​​the high-energy connected domain and the smaller the number, and the less the corresponding target spectrogram is affected by noise, that is, the less the extreme points in the teaching audio interval to be tested corresponding to the target spectrogram are disturbed by noise.

[0083] Since the less the target spectrogram is affected by noise, the more concentrated the distribution of the pixels representing the human voice information is, that is, the more concentrated the distribution of the corresponding high-energy connected domains is. Considering that the concentration of high-energy connected domains can be characterized by distance, the embodiment of the present invention combines any two high-energy connected domains to obtain all high-energy connected domain tuples; the distance between the centroids of the two high-energy connected domains in each high-energy connected domain tuple is used as the reference distance corresponding to each high-energy connected domain tuple; the reference distance accumulation value corresponding to all high-energy connected domain tuples is used as the audio energy discreteness of the target spectrogram. When the target spectrogram is more affected by noise, the distribution of high-energy connected domains is more discrete and the number is greater, so the corresponding reference distance is larger, and the number of accumulations is larger, that is, the corresponding audio energy discreteness is larger, that is, the extreme value points in the target spectrogram are more affected by noise. It should be noted that when there is only one high-energy connected domain in the target spectrogram, it means that the distribution of pixel points corresponding to the human voice information is very concentrated, that is, the impact of noise is very small. Since there are no other high-energy connected domains, there is no reference distance. The embodiment of the present invention sets the audio energy discreteness of the target spectrogram with only one high-energy connected domain to 0.

[0084] Further considering that the less the teaching audio interval to be tested is affected by noise interference, the more uniform the grayscale value distribution in the corresponding spectrogram is, that is, the less interference the extreme points therein are caused by noise, the uniformity of the grayscale distribution of all pixels in the target spectrogram is further analyzed. In the embodiment of the present invention, the variance of all grayscale values ​​in the target spectrogram is used as the degree of grayscale change of the target spectrogram. The larger the corresponding grayscale change distance, the more chaotic the grayscale value distribution in the target spectrogram is. The implementer can also characterize the degree of grayscale change through parameters such as standard deviation that characterize the uniformity, which will not be further elaborated here.

[0085] In combination with the relationship between the degree of audio energy concentration, the degree of audio energy dispersion, the degree of grayscale change and the influence of noise, the embodiment of the present invention further obtains the degree of noise interference of each extreme point in the target spectrogram according to the degree of audio energy concentration, the degree of audio energy dispersion and the degree of grayscale change. The degree of audio energy concentration is negatively correlated with the degree of noise interference, and the degree of audio energy dispersion and the degree of grayscale change are positively correlated with the degree of noise interference.

[0086] In the embodiment of the present invention, the method for obtaining the noise interference degree of each extreme point in the u-th spectrogram corresponding to the target spectrogram is expressed in the formula as follows:

[0087]

[0088] Among them, Z u is the noise interference degree of each extreme point in the target spectrogram, G u is the intensity of grayscale change of the target spectrogram, S u is the total area of ​​all high-energy connected domains in the target spectrogram, M u is the number of high-energy connected domains in the target spectrogram, R is the number of high-energy connected domain tuples corresponding to the target spectrogram, and l v is the reference distance of the vth high-energy connected domain binary in the target spectrogram, exp() is an exponential function with a natural constant as the base, and Norm() is a normalization function; is the audio energy concentration degree corresponding to the target spectrogram, is the audio energy discreteness of the target spectrogram. It should be noted that the implementer can also obtain the corresponding noise interference degree according to the audio energy concentration degree, the audio energy discreteness and the grayscale change intensity through other forms of formulas; for example, multiplying the energy discreteness with the grayscale change intensity, multiplying the product with the negative correlation mapping value of the audio energy concentration degree and normalizing it to obtain the noise interference degree, which will not be further elaborated here.

[0089] After obtaining the noise interference degree of each extreme point in each teaching audio interval to be tested, the effective extreme points are further screened according to the noise interference degree. The embodiment of the present invention screens out the effective extreme points in the multimedia teaching audio data according to the noise interference degree and the distribution of the extreme points.

[0090] Preferably, the method for obtaining effective extreme value points includes:

[0091] The extreme value points in the teaching audio interval to be tested are used as reference extreme value points; in the multimedia teaching audio data, a preset number of extreme value points of the same extreme value point type and nearest neighbor to each reference extreme value point are used as comparison extreme value points corresponding to each reference extreme value point. In the embodiment of the present invention, the preset number is set to 2, and there are two types of extreme value points in the embodiment of the present invention, namely, maximum value points and minimum value points, that is, when the reference extreme value point is a maximum value, the corresponding comparison extreme value point is also a maximum value, and the effectiveness of the reference extreme value point is further judged by comparing it with the extreme value points of the same type and nearest neighbor.

[0092] The amplitude difference between each reference extreme point and each corresponding comparison extreme point is calculated, and the sum of all amplitude differences corresponding to each reference extreme point is used as the reference weight corresponding to each reference extreme point. When the reference extreme point is less affected by noise, the interference degree of noise in its adjacent interval is smaller, and the difference between the corresponding adjacent extreme points of the same type is usually smaller. Therefore, the embodiment of the present invention introduces reference weights to screen effective extreme points according to the distribution of extreme points.

[0093] According to the reference weight and the noise interference degree, the decomposition contribution degree corresponding to each reference extreme value point is obtained, the reference weight is positively correlated with the decomposition contribution degree, and the noise interference degree is negatively correlated with the decomposition contribution degree. In an embodiment of the present invention, the normalized value of the product between the negative correlation mapping value of the noise interference degree of the reference extreme value point and the reference weight is used as the decomposition contribution degree. The greater the corresponding decomposition contribution degree, the greater the contribution degree of the corresponding reference extreme value point when performing EMD decomposition, that is, the more likely it is to be a valid extreme value point. It should be noted that the implementer can also obtain the decomposition contribution degree by other methods, such as taking the normalized value of the ratio between the reference weight and the noise interference degree as the decomposition contribution degree, which will not be further elaborated here.

[0094] In the embodiment of the present invention, each reference extreme value point is sequentially used as the bth reference extreme value point, and the method for obtaining the decomposition contribution degree of the bth reference extreme value point is expressed in the formula as follows:

[0095] F b =Norm((Δf b1 +Δf b2 )×(1-Z b ))

[0096] Among them, F b is the decomposition contribution of the bth reference extreme point, Δf b1 is the amplitude difference of the first comparison extreme point corresponding to the bth reference extreme point, Δf b2 is the amplitude difference of the second comparison extreme point corresponding to the bth reference extreme point, Z bis the noise interference degree corresponding to the bth reference extreme point, and Norm() is the normalization function.

[0097] Considering that the greater the decomposition contribution, the more likely the corresponding reference extreme point is to be a valid extreme point, the embodiment of the present invention uses the reference extreme point whose decomposition contribution is greater than the preset effective threshold as a valid extreme point, and the preset effective threshold is greater than 0. In the embodiment of the present invention, the preset effective threshold is set to 0.5, and the implementer can set it according to the specific environment.

[0098] Step S4: Performing empirical mode decomposition denoising processing according to the effective extreme value points in the multimedia teaching audio data to obtain multimedia teaching denoised audio data.

[0099] After obtaining the effective extreme value points in the multimedia teaching audio data, the multimedia teaching audio data is further denoised according to the effective extreme value points. The embodiment of the present invention performs empirical mode decomposition denoising according to the effective extreme value points in the multimedia teaching audio data to obtain multimedia teaching denoised audio data. It should be noted that empirical mode decomposition is a prior art well known to those skilled in the art and will not be further limited or elaborated herein.

[0100] Preferably, the method for obtaining the noise-removing frequency data for multimedia teaching includes:

[0101] According to the effective extreme points in the multimedia teaching audio data, empirical mode decomposition is performed to obtain the corresponding intrinsic mode function components; after wavelet denoising is used for each intrinsic mode function component, the audio data is reconstructed to obtain the multimedia teaching denoised audio data. In the embodiment of the present invention, the number of layers of wavelet decomposition in wavelet denoising is set to 7 layers, and the implementer can set it according to the specific implementation environment. It should be noted that the method of obtaining the intrinsic mode function components by performing empirical mode decomposition according to the extreme points, the wavelet denoising method and the method of reconstructing the audio data are prior arts well known to those skilled in the art, and are not further limited or elaborated here.

[0102] Step S5: compress and store the multimedia teaching noise-removed video data.

[0103] After obtaining the denoised multimedia teaching denoised audio data, the multimedia teaching denoised audio data is further compressed and stored, so as to perform IoT multimedia teaching management. In an embodiment of the present invention, AAC lossy compression is used to encode and compress the acquired multimedia teaching denoised audio data, and the compressed audio data is transmitted to the cloud storage service platform using the MQTT IoT protocol. It should be noted that the implementer can compress and store the multimedia teaching denoised audio data by other methods according to the specific implementation environment, which will not be further elaborated here.

[0104] In summary, the present invention analyzes the distribution of audio fluctuation amplitude and audio fluctuation similarity of multimedia teaching audio data to obtain the teaching audio interval to be tested containing human voice information; further analyzes the grayscale distribution of the spectrogram corresponding to the teaching audio interval to be tested to obtain the noise interference degree of the extreme points in each teaching audio interval to be tested; performs empirical mode decomposition denoising on the multimedia teaching audio data according to the effective extreme points obtained from the noise interference degree and the distribution of extreme points to obtain multimedia teaching denoised audio data, and finally compresses and stores the multimedia teaching denoised audio data, thereby performing multimedia teaching management of the Internet of Things. The present invention has a better management effect on the multimedia teaching data of the Internet of Things by obtaining the effective extreme points for empirical mode decomposition denoising.

[0105] The present invention also proposes a smart Internet of Things multimedia teaching management system, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any one of the steps of a smart Internet of Things multimedia teaching management method.

[0106] A method for denoising multimedia teaching audio data:

[0107] The existing method for denoising multimedia teaching audio data is mainly to decompose the collected multimedia teaching audio data into different intrinsic mode function components through the empirical mode decomposition method, denoise each intrinsic mode function component through wavelet denoising, reconstruct the denoised intrinsic mode function components, and denoise the multimedia teaching audio data. However, when the empirical mode decomposition is used to decompose the collected multimedia teaching audio data, it is greatly affected by the extreme points in the multimedia teaching audio data. The existence of extreme points caused by noise will lead to inaccurate extraction of intrinsic mode function components or the generation of pseudo-modes, reducing the accuracy of the empirical mode decomposition, thereby making the denoising effect of the multimedia teaching audio data poor. In order to solve the problem that the prior art has poor effect of denoising the collected audio data through the empirical mode decomposition method, an embodiment of the present invention provides a method for denoising multimedia teaching audio data. Please refer to Figure 2 , which shows a flow chart of a method for denoising multimedia teaching audio data, the method comprising:

[0108] Step S01: Acquire multimedia teaching audio data;

[0109] Step S02: according to the distribution of the audio fluctuation amplitude of the multimedia teaching audio data, at least two strong amplitude continuous similar audio intervals and at least two weak amplitude continuous similar audio intervals are obtained; according to the audio fluctuation similarity of each weak amplitude continuous similar audio interval, a reference amplitude continuous similar audio interval is screened out from all the weak amplitude continuous similar audio intervals;

[0110] Step S03: taking the strong amplitude continuous similar audio interval and the reference amplitude continuous similar audio interval as the teaching audio interval to be tested; obtaining the noise interference degree of each extreme point in each teaching audio interval to be tested according to the grayscale distribution characteristics of the spectrogram of each teaching audio interval to be tested; and screening out the effective extreme points in the multimedia teaching audio data according to the noise interference degree and the distribution of the extreme points;

[0111] Step S04: Performing empirical mode decomposition denoising processing according to the effective extreme value points in the multimedia teaching audio data to obtain multimedia teaching denoised audio data.

[0112] Among them, steps S01 to S04 have been described in detail in the above-mentioned smart Internet of Things multimedia teaching management method and system embodiment, and will not be repeated here.

[0113] This multimedia teaching audio data denoising method relates to the technical field of audio noise filtering. The method takes into account that in the teaching environment, the corresponding multimedia teaching audio data can be roughly divided into two types of noise, one of which is the relatively random noise of table and chair collision; the other is relatively regular mechanical noise, such as the noise generated by air conditioning, fans, and computer operation; and the human voice in the teaching environment, that is, the effective audio information, usually contains a large amount of energy and the information carried by the human voice is less repeated. Therefore, according to this characteristic, combined with the fact that noise is usually an isolated fluctuation characteristic, the embodiment of the present invention obtains a strong amplitude continuous similar audio interval and a reference amplitude continuous similar audio interval in a weak amplitude continuous similar audio interval through the distribution of the audio fluctuation amplitude of multimedia teaching, so that the subsequent screening of effective extreme points based on the further obtained teaching audio interval to be tested is more accurate and improves the subsequent audio denoising effect. Furthermore, based on the characteristic that the energy carried by the spectrogram corresponding to the human voice in the teaching environment is usually large and concentrated, the effective extreme points in the multimedia teaching audio data are obtained by combining the grayscale distribution characteristics of the spectrogram of the teaching audio interval to be tested with the distribution of extreme points; thereby improving the accuracy of the empirical mode decomposition denoising processing based on the more accurate effective extreme points, so that the denoising effect of the multimedia teaching audio data is better.

[0114] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0115] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

Claims

1. A smart Internet of Things multimedia teaching management method, characterized in that: The method comprises: Obtain multimedia teaching audio data; According to the distribution of the audio fluctuation amplitude of the multimedia teaching audio data, at least two strong amplitude continuous similar audio intervals and at least two weak amplitude continuous similar audio intervals are obtained; according to the audio fluctuation similarity of each weak amplitude continuous similar audio interval, a reference amplitude continuous similar audio interval is screened out from all the weak amplitude continuous similar audio intervals; The strong amplitude continuous similar audio interval and the reference amplitude continuous similar audio interval are used as the teaching audio interval to be tested; according to the grayscale distribution characteristics of the spectrogram of each teaching audio interval to be tested, the noise interference degree of each extreme point in each teaching audio interval to be tested is obtained; according to the noise interference degree and the distribution of the extreme points, the effective extreme points in the multimedia teaching audio data are screened out; Performing empirical mode decomposition denoising processing according to the effective extreme value points in the multimedia teaching audio data to obtain multimedia teaching denoised audio data; Compressing and storing the multimedia teaching noise-removed video data; The method for obtaining the strong amplitude continuous similar audio interval comprises: Counting all extreme value points of the multimedia teaching audio data; taking the interval between two adjacent extreme value points as the amplitude interval; taking the amplitude range in each amplitude interval as the fluctuation amplitude of each amplitude interval; arranging all the fluctuation amplitudes in the time sequence of the corresponding amplitude intervals to obtain a fluctuation amplitude sequence; Calculate the fluctuation amplitude mean corresponding to all fluctuation amplitudes, and in the fluctuation amplitude sequence, use the fluctuation amplitude less than the fluctuation amplitude mean as the weak reference amplitude, and use the fluctuation amplitude greater than the fluctuation amplitude mean as the strong reference amplitude; use the extreme point between the strong reference amplitude and the weak reference amplitude adjacent to it in the multimedia teaching audio data as the dividing point; divide the multimedia teaching audio data into at least two initial audio intervals through the dividing point; The initial audio interval in which the number of strong reference amplitudes is greater than or equal to a preset continuous threshold is used as a strong amplitude continuous similar audio interval; The method for obtaining the weak-amplitude continuous similar audio intervals comprises: The initial audio interval in which the number of weak reference amplitudes is greater than or equal to a preset continuous threshold is used as a weak amplitude continuous similar audio interval; The method for obtaining the continuous similar audio intervals of the reference amplitude includes: Sequentially taking each weak-amplitude continuous similar audio interval as a target audio interval, taking other weak-amplitude continuous similar audio intervals outside the target audio interval as comparison audio intervals, taking the audio data curve corresponding to the target audio interval as a target data curve, and taking the audio data curve corresponding to the comparison audio interval as a comparison data curve; For any comparison data curve: obtain the matching data pairs between the data of the target data curve and the data of the comparison data curve and the corresponding DTW distance through the DTW algorithm; construct a rectangular coordinate system with the index value of the matching data pair as the horizontal coordinate and the DTW distance corresponding to the matching data pair as the vertical coordinate; perform curve fitting according to the coordinate points corresponding to each matching data pair in the rectangular coordinate system to obtain the corresponding DTW distance fluctuation curve; obtain the fluctuation similarity corresponding to the comparison data curve according to the slope distribution of the coordinate points of each matching data pair in the DTW distance fluctuation curve; According to the distribution of fluctuation similarities corresponding to all comparison data curves, the overall fluctuation similarity corresponding to the target audio interval is obtained; Among all weak amplitude continuous similar audio intervals, the weak amplitude continuous similar audio intervals whose overall fluctuation similarity is less than or equal to a preset fluctuation threshold are used as reference amplitude continuous similar audio intervals, and the preset fluctuation threshold is greater than 0; The method for obtaining the noise interference degree includes: The spectrogram of each teaching audio interval to be tested is used as the target spectrogram in turn; According to the gray value of each pixel point in the target spectrogram, a segmentation threshold is obtained by using the maximum inter-class variance method; a connected domain composed of pixels whose gray value is greater than the segmentation threshold is used as a high-energy connected domain; the total area of ​​all high-energy connected domains and the number of high-energy connected domains in the target spectrogram are calculated; the ratio between the total area of ​​all high-energy connected domains corresponding to the target spectrogram and the number of high-energy connected domains is used as the degree of audio energy concentration corresponding to the target spectrogram; Combine any two high-energy connected domains to obtain all high-energy connected domain binary groups; use the distance between the centroids of the two high-energy connected domains in each high-energy connected domain binary group as the reference distance corresponding to each high-energy connected domain binary group; and use the cumulative value of the reference distances corresponding to all high-energy connected domain binary groups as the audio energy discreteness of the target spectrogram; The variance of all grayscale values ​​in the target spectrogram is used as the grayscale change intensity of the target spectrogram; the noise interference degree of each extreme point in the target spectrogram is obtained according to the audio energy concentration degree, the audio energy dispersion degree and the grayscale change intensity, the audio energy concentration degree is negatively correlated with the noise interference degree, and the audio energy dispersion degree and the grayscale change intensity are both positively correlated with the noise interference degree; The method for obtaining the effective extreme value point comprises: The extreme value points in the teaching audio interval to be tested are used as reference extreme value points; in the multimedia teaching audio data, a preset number of extreme value points of the same extreme value point type and nearest neighbors as each reference extreme value point are used as comparison extreme value points corresponding to each reference extreme value point; the amplitude difference between each reference extreme value point and each corresponding comparison extreme value point is calculated, and the sum of all amplitude differences corresponding to each reference extreme value point is used as the reference weight corresponding to each reference extreme value point; According to the reference weight and the noise interference degree, a decomposition contribution degree corresponding to each reference extreme value point is obtained, wherein the reference weight is positively correlated with the decomposition contribution degree, and the noise interference degree is negatively correlated with the decomposition contribution degree; The reference extreme value point whose decomposition contribution degree is greater than a preset effective threshold is taken as an effective extreme value point, and the preset effective threshold is greater than 0.

2. According to claim 1, a smart Internet of Things multimedia teaching management method is characterized in that: The method for obtaining the fluctuation similarity includes: The tangent slope of the coordinate point corresponding to each matching data pair in the DTW distance fluctuation curve is used as the reference slope corresponding to each matching data pair; the accumulated values ​​of the absolute values ​​of all reference slopes are negatively correlated to obtain the fluctuation similarity corresponding to the comparison data curve.

3. According to claim 1, a smart Internet of Things multimedia teaching management method is characterized in that: The calculation formula of the overall fluctuation similarity degree includes: Among them, D m is the overall fluctuation similarity corresponding to the target audio interval, m is the number of continuous similar audio intervals with weak amplitude, X i is the fluctuation similarity corresponding to the ith contrast audio interval, X max is the maximum value of fluctuation similarity in all contrasting audio intervals, X min is the minimum value of fluctuation similarity in all contrasting audio intervals, and Norm() is the normalization function.

4. The intelligent Internet of Things multimedia teaching management method according to claim 1 is characterized in that: The method for obtaining the denoised frequency data of multimedia teaching comprises: According to the effective extreme value points in the multimedia teaching audio data, empirical mode decomposition is performed to obtain the corresponding intrinsic mode function components; after wavelet denoising is used for each intrinsic mode function component, the audio data is reconstructed to obtain the multimedia teaching denoised audio data.

5. A smart Internet of Things multimedia teaching management system, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.