Multi-source teaching data processing method based on deep learning

Abstract

The invention relates to a multi-source teaching data processing method based on deep learning, and the method comprises the steps: obtaining multi-modal data which comprises text data, audio data and video data; preprocessing the multi-modal data by adopting a modal separation technology to obtain a plurality of independent feature sets; generating a fused feature set by adopting a feature fusion algorithm according to the plurality of independent feature sets; for the fused feature set, determining a feature extraction range by adopting a convolution adjustment algorithm; processing the feature extraction range through a multi-scale feature fusion network to obtain high-dimensional feature space representation; generating a clear feature set by adopting a noise suppression algorithm according to the high-dimensional feature space representation; aiming at the clear feature set, determining difficulty gradient division by adopting a difficulty perception model; difficulty gradient division is processed through a network optimization technology, and a hierarchy division scheme with real-time optimization is obtained; and generating a final output according to a hierarchy division scheme of real-time optimization.

Description

Technical Field

[0001] The present application relates to the technical field of electronic digital data processing, and in particular to a multi-source teaching data processing method based on deep learning. Background Art

[0002] When using deep learning technology to process multi-source teaching data to achieve dense layered design of teaching content, a hidden and unique technical contradiction gradually emerges: when convolutional neural networks are used to extract subtle features in multi-source teaching data, the system needs to accurately divide the microscopic difficulty gradient in the high-dimensional feature space. However, this process faces the dual constraints of data heterogeneity and model computational complexity.

[0003] Multi-source teaching data usually contains multiple modalities such as text, audio, and video. The feature distribution of each modality varies significantly, including the difficulty in aligning the semantic density of text data and the inter-frame temporal correlation of video data in terms of magnitude and structure. This heterogeneity causes convolutional neural networks to easily ignore certain modality-specific micro-difficulty clues during cross-modal fusion due to the fixed convolution kernel, such as changes in speech speed in short-term audio or fluctuations in sentence complexity in text. At the same time, in order to capture these subtle features, the network hierarchy needs to be deepened to improve the resolution ability, but the deepening of the hierarchy makes the computational complexity increase exponentially, especially when dividing dense hierarchies in real time. The system may not be able to meet the immediacy requirements of teaching scenarios due to excessive latency.

[0004] In addition, noise interference from multi-source data further exacerbates the contradiction: background noise may mask key difficulty signals in the audio, while non-standard expressions in the text may mislead semantic parsing, making the dividing boundaries of the difficulty gradient unclear.

[0005] This contradiction is specifically manifested in business scenarios. When the system attempts to break down teaching content into dozens or even hundreds of dense levels, it must ensure the complete extraction of cross-modal features while avoiding the loss of real-time performance of the hierarchical division due to excessive computational burden, forming a deep contradiction between technology and scenario requirements. Summary of the invention

[0006] In order to solve the problems existing in the above-mentioned prior art, the purpose of this application is to provide a multi-source teaching data processing method based on deep learning.

[0007] The present application discloses a multi-source teaching data processing method based on deep learning, comprising the following steps: Step S101, after obtaining multi-source teaching data, pre-process the text, audio, and video data through modality separation technology, extract independent feature sets from each modality, and obtain preliminary mapping results of modality feature distribution; Step S102, based on the preliminary mapping results of the modal feature distribution, an adaptive convolution kernel adjustment algorithm is used to dynamically adjust the convolution kernel size according to the difference in the modal feature distribution to determine the feature extraction range before cross-modal fusion; Step S103, processing the adjusted feature extraction range through a multi-scale feature fusion network, fusing text semantic density, audio speech rate change and temporal correlation between video frames at different scales to obtain a unified high-dimensional feature space representation; Step S104, based on the unified high-dimensional feature space representation, a noise suppression algorithm is used to filter the data segment with large background noise or non-standard expression noise interference. If the noise detected exceeds a preset threshold, the corresponding feature weight is attenuated to obtain a clear feature set; Step S105, after obtaining a clear feature set, the feature set is scanned layer by layer using a micro-difficulty perception model, and the initial division boundary of the difficulty gradient is determined by matching the micro-difficulty clues through a difficulty gradient template based on semantic complexity or speech speed changes; Step S106, for the preliminary division boundary of the difficulty gradient, the convolution layer or the fully connected layer is optimized by using lightweight network compression technology, so as to reduce the computational complexity while maintaining the resolution capability, and obtain a hierarchical division scheme with real-time optimization; Step S107, through the hierarchical division scheme optimized for real-time performance, combined with the dynamic time window algorithm, the dense hierarchical division is iteratively adjusted. If the fuzziness of the division boundary calculated based on the feature overlap rate or confidence is higher than the preset threshold, the context semantics or audio spectrum features are introduced for recalibration to obtain the final dense hierarchical division result; Step S108, according to the final dense hierarchical division result, using the cross-modal fusion effect evaluation mechanism based on accuracy or recall rate, if the fusion accuracy is lower than the preset threshold, adjust the adaptive convolution kernel parameters, update the feature extraction range and recalculate to obtain optimized teaching content layered output.

[0008] Preferably, in step S101, the multimodal data is preprocessed using a modal separation technique to obtain a plurality of independent feature sets, including: for the text data, a text feature extraction algorithm is used to obtain a text independent feature set; For the audio data, obtain an audio independent feature set by audio data processing, and determine a feature distribution of the audio independent feature set; For the video data, a video feature extraction technology is used to obtain a video independent feature set, and a modal feature of the video independent feature set is determined; fusing the text-independent feature set, the audio-independent feature set, and the video-independent feature set to obtain a multimodal feature set; A clustering algorithm is used for the multimodal feature set to obtain a modal feature distribution mapping result.

[0009] Preferably, in the step S102, for the fused feature set, a convolution adjustment algorithm is used to determine the feature extraction range, including: acquiring feature distribution data of the fused feature set, and obtaining a quantitative representation of distribution differences through a preset feature distribution analysis method; According to the quantitative representation of the distribution difference, an adaptive convolution kernel adjustment algorithm is used to dynamically adjust the convolution kernel size to obtain adjusted convolution kernel parameters; According to the adjusted convolution kernel parameters, a feature extraction algorithm is used to extract feature data before cross-modal fusion from the fusion feature set to determine a feature extraction range; If the modal consistency within the feature extraction range is lower than a preset threshold, the convolution kernel size is readjusted according to the distribution difference to obtain an updated feature extraction range.

[0010] Preferably, in the step S103, the feature extraction range is processed by a multi-scale feature fusion network to obtain a high-dimensional feature space representation, including: extracting text semantic density features for the text data; Extracting audio speech rate change features from the audio data; For the video data, extracting temporal correlation features between video frames; The text semantic density feature, the audio speech rate change feature and the video frame temporal correlation feature are integrated to obtain a preliminary multi-scale feature representation; Processing the preliminary multi-scale feature representation through a convolutional neural network, combining it with an attention mechanism to process the relationship between frames, and generating a comprehensive spatiotemporal feature representation; A unified high-dimensional feature space representation is generated based on the comprehensive spatiotemporal feature representation.

[0011] Preferably, in step S104, according to the high-dimensional feature space representation, a noise suppression algorithm is used to generate a clear feature set, including: for the high-dimensional feature space representation, using a noise suppression algorithm to detect interference data formed by background noise and non-standard expression, and determining an interference data segment; Extracting noise intensity from the interference data segment, and if the noise intensity exceeds a preset threshold, adjusting the weight of the corresponding feature by a weight decay algorithm to obtain an adjusted feature set; According to the adjusted feature set, a clustering algorithm is used to obtain a clear feature subset; The data segment is reconstructed in the feature space through the clear feature subset to obtain the feature representation after filtering.

[0012] Preferably, in the step S105, for the clear feature set, a difficulty perception model is used to determine the difficulty gradient division, including: using a microscopic difficulty perception model to scan the clear feature set layer by layer to obtain a feature level sequence after scanning; Matching microscopic clues in the feature level sequence through a preset gradient template to determine the initial change trend of semantic complexity or speech speed change; According to the preliminary change trend, semantic complexity analysis is used to determine the difficulty gradient fluctuation, and a fluctuation detection result is obtained; If the fluctuation detection result exceeds a preset threshold, the division boundary is adjusted through template matching to determine an optimized difficulty gradient boundary.

[0013] Preferably, in step S106, the difficulty gradient division is processed by network optimization technology to obtain a hierarchical division scheme with real-time optimization, including: obtaining a convolutional neural network model to be optimized, and processing the convolutional layer by using lightweight network compression technology to obtain an optimized convolutional layer structure; According to the optimized convolutional layer structure, the parameters of the fully connected layer are adjusted to obtain simplified parameters of the fully connected layer; For the simplified fully connected layer parameters, evaluating the computational complexity to obtain a network model with reduced complexity; According to the network model with reduced complexity, a preliminary hierarchical structure is divided. If the resolution capability does not meet the preset requirements, the division boundary is adjusted to obtain an optimized hierarchical division result.

[0014] Preferably, in step S107, generating the final output according to the real-time optimized hierarchical division scheme includes: iteratively adjusting the real-time optimized hierarchical division scheme using a dynamic time window algorithm to obtain a preliminary dense hierarchical structure; Calculating the feature overlap rate of the division boundary in the preliminary dense hierarchical structure to obtain a fuzziness evaluation value; If the fuzziness evaluation value is higher than a preset threshold, calibrating the division boundary according to the context semantic features and the audio spectrum features to obtain a calibrated dense hierarchical structure; According to the calibrated dense hierarchical structure, the confidence is calculated to generate a final dense hierarchical partitioning result.

[0015] The multi-source teaching data processing method based on deep learning described in the present application has the advantages that, in order to solve the problem of large differences in the distribution of different modal features such as text, audio, and video in multi-source teaching data, the present invention adopts an adaptive convolution kernel adjustment algorithm to dynamically determine the feature extraction range, and realizes the unified representation of cross-modal information through a multi-scale feature fusion network. In order to solve the problems of noise interference and inaccurate difficulty division in the teaching content, the present invention utilizes a noise suppression algorithm and a microscopic difficulty perception model to perform feature optimization and difficulty gradient division. Through lightweight network compression and a dynamic time window algorithm, the present invention realizes real-time optimized dense hierarchical division. Finally, based on the cross-modal fusion effect evaluation mechanism, the present invention can output high-precision teaching content stratification results, effectively improving the intelligent processing and refined management level of multi-modal teaching resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is the process of a multi-source teaching data processing method based on deep learning described in this application Figure 1 ; Figure 2 This is the process of a multi-source teaching data processing method based on deep learning described in this application Figure 2 . DETAILED DESCRIPTION

[0017] like Figure 1-Figure 2 As shown, the multi-source teaching data processing method based on deep learning described in this application includes the following steps: Step S101, after obtaining multi-source teaching data, pre-process the text, audio, and video data through modality separation technology, extract independent feature sets from each modality, and obtain preliminary mapping results of modality feature distribution; Step S102, based on the preliminary mapping results of the modal feature distribution, an adaptive convolution kernel adjustment algorithm is used to dynamically adjust the convolution kernel size according to the difference in the modal feature distribution to determine the feature extraction range before cross-modal fusion; Step S103, processing the adjusted feature extraction range through a multi-scale feature fusion network, fusing text semantic density, audio speech rate change and temporal correlation between video frames at different scales to obtain a unified high-dimensional feature space representation; Step S104, based on the unified high-dimensional feature space representation, a noise suppression algorithm is used to filter the data segment with large background noise or non-standard expression noise interference. If the noise detected exceeds a preset threshold, the corresponding feature weight is attenuated to obtain a clear feature set; Step S105, after obtaining a clear feature set, the feature set is scanned layer by layer using a micro-difficulty perception model, and the initial division boundary of the difficulty gradient is determined by matching the micro-difficulty clues through a difficulty gradient template based on semantic complexity or speech speed changes; Step S106, for the preliminary division boundary of the difficulty gradient, the convolution layer or the fully connected layer is optimized by using lightweight network compression technology, so as to reduce the computational complexity while maintaining the resolution capability, and obtain a hierarchical division scheme with real-time optimization; Step S107, through the hierarchical division scheme optimized for real-time performance, combined with the dynamic time window algorithm, the dense hierarchical division is iteratively adjusted. If the fuzziness of the division boundary calculated based on the feature overlap rate or confidence is higher than the preset threshold, the context semantics or audio spectrum features are introduced for recalibration to obtain the final dense hierarchical division result; Step S108, according to the final dense hierarchical division result, using the cross-modal fusion effect evaluation mechanism based on accuracy or recall rate, if the fusion accuracy is lower than the preset threshold, adjust the adaptive convolution kernel parameters, update the feature extraction range and recalculate to obtain optimized teaching content layered output.

[0018] like Figure 1-Figure 2 As shown, in step S101, after acquiring multi-source teaching data, the text, audio, and video data are preprocessed using modal separation technology, and independent feature sets are extracted from each modality to obtain a preliminary mapping result of the modal feature distribution.

[0019] Further, in step S101, multi-source teaching data is obtained, and the multi-source teaching data includes text data, audio data and video data; The multi-source teaching data is processed by using modality separation technology to obtain text modality data, audio modality data and video modality data; For text modal data, a text feature extraction algorithm is used to obtain a text-independent feature set; For the audio modal data, extract the audio independent feature set through audio data processing, and determine the feature distribution of the audio independent feature set; For video modal data, video feature extraction technology is used to extract video independent feature sets and determine the modal features of the video independent feature sets; The text-independent feature set, the audio-independent feature set and the video-independent feature set are integrated to obtain a multimodal feature set; For the multimodal feature set, a clustering algorithm is used to obtain the modal feature distribution mapping result; Extract key features from the modal feature distribution mapping results and generate preliminary modal feature distribution results; Based on the preliminary modal feature distribution results, the feature space unification technique is used to generate feature outputs suitable for downstream tasks.

[0020] Specifically, in step S101, multi-source teaching data is obtained, and the multi-source teaching data includes text data, audio data and video data; The multi-source teaching data is processed using modality separation technology. If the data is in mixed format, the text modality data, audio modality data and video modality data are separated through the FFmpeg tool; For text modal data, the TF-IDF algorithm is used to extract text-independent feature sets, and the feature dimension is set to 300 dimensions; For audio modal data, the audio independent feature set is extracted through Mel spectrum analysis, and the K-means clustering algorithm is used to determine its feature distribution, with the number of cluster centers being 5; For video modal data, the OpenCV library is used to extract the optical flow features between video frames, and a convolutional neural network is used to determine the modal features of the video independent feature set. The network has three layers. The text-independent feature set, audio-independent feature set, and video-independent feature set are integrated, and the weighted average method is used to obtain the multimodal feature set, with weights of 0.4, 0.3, and 0.3 respectively; For the multimodal feature set, the DBSCAN clustering algorithm is used to obtain the modal feature distribution mapping results, and the neighborhood radius is set to 0.5; Extract key features from the modal feature distribution mapping results, use PCA dimensionality reduction technology to generate preliminary modal feature distribution results, and retain 80% of the variance; According to the preliminary modal feature distribution results, the feature space unification technology is adopted to generate feature outputs suitable for downstream tasks through linear discriminant analysis, and the feature dimension is 100 dimensions.

[0021] like Figure 1-Figure 2 As shown, in step S102, based on the preliminary mapping results of the modal feature distribution, an adaptive convolution kernel adjustment algorithm is used to dynamically adjust the convolution kernel size according to the difference in the modal feature distribution to determine the feature extraction range before cross-modal fusion.

[0022] Furthermore, in step S102, preliminary data of modal features are obtained, and a quantitative representation of distribution differences is obtained through a preset feature distribution analysis method; According to the quantitative representation of the distribution difference, an adaptive convolution kernel adjustment algorithm is used to dynamically adjust the convolution kernel size to obtain the adjusted convolution kernel parameters; According to the adjusted convolution kernel parameters, a feature extraction algorithm is used to extract feature data before cross-modal fusion from the modal features and determine the feature extraction range; Acquire data within the feature extraction range and generate a fusion feature set through a cross-modal fusion algorithm; If the modal consistency of the fused feature set is lower than the preset threshold, the convolution kernel size is readjusted according to the distribution difference to obtain an updated feature extraction range; Through the updated feature extraction range, the feature extraction algorithm is used to process the modal features to obtain the optimized fusion feature set; According to the optimized fusion feature set, the feature consistency of cross-modal fusion is judged through the preset verification method to obtain the final feature representation; Obtain cross-modal fusion data from dense hierarchical partitioning results, use accuracy evaluation and recall evaluation to calculate the initial fusion accuracy value; If the initial fusion accuracy value is lower than the preset threshold, the adaptive convolution kernel parameters are adjusted and the feature extraction range is updated to obtain optimized feature extraction data.

[0023] Specifically, in step S102, preliminary data of modal features are obtained, and the feature distribution is analyzed by a Gaussian mixture model (GMM) to obtain a quantitative representation of the distribution difference, including calculating the inter-modal Kullback-Leibler divergence (KL divergence) as 0.75; According to the quantitative representation of the distribution difference, an adaptive convolution kernel adjustment algorithm is adopted, including using a dynamic window adjustment strategy to adjust the convolution kernel size from 3x3 to 5x5, and obtain the adjusted convolution kernel parameters; According to the adjusted convolution kernel parameters, the convolutional neural network (CNN) feature extraction algorithm is used to extract feature data before cross-modal fusion from the modal features, and the feature extraction range is determined to be a 128-dimensional vector; Obtain data within the feature extraction range, and generate a fused feature set using a cross-modal fusion algorithm based on an attention mechanism, including a feature dimension of 256; If the modality consistency of the fused feature set is lower than the preset threshold of 0.9, the convolution kernel size is readjusted according to the distribution difference, including adjusting the convolution kernel size from 5x5 to 7x7 to obtain an updated feature extraction range; Through the updated feature extraction range, the CNN feature extraction algorithm is used to process the modal features to obtain the optimized fusion feature set, including a feature dimension of 512; Based on the optimized fusion feature set, the consistency of the cross-modal fusion features was determined by the cross-validation method to obtain the final feature representation, including a consistency score of 0.95; Obtain cross-modal fusion data from the dense hierarchical partitioning results, use accuracy evaluation and recall evaluation to calculate the initial fusion accuracy values, including an accuracy of 0.85 and a recall of 0.80; If the initial fusion accuracy value is lower than the preset threshold value of 0.9, the adaptive convolution kernel parameters are adjusted, including adjusting the convolution kernel size from 7x7 to 9x9, updating the feature extraction range, and obtaining optimized feature extraction data, including a feature dimension of 1024. like Figure 1-Figure 2 As shown, in step S103, the adjusted feature extraction range is processed by a multi-scale feature fusion network, and the text semantic density, audio speech rate change and temporal correlation between video frames are fused at different scales to obtain a unified high-dimensional feature space representation.

[0024] Further, in step S103, multimodal data is obtained, wherein the multimodal data includes text data, audio data and video data; Extracting text semantic density features from the text data; Extracting audio speech rate change features from the audio data; For the video data, extracting temporal correlation features between video frames; The text semantic density feature, the audio speech rate change feature and the video frame temporal correlation feature are integrated to obtain a preliminary multi-scale feature representation; Using a convolutional neural network to process the text semantic density features to obtain an enhanced semantic density representation; The enhanced semantic density representation and the audio speech rate change feature are integrated, and the speech rate change sequence is analyzed through a recurrent neural network to obtain a time-series enhanced semantic feature; Combining the temporal correlation features between video frames, an attention mechanism is used to process the relationship between frames to obtain a comprehensive spatiotemporal feature representation; According to the comprehensive spatiotemporal feature representation, a unified high-dimensional feature space representation is generated through network processing; Determine whether the feature distribution represented by the high-dimensional feature space meets a preset threshold, and if so, output a final feature representation; Through feature space unification technology, feature output suitable for downstream tasks is generated.

[0025] Specifically, in step S103, text data, audio data and video data are obtained through the data acquisition interface, wherein the text data is a data set containing 1000 sentences, the audio data is a voice clip with a sampling rate of 44.1kHz, and the video data is a video clip with a frame rate of 30fps; The BERT model is used to process the text data, extract the semantic density features of each sentence, and calculate the lexical density value to be 0.85; Short-time Fourier transform was used to extract speech rate variation characteristics from audio data, and the average speech rate variation rate per second was 0.12; The optical flow method is used to extract the temporal correlation features between frames of video data, and the inter-frame motion vector is 0.35; The text semantic density features, audio speech rate change features and video frame temporal correlation features are input into the multi-scale feature fusion network, and the weighted average method is used for preliminary fusion to obtain a multi-scale feature representation vector. A convolutional neural network is used to process the semantic density features of text. The convolution kernel size is 3×3, and the output is an enhanced semantic density representation vector with a dimension of 128. The enhanced semantic density representation and audio speech rate change features are input into the LSTM network, the number of hidden layer units is set to 64, and the time-series enhanced semantic feature vector is output; Combined with the temporal correlation features between video frames, a multi-head attention mechanism is used to calculate the inter-frame relationship weights and output a comprehensive spatiotemporal feature representation vector with a dimension of 256; The comprehensive spatiotemporal feature representation is input into a fully connected network, and a unified high-dimensional feature space representation vector with a dimension of 512 is output; The difference between the feature distribution represented in the high-dimensional feature space and the preset threshold is calculated by KL divergence. If the difference value is less than 0.01, the final feature representation is output; The principal component analysis method is used to reduce the dimension of the final feature representation to generate a 128-dimensional feature output suitable for downstream tasks.

[0026] like Figure 1-Figure 2 As shown, in step S104, according to the unified high-dimensional feature space representation, a noise suppression algorithm is used to filter the data segments with large background noise or non-standard expression noise interference. If the noise is detected to exceed a preset threshold, the corresponding feature weight is attenuated to obtain a clear feature set.

[0027] Furthermore, in step S104, input data is obtained and mapped into a high-dimensional feature set; Use noise suppression algorithm to detect background noise and interference data formed by non-standard expressions in high-dimensional feature sets, and determine interference data segments; Extracting noise intensity from the interference data segment and determining whether the noise intensity exceeds a preset threshold; If it exceeds, the corresponding feature is marked as a feature to be processed; The weights of the features to be processed are adjusted by a weight decay algorithm to obtain an adjusted feature set; Clustering algorithm is used to group the adjusted feature set to obtain clear feature subsets; Reconstruct the data segment in the feature space according to the clear feature subset to obtain the feature representation after filtering; Extract the final output from the feature representation after filtering, and determine whether the final output meets the preset clarity condition; If not, return to the noise suppression algorithm step and reprocess; Obtain preliminary data of modal characteristics and obtain quantitative representation of distribution differences through a preset characteristic distribution analysis method; According to the quantitative representation of the distribution difference, an adaptive convolution kernel adjustment algorithm is used to dynamically adjust the convolution kernel size to obtain the adjusted convolution kernel parameters.

[0028] Specifically, in step S104, input data is obtained, and the input data is mapped into a high-dimensional feature set through a principal component analysis (PCA) algorithm to generate a feature space containing a 128-dimensional feature vector; A noise suppression algorithm based on wavelet transform is used to detect the background noise and interference data formed by non-standard expressions in the high-dimensional feature set, and the interference data segment with a signal-to-noise ratio lower than 10dB is determined by calculating the signal-to-noise ratio (SNR); Extract the noise intensity from the interference data segment, use the fast Fourier transform (FFT) to calculate the noise spectrum, and if the peak value in the noise spectrum exceeds the preset threshold of -20dB, mark the corresponding feature as a feature to be processed; The features to be processed are adjusted through the weight decay algorithm, and the feature weight is decayed from 1.0 to 0.3 using the exponential decay function to obtain the adjusted feature set; The K-means clustering algorithm is used to group the adjusted feature set, and the number of clusters is set to 5 to obtain a clear feature subset; Reconstruct the data segments from the subset of clear features in the feature space and generate filtered feature representations using inverse Fourier transform; The final output is extracted from the feature representation after filtering, and the structural similarity index (SSIM) is used to determine whether the final output meets the preset clarity condition of 0.9. If not, the noise suppression algorithm step is returned to reprocess; Obtain preliminary data of modal features, perform feature distribution analysis through Gaussian mixture model (GMM), obtain quantitative representation of distribution differences, and calculate KL divergence value as 0.5; According to the quantitative representation of the distribution difference, the adaptive convolution kernel adjustment algorithm is used to dynamically adjust the convolution kernel size. The convolution kernel is adjusted from 3x3 to 5x5 to obtain the adjusted convolution kernel parameters.

[0029] like Figure 1-Figure 2 As shown, in step S105, after obtaining a clear feature set, the feature set is scanned layer by layer using a micro-difficulty perception model, and the preliminary division boundary of the difficulty gradient is determined by matching the micro-difficulty clues through a difficulty gradient template based on semantic complexity or speech speed changes.

[0030] Furthermore, in step S105, audio data is obtained, and a dynamic time window algorithm is used to hierarchically divide the audio data to obtain a preliminary dense hierarchical structure; For the preliminary dense hierarchical structure, the feature overlap rate of each partition boundary is calculated to obtain the fuzziness evaluation value of each partition boundary; Obtaining contextual semantic features of each partition boundary in the preliminary dense hierarchical structure, and if the fuzziness evaluation value is higher than a preset threshold, adjusting the partition boundary according to the contextual semantic features to obtain an adjusted dense hierarchical structure; Extracting frequency spectrum features of the audio data, and calibrating the adjusted dense hierarchical structure according to the frequency spectrum features to obtain a calibrated dense hierarchical structure; For each partition boundary in the calibrated dense hierarchical structure, the confidence is calculated to obtain a stability confirmation value; According to the stability confirmation value, the real-time optimization scheme is iteratively updated using the window algorithm to obtain the final dense hierarchical structure; Extract the matching relationship between dense levels and real-time optimization from the final dense hierarchical structure, and determine the dense hierarchical division scheme; Obtain a feature set in a dense hierarchical partitioning scheme, and use a microscopic difficulty perception model to scan the feature set layer by layer to obtain a feature hierarchical sequence after scanning; By matching the microscopic clues in the feature hierarchy sequence with the preset gradient template, the initial change trend of semantic complexity or speech rate change is determined, and the preliminary division boundary of the difficulty gradient is obtained.

[0031] Specifically, in step S105, after the audio data is acquired, the audio data is hierarchically divided into 50 milliseconds as a basic unit using a dynamic time window algorithm, and the frequency features are extracted by FFT (Fast Fourier Transform) to obtain a preliminary dense hierarchical structure; For the preliminary dense hierarchical structure, the feature overlap rate of each partition boundary is calculated and evaluated using the Jaccard similarity coefficient. If the overlap rate exceeds 0.7, it is marked as high fuzziness, and the fuzziness evaluation value of each partition boundary is generated; Obtain the contextual semantic features of each partition boundary in the preliminary dense hierarchical structure. If the fuzziness evaluation value is higher than the preset threshold of 0.8, analyze the contextual semantics through the BERT model, adjust the partition boundary, and obtain the adjusted dense hierarchical structure. Extract the spectral features of the audio data, and use the Mel Frequency Cepstral Coefficient (MFCC) to calibrate the adjusted dense hierarchical structure to ensure that the frequency distribution is consistent with the semantic features, and obtain the calibrated dense hierarchical structure; For each partition boundary in the calibrated dense hierarchical structure, its confidence is calculated, and the Bayesian probability model is used to evaluate the boundary stability. If the confidence is lower than 0.9, it is marked as unstable and the stability confirmation value is obtained; According to the stability confirmation value, the sliding window algorithm is used to iteratively update the real-time optimization solution in units of 10 milliseconds to ensure the real-time and accuracy of boundary division and obtain the final dense hierarchical structure; Extract the matching relationship between dense levels and real-time optimization from the final dense hierarchical structure, use the KNN algorithm (K nearest neighbor algorithm) for feature matching, and determine the dense hierarchical division scheme; Obtain the feature set in the dense hierarchical partitioning scheme, use the microscopic difficulty perception model to scan the feature set layer by layer, extract the hierarchical sequence through LSTM (Long Short-Term Memory Network), and obtain the scanned feature hierarchical sequence; Through the preset gradient template matching feature level sequence of micro clues, the DTW (dynamic time warping) algorithm is used to analyze the initial change trend of semantic complexity or speech rate change. If the change exceeds 20%, it is marked as the difficulty gradient boundary, and the preliminary division boundary of the difficulty gradient is obtained.

[0032] like Figure 1-Figure 2 As shown, in step S106, with respect to the preliminary division boundary of the difficulty gradient, the convolutional layer or the fully connected layer is optimized by means of lightweight network compression technology, so as to reduce the computational complexity while maintaining the resolution capability, and obtain a hierarchical division scheme with real-time optimization.

[0033] Further, in step S106, a convolutional neural network model to be optimized is obtained, wherein the model includes at least one convolutional layer and at least one fully connected layer; The convolution layer is processed by using a lightweight network compression technology to obtain an optimized convolution layer structure; According to the optimized convolutional layer structure, the structure of the fully connected layer is adjusted to obtain streamlined fully connected layer parameters; For the simplified fully connected layer parameters, the computational complexity is evaluated to obtain a network model with reduced complexity. If the network model after complexity reduction meets the preset complexity threshold, the preliminary hierarchical structure is divided according to the combination of convolutional layer and fully connected layer to obtain the preliminary hierarchical division result; According to the preliminary hierarchical division results, analyze its resolution ability to determine whether it meets the preset real-time optimization requirements; If the preset real-time optimization requirements are not met, the optimized hierarchical division results are obtained by adjusting the division boundaries in the preliminary hierarchical division results; Determine the optimized hierarchical division result as the final real-time optimization solution; After obtaining the final real-time optimization plan, the rationality of the final hierarchical division results is confirmed by verifying the difficulty gradient of each level in the plan.

[0034] Specifically, in step S106, a convolutional neural network model to be optimized is obtained, wherein the model includes at least one convolutional layer and at least one fully connected layer, including a VGG network including 5 convolutional layers and 3 fully connected layers; The convolution layer is processed by using a lightweight network compression technology, including using a pruning technology to reduce the number of parameters of the convolution layer by 30%, thereby obtaining an optimized convolution layer structure; According to the optimized convolutional layer structure, the structure of the fully connected layer is adjusted, including reducing the number of neurons in the fully connected layer from 4096 to 1024, so as to obtain streamlined parameters of the fully connected layer; The computational complexity of the streamlined fully connected layer parameters was evaluated, including calculations using floating point operations (FLOPs). It was found that the amount of computation was reduced by 40%, resulting in a network model with reduced complexity. If the network model after complexity reduction meets the preset complexity threshold, including FLOPs reduced to less than 50% of the original model, then the preliminary hierarchical structure is divided according to the combination of convolutional layer and fully connected layer, including dividing the network into input layer, feature extraction layer and classification layer, and obtaining the preliminary hierarchical division result; Based on the preliminary hierarchical classification results, the resolution capability was analyzed, including the classification accuracy evaluation. It was found that the accuracy remained above 95%, and it was determined whether the preset real-time optimization requirements were met; If the preset real-time optimization requirements are not met, including the accuracy rate being lower than 90%, the optimized hierarchical division result is obtained by adjusting the division boundaries in the preliminary hierarchical division result, including redefining the number of convolution kernels of the feature extraction layer; Determine the optimized hierarchical division result as the final real-time optimization solution; After obtaining the final real-time optimization plan, the rationality of the final hierarchical division result is confirmed by verifying the difficulty gradient of each level in the plan, including using the gradient descent method to evaluate the weight update speed of each layer.

[0035] like Figure 1-Figure 2 As shown, in step S107, the dense hierarchical division is iteratively adjusted through a real-time optimized hierarchical division scheme combined with a dynamic time window algorithm. If the fuzziness of the division boundary calculated based on the feature overlap rate or confidence is higher than a preset threshold, the contextual semantics or audio spectrum features are introduced for recalibration to obtain the final dense hierarchical division result.

[0036] Furthermore, in step S107, audio data is obtained, and a dynamic time window algorithm is used to hierarchically divide the audio data to obtain a preliminary dense hierarchical structure; For the preliminary dense hierarchical structure, the feature overlap rate of each partition boundary is calculated to obtain the fuzziness evaluation value of each partition boundary; Obtaining contextual semantic features of each partition boundary in the preliminary dense hierarchical structure, and if the fuzziness evaluation value is higher than a preset threshold, adjusting the partition boundary according to the contextual semantic features to obtain an adjusted dense hierarchical structure; Extracting frequency spectrum features of the audio data, and calibrating the adjusted dense hierarchical structure according to the frequency spectrum features to obtain a calibrated dense hierarchical structure; For each partition boundary in the calibrated dense hierarchical structure, the confidence is calculated to obtain a stability confirmation value; According to the stability confirmation value, the real-time optimization scheme is iteratively updated using the window algorithm to obtain the final dense hierarchical structure; Extract the matching relationship between dense levels and real-time optimization from the final dense hierarchical structure, and determine the dense hierarchical division scheme; Obtain a feature set, and use a microscopic difficulty perception model to scan the feature set layer by layer to obtain a feature level sequence after scanning; By matching the micro clues in the feature hierarchy sequence with a preset gradient template, the preliminary change trend of semantic complexity or speech speed change is determined. Based on the preliminary change trend, it is judged whether there is a significant fluctuation in the difficulty gradient to obtain the fluctuation detection result.

[0037] Specifically, in step S107, after the audio data is acquired, a dynamic time window algorithm is used to hierarchically divide the audio data with an initial window size of 50 milliseconds to obtain a preliminary dense hierarchical structure; For the preliminary dense hierarchical structure, the feature overlap rate of each partition boundary is calculated, and the cosine similarity algorithm is used. If the overlap rate is lower than 0.8, it is marked as a high fuzziness boundary, and the fuzziness evaluation value of each partition boundary is obtained; Obtain the contextual semantic features of each partition boundary in the preliminary dense hierarchical structure. If the fuzziness evaluation value is higher than the preset threshold of 0.7, extract the contextual semantic features based on the BERT model, adjust the partition boundary, and obtain the adjusted dense hierarchical structure. Extracting the spectral features of the audio data, extracting the spectral features in the frequency range of 0-8000 Hz by fast Fourier transform, calibrating the adjusted dense hierarchical structure according to the spectral features, and obtaining a calibrated dense hierarchical structure; For each partition boundary in the calibrated dense hierarchical structure, the support vector machine is used to calculate its confidence. If the confidence is lower than 0.9, it is marked as an unstable boundary to obtain the stability confirmation value; According to the stability confirmation value, the real-time optimization scheme is iteratively updated with a sliding window algorithm in steps of 20 milliseconds to obtain the final dense hierarchical structure; Extract the matching relationship between dense hierarchy and real-time optimization from the final dense hierarchy structure, divide the matching relationship into three categories through K-means clustering algorithm, and determine the dense hierarchy division scheme; Obtain a feature set, and use a microscopic difficulty perception model to scan the feature set layer by layer at a time interval of 0.1 seconds to obtain a feature level sequence after scanning; Through the preset gradient template matching feature level sequence of micro clues, the dynamic time warping algorithm is used to calculate the initial change trend of semantic complexity or speech rate change. If the change amplitude exceeds 15%, it is marked as a significant fluctuation to obtain the fluctuation detection result.

[0038] like Figure 1-Figure 2 As shown, in step S108, according to the final dense hierarchical division result, a cross-modal fusion effect evaluation mechanism based on accuracy or recall is used. If the fusion accuracy is lower than the preset threshold, the adaptive convolution kernel parameters are adjusted, the feature extraction range is updated and recalculated to obtain an optimized hierarchical output of teaching content.

[0039] Furthermore, in step S108, the cross-modal fusion data in the dense hierarchical division result is obtained, and the initial fusion accuracy value is calculated by using the accuracy evaluation and the recall evaluation; If the initial fusion accuracy value is lower than the preset threshold, the adaptive convolution kernel parameters are adjusted, the feature extraction range is updated, and the optimized feature extraction data is obtained; Perform cross-modal fusion processing on the optimized feature extraction data to obtain an updated fusion accuracy value; Determine whether the updated fusion accuracy value is lower than a preset threshold. If so, repeat the adaptive convolution kernel parameter adjustment and feature extraction range update to obtain optimized feature extraction data again. Recalculate the cross-modal fusion result based on the re-optimized feature extraction data to obtain a fusion accuracy value that meets the preset threshold; Extract teaching content from the fusion accuracy values ​​that meet the preset threshold, and generate preliminary hierarchical output data in combination with the dense hierarchical division structure; Format the preliminary hierarchical output data and adopt the hierarchical output mechanism to obtain the final optimized hierarchical output results of teaching content; Obtain preliminary data of modal characteristics and obtain quantitative representation of distribution differences through a preset characteristic distribution analysis method; According to the quantitative representation of the distribution difference, an adaptive convolution kernel adjustment algorithm is used to dynamically adjust the convolution kernel size to obtain the adjusted convolution kernel parameters.

[0040] Specifically, in step S108, the cross-modal fusion data in the dense hierarchical division result is obtained, and the initial fusion accuracy value is calculated by using the accuracy evaluation and the recall evaluation, including by comparing the matching degree of text and image, the accuracy is 85%, and the recall is 78%; If the initial fusion accuracy value is lower than the preset threshold of 90%, the adaptive convolution kernel parameters are adjusted to update the feature extraction range, including adjusting the convolution kernel size from 3x3 to 5x5 to obtain optimized feature extraction data; The optimized feature extraction data is subjected to cross-modal fusion processing, including fusing text and image features through weighted average method, and the updated fusion accuracy value is calculated to be 92%; Determine whether the updated fusion accuracy value is lower than a preset threshold. If so, repeat the adaptive convolution kernel parameter adjustment and feature extraction range update, including adjusting the convolution kernel size from 5x5 to 7x7, to obtain optimized feature extraction data again. Recalculate the cross-modal fusion results based on the optimized feature extraction data, including reducing the dimensionality of the fusion features through principal component analysis, and obtain a fusion accuracy value that meets the preset threshold of 95%; Extracting teaching content from fusion accuracy values ​​that meet a preset threshold, combined with a dense hierarchical partitioning structure, including mapping fusion features to knowledge graph nodes, to generate preliminary hierarchical output data; Format the preliminary hierarchical output data and adopt a hierarchical output mechanism, including describing the hierarchical relationship of teaching content through XML structured language, to obtain the final optimized hierarchical output result of teaching content; Obtain preliminary data of modal features, and use a preset feature distribution analysis method, including calculating the KL divergence of text and image features, to obtain a quantitative representation of the distribution difference of 0.15; According to the quantitative representation of the distribution difference, an adaptive convolution kernel adjustment algorithm is used to dynamically adjust the convolution kernel size, including adjusting the convolution kernel step size from 1 to 2, to obtain the adjusted convolution kernel parameters.

[0041] For those skilled in the art, various other corresponding changes and deformations can be made according to the technical solutions and concepts described above, and all of these changes and deformations should fall within the protection scope of the claims of this application.

Claims

1. A multi-source teaching data processing method based on deep learning, characterized in that: The following steps are involved: Step S101, after obtaining multi-source teaching data, pre-process the text, audio, and video data through modality separation technology, extract independent feature sets from each modality, and obtain preliminary mapping results of modality feature distribution; Step S102, based on the preliminary mapping results of the modal feature distribution, an adaptive convolution kernel adjustment algorithm is used to dynamically adjust the convolution kernel size according to the difference in the modal feature distribution to determine the feature extraction range before cross-modal fusion; Step S103, processing the adjusted feature extraction range through a multi-scale feature fusion network, fusing text semantic density, audio speech rate change and temporal correlation between video frames at different scales to obtain a unified high-dimensional feature space representation; Step S104, based on the unified high-dimensional feature space representation, a noise suppression algorithm is used to filter the data segment with large background noise or non-standard expression noise interference. If the noise detected exceeds a preset threshold, the corresponding feature weight is attenuated to obtain a clear feature set; Step S105, after obtaining a clear feature set, the feature set is scanned layer by layer using a micro-difficulty perception model, and the initial division boundary of the difficulty gradient is determined by matching the micro-difficulty clues through a difficulty gradient template based on semantic complexity or speech speed changes; Step S106, for the preliminary division boundary of the difficulty gradient, the convolution layer or the fully connected layer is optimized by using lightweight network compression technology, so as to reduce the computational complexity while maintaining the resolution capability, and obtain a hierarchical division scheme with real-time optimization; Step S107, through the hierarchical division scheme optimized for real-time performance, combined with the dynamic time window algorithm, the dense hierarchical division is iteratively adjusted. If the fuzziness of the division boundary calculated based on the feature overlap rate or confidence is higher than the preset threshold, the context semantics or audio spectrum features are introduced for recalibration to obtain the final dense hierarchical division result; Step S108, according to the final dense hierarchical division result, using the cross-modal fusion effect evaluation mechanism based on accuracy or recall rate, if the fusion accuracy is lower than the preset threshold, adjust the adaptive convolution kernel parameters, update the feature extraction range and recalculate to obtain optimized teaching content layered output.

2. According to claim 1, a multi-source teaching data processing method based on deep learning is characterized in that: In the step S101, multi-source teaching data is obtained, and the multi-source teaching data includes text data, audio data and video data; The multi-source teaching data is processed by using modal separation technology to obtain text modal data, audio modal data and video modal data; For text modal data, a text feature extraction algorithm is used to obtain a text-independent feature set; For the audio modal data, extract the audio independent feature set through audio data processing, and determine the feature distribution of the audio independent feature set; For video modal data, video feature extraction technology is used to extract video independent feature sets and determine the modal features of the video independent feature sets; The text-independent feature set, the audio-independent feature set and the video-independent feature set are integrated to obtain a multimodal feature set; For the multimodal feature set, a clustering algorithm is used to obtain the modal feature distribution mapping result; Extract key features from the modal feature distribution mapping results and generate preliminary modal feature distribution results; Based on the preliminary modal feature distribution results, the feature space unification technique is used to obtain feature outputs suitable for downstream tasks.

3. According to the multi-source teaching data processing method based on deep learning in claim 1, it is characterized in that: In the step S102, preliminary data of modal features are obtained, and a quantitative representation of distribution differences is obtained through a preset feature distribution analysis method; According to the quantitative representation of the distribution difference, an adaptive convolution kernel adjustment algorithm is used to dynamically adjust the convolution kernel size to obtain the adjusted convolution kernel parameters; Based on the adjusted convolution kernel parameters, a feature extraction algorithm is used to extract feature data before cross-modal fusion from the modal features to determine the feature extraction range; Acquire data within the feature extraction range and generate a fusion feature set through a cross-modal fusion algorithm; If the modal consistency of the fused feature set is lower than the preset threshold, the convolution kernel size is readjusted according to the distribution difference to obtain an updated feature extraction range; Through the updated feature extraction range, the feature extraction algorithm is used to process the modal features to obtain the optimized fusion feature set; According to the optimized fusion feature set, the feature consistency of cross-modal fusion is judged through the preset verification method to obtain the final feature representation; Obtain cross-modal fusion data from dense hierarchical partitioning results, use accuracy evaluation and recall evaluation to calculate the initial fusion accuracy value; If the initial fusion accuracy value is lower than the preset threshold, the adaptive convolution kernel parameters are adjusted and the feature extraction range is updated to obtain optimized feature extraction data.

4. According to claim 1, a multi-source teaching data processing method based on deep learning is characterized in that: In the step S103, multimodal data is obtained, and the multimodal data includes text data, audio data and video data; Extracting text semantic density features from the text data; Extracting audio speech rate change features from the audio data; For the video data, extracting temporal correlation features between video frames; The text semantic density feature, the audio speech rate change feature and the video frame temporal correlation feature are integrated to obtain a preliminary multi-scale feature representation; Using a convolutional neural network to process the text semantic density features to obtain an enhanced semantic density representation; The enhanced semantic density representation and the audio speech rate change feature are integrated, and the speech rate change sequence is analyzed through a recurrent neural network to obtain a time-series enhanced semantic feature; Combining the temporal correlation features between video frames, an attention mechanism is used to process the relationship between frames to obtain a comprehensive spatiotemporal feature representation; According to the comprehensive spatiotemporal feature representation, a unified high-dimensional feature space representation is generated through network processing; Determine whether the feature distribution represented by the high-dimensional feature space meets a preset threshold, and if so, output a final feature representation; Through feature space unification technology, feature output suitable for downstream tasks is generated.

5. According to the multi-source teaching data processing method based on deep learning in claim 1, it is characterized in that: In the step S104, input data is obtained and mapped into a high-dimensional feature set; Use noise suppression algorithm to detect background noise and interference data formed by non-standard expressions in high-dimensional feature sets, and determine interference data segments; Extracting noise intensity from the interference data segment and determining whether the noise intensity exceeds a preset threshold; If it exceeds, the corresponding feature is marked as a feature to be processed; The weights of the features to be processed are adjusted by a weight decay algorithm to obtain an adjusted feature set; Clustering algorithm is used to group the adjusted feature set to obtain clear feature subsets; Reconstruct the data segment in the feature space according to the clear feature subset to obtain the feature representation after filtering; Extract the final output from the feature representation after filtering, and determine whether the final output meets the preset clarity condition; If not, return to the noise suppression algorithm step and reprocess; Obtain preliminary data of modal characteristics and obtain quantitative representation of distribution differences through a preset characteristic distribution analysis method; According to the quantitative representation of the distribution difference, an adaptive convolution kernel adjustment algorithm is used to dynamically adjust the convolution kernel size to obtain the adjusted convolution kernel parameters.

6. According to the multi-source teaching data processing method based on deep learning in claim 1, it is characterized in that: In the step S105, audio data is obtained, and a dynamic time window algorithm is used to hierarchically divide the audio data to obtain a preliminary dense hierarchical structure; For the preliminary dense hierarchical structure, the feature overlap rate of each partition boundary is calculated to obtain the fuzziness evaluation value of each partition boundary; Obtaining contextual semantic features of each partition boundary in the preliminary dense hierarchical structure, and if the fuzziness evaluation value is higher than a preset threshold, adjusting the partition boundary according to the contextual semantic features to obtain an adjusted dense hierarchical structure; Extracting frequency spectrum features of the audio data, and calibrating the adjusted dense hierarchical structure according to the frequency spectrum features to obtain a calibrated dense hierarchical structure; For each partition boundary in the calibrated dense hierarchical structure, the confidence is calculated to obtain a stability confirmation value; According to the stability confirmation value, the real-time optimization scheme is iteratively updated using the window algorithm to obtain the final dense hierarchical structure; Extract the matching relationship between dense levels and real-time optimization from the final dense hierarchical structure, and determine the dense hierarchical division scheme; Obtain a feature set in a dense hierarchical partitioning scheme, and use a microscopic difficulty perception model to scan the feature set layer by layer to obtain a feature hierarchical sequence after scanning; By matching the microscopic clues in the feature hierarchy sequence with the preset gradient template, the initial change trend of semantic complexity or speech rate change is determined, and the preliminary division boundary of the difficulty gradient is obtained.

7. The multi-source teaching data processing method based on deep learning according to claim 1 is characterized in that: In the step S106, a convolutional neural network model to be optimized is obtained, wherein the model includes at least one convolutional layer and at least one fully connected layer; The convolution layer is processed by using a lightweight network compression technology to obtain an optimized convolution layer structure; According to the optimized convolutional layer structure, the structure of the fully connected layer is adjusted to obtain streamlined fully connected layer parameters; For the simplified fully connected layer parameters, the computational complexity is evaluated to obtain a network model with reduced complexity. If the network model after complexity reduction meets the preset complexity threshold, the preliminary hierarchical structure is divided according to the combination of convolutional layer and fully connected layer to obtain the preliminary hierarchical division result; According to the preliminary hierarchical division results, analyze its resolution ability to determine whether it meets the preset real-time optimization requirements; If the preset real-time optimization requirements are not met, the optimized hierarchical division results are obtained by adjusting the division boundaries in the preliminary hierarchical division results; Determine the optimized hierarchical division result as the final real-time optimization solution; After obtaining the final real-time optimization plan, the rationality of the final hierarchical division results is confirmed by verifying the difficulty gradient of each level in the plan.

8. The multi-source teaching data processing method based on deep learning according to claim 1 is characterized in that: In the step S107, audio data is obtained, and a dynamic time window algorithm is used to hierarchically divide the audio data to obtain a preliminary dense hierarchical structure; For the preliminary dense hierarchical structure, the feature overlap rate of each partition boundary is calculated to obtain the fuzziness evaluation value of each partition boundary; Obtaining contextual semantic features of each partition boundary in the preliminary dense hierarchical structure, and if the fuzziness evaluation value is higher than a preset threshold, adjusting the partition boundary according to the contextual semantic features to obtain an adjusted dense hierarchical structure; Extracting frequency spectrum features of the audio data, and calibrating the adjusted dense hierarchical structure according to the frequency spectrum features to obtain a calibrated dense hierarchical structure; For each partition boundary in the calibrated dense hierarchical structure, the confidence is calculated to obtain a stability confirmation value; According to the stability confirmation value, the real-time optimization scheme is iteratively updated using the window algorithm to obtain the final dense hierarchical structure; Extract the matching relationship between dense levels and real-time optimization from the final dense hierarchical structure, and determine the dense hierarchical division scheme; Obtain a feature set, and use a microscopic difficulty perception model to scan the feature set layer by layer to obtain a feature level sequence after scanning; By matching the micro clues in the feature hierarchy sequence with a preset gradient template, the preliminary change trend of semantic complexity or speech speed change is determined. Based on the preliminary change trend, it is judged whether there is a significant fluctuation in the difficulty gradient to obtain the fluctuation detection result.

9. The multi-source teaching data processing method based on deep learning according to claim 1 is characterized in that: In the step S108, the cross-modal fusion data in the dense hierarchical division result is obtained, and the initial fusion accuracy value is calculated by using the accuracy evaluation and the recall evaluation; If the initial fusion accuracy value is lower than the preset threshold, the adaptive convolution kernel parameters are adjusted, the feature extraction range is updated, and the optimized feature extraction data is obtained; Perform cross-modal fusion processing on the optimized feature extraction data to obtain an updated fusion accuracy value; Determine whether the updated fusion accuracy value is lower than a preset threshold. If so, repeat the adaptive convolution kernel parameter adjustment and feature extraction range update to obtain optimized feature extraction data again. Recalculate the cross-modal fusion result based on the re-optimized feature extraction data to obtain a fusion accuracy value that meets the preset threshold; Extract teaching content from the fusion accuracy values ​​that meet the preset threshold, and generate preliminary hierarchical output data in combination with the dense hierarchical division structure; Format the preliminary hierarchical output data and adopt the hierarchical output mechanism to obtain the final optimized hierarchical output results of teaching content; Obtain preliminary data of modal characteristics and obtain quantitative representation of distribution differences through a preset characteristic distribution analysis method; According to the quantitative representation of the distribution difference, an adaptive convolution kernel adjustment algorithm is used to dynamically adjust the convolution kernel size to obtain the adjusted convolution kernel parameters.

Images