Video education resource query method and system based on natural language processing

Abstract

The application discloses a video education resource query method and system based on natural language processing, relates to the technical field of natural language processing, and comprises the following steps: extracting knowledge vectors in video education resource data according to a multi-modal natural language processing network, and constructing a video education resource knowledge graph; analyzing the multi-dimensional nonlinear topological structure between adjacent knowledge points in a knowledge path sequence, training a natural language teaching intention classifier, and obtaining the teaching rationality confidence between knowledge point pairs; inputting the knowledge point sequence of real-time query voice into the natural language teaching intention classifier, generating the teaching rationality confidence of the knowledge point pair of real-time query voice, performing maximum probability path search on the video education resource knowledge graph, and generating optimal matching video education resources of real-time query voice. The beneficial effect lies in solving the problem that the query result deviates from the real intention of a user due to the knowledge point association jump in a nonlinear and multi-association education scene.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, specifically to a method and system for querying video educational resources based on natural language processing. Background Technology

[0002] Existing video educational resource query technologies, whether based on keyword-based literal matching or semantic vector-based similarity retrieval, treat user queries as a "point matching" problem of one or more isolated knowledge points, assuming by default that knowledge points follow a single linear progression. However, in educational resource scenarios, knowledge points in natural conversational language exhibit a complex topological structure of non-linear jumps and many-to-many interweaving. Existing technical solutions cannot model the "cognitive jump" events implicit in user queries, i.e., they cannot identify the necessary prerequisite knowledge or connecting concepts missing when jumping from one knowledge point to another not directly related knowledge point. This results in search results often only hitting a fragment of an isolated knowledge point explicitly mentioned in the query, ignoring the complete knowledge logic chain that the user actually needs to understand. Therefore, in non-linear, multi-relational educational scenarios, existing solutions generally suffer from technical defects due to their inability to perceive and handle the jumps and connections between knowledge points, causing search results to deviate from the user's true learning intentions. Summary of the Invention

[0003] To address the aforementioned technical issues, this technical solution provides a method and system for querying video educational resources based on natural language processing, thus resolving the problems described above.

[0004] To achieve the above objectives, the technical solution adopted by the present invention is as follows: Methods for querying video educational resources based on natural language processing include: S1. Based on the video education resource database of the digital multimedia cloud platform, according to the multimodal natural language processing network, extract knowledge vectors from the video education resource data, analyze the probability distribution of the pre- and post-relationship of knowledge vectors in video education resources, and construct a knowledge graph of video education resources; S2. Based on the knowledge graph of video education resources, analyze the multidimensional nonlinear topological structure between adjacent knowledge points in the knowledge path sequence, quantify the nonlinear feature vector of knowledge point pairs, train a natural language teaching intention classifier, and obtain the teaching rationality confidence between knowledge point pairs. S3. Based on edge sensors, the knowledge point sequence of real-time query speech is obtained and fed into a natural language teaching intent classifier to generate the teaching rationality confidence of the real-time query speech knowledge point pair. The maximum probability path search of the video education resource knowledge graph is then performed to generate the optimal matching video education resource for the real-time query speech.

[0005] Preferably, step S1 specifically includes: Based on the video education resource database, using a unit time as the observation window and video education resource data as the observation object, the audio segment sequence and image frame sequence of the video education resources are extracted. Based on the Transformer architecture, a multimodal natural language processing network is constructed using lightweight GLM-ASR automatic speech recognition as the auditory channel encoder and YOLOv8 target detection as the visual channel encoder. Specifically, the auditory channel encoder is: Using the audio segment sequence of video educational resources as input for lightweight GLM-ASR automatic speech recognition, the audio segment signal is segmented into overlapping audio frames. The Mel-frequency cepstral coefficients of the audio frames are calculated as acoustic features of the audio segment signal. The latent state sequence of the video educational resources is input to the encoder to extract the acoustic features. The latent state sequence of the video educational resources is decoded by an autoregressive decoder to generate the audio segment character sequence of the video educational resources. The latent state sequence of the video educational resources is input to a cross-attention layer to calculate the probability distribution of the characters to be decoded in the audio segment. The probability distribution of the decoded characters is searched according to the beam search to obtain the audio segment transcribed text sequence of the video educational resources. Specifically, the visual channel encoder is: Using the image frame sequence of video educational resources as the input for YOLOv8 target monitoring, image frames with pixel differences exceeding a threshold between adjacent frames are removed according to the mean square error sampling strategy. The pixel values ​​of the video educational resources' image frames are then normalized before being input into the backbone network. Convolutional layers are used for downsampling to obtain the feature images of the video educational resources. The CSP stage divides the video educational resources' images into two paths. The first path passes through several residual blocks to obtain deep, high-dimensional features of the video educational resources' feature images. The second path reaches the end of the CSP stage to obtain low-level detail features of the video educational resources' feature images. Additive concatenation combines the deep, high-dimensional features of the video educational resources' feature images with the low-level detail features. Convolutional layers output feature maps of the video educational resources at three scales. These are then input to a target bounding box regression detection head, a target category recognition detection head, and a target confidence detection head to obtain the image target bounding box offset, the image target category probability, and the image target confidence of the video educational resources. A three-dimensional tensor of the video educational resources' image is constructed and input to the CTC decoder for OCR text recognition monitoring, outputting a sequence of image frames transcribed into text from the video educational resources.

[0006] Preferably, step S1 further includes: The SemanticNet semantic network is embedded as an attention mechanism to extract the topical global semantic vector between the metadata topic text of video educational resources and the audio segment transcribed text sequence and image frame transcribed text sequence of video educational resources. An audio-video text sequence modality stacking matrix of video educational resources is established. Learnable parameters are randomly initialized as queries, and the audio-video modality of video educational resources is used as the key. The attention score of each audio segment transcribed text modality and image frame transcribed text modality under the global semantic vector is calculated. The scores are then normalized by the Softmax function to obtain the global semantic vector weight, audio text vector weight, and video text vector weight of video educational resources. The knowledge vector in the video educational resource data is generated by weighted fusion.

[0007] Preferably, step S1 further includes: Based on the set time window constraint, the knowledge vectors in the video education resource data are sequentially traversed. The knowledge vectors in the video education resources whose time interval between the preceding and following relationships is less than the preset threshold time are retained. The cosine similarity between each knowledge point pair is calculated. The positive correlation similarity is used for secondary filtering to obtain the set of preceding and following knowledge point pairs in the video education resource data. Determine the prior probability distribution of the teaching logic type for known knowledge points; Based on the set of pre- and post-knowledge point pairs in the video education resource data, according to the pre- and post-knowledge point dependencies, with the pre-knowledge point as the parent node and the post-knowledge point as the child node, and the prior probability distribution of the teaching logic type of the known knowledge point as the edge between the parent node and the child node, a directed graph of video education resource knowledge is obtained. Based on the directed graph of video education resources, and according to the prior probability distribution of the teaching logic type of known knowledge points in the knowledge point pair set, and the given observation of the preceding and following time order and the cosine similarity of the knowledge vector, the posterior probability of the preceding and following teaching relationship between each knowledge point pair is calculated as the weight of the corresponding directed edge by Bayes inference, thus constructing a knowledge graph of video education resources.

[0008] Preferably, step S2 specifically includes: Based on any two knowledge point pairs in the knowledge graph of video education resources, linear structured paths and non-linear structured paths are marked according to the maximum prior path. Based on the linear structured path in the knowledge graph of video educational resources, verify the expected path of any adjacent knowledge points in the linear structured path, and calculate the topological deviation contribution value of knowledge point pairs in the nonlinear structured path. Based on second-order difference, this paper analyzes the angle between the instantaneous velocity vector and instantaneous acceleration vector of knowledge point pairs in the topic global semantic vector and the cosine similarity of knowledge point pairs in the topic global semantic vector in the nonlinear structured path of video education resource knowledge graph, and calculates the topological nonlinear mutation degree index of knowledge point pairs in the nonlinear structured path. Based on the knowledge graph of video education resources, the first filter is the pre- and post-dependent relationship of knowledge point pairs. All sets of directly preceding knowledge points for knowledge point pairs are extracted. Random walks are performed on the association paths of the sets of directly preceding knowledge point pairs, and sets of irrelevant knowledge points on the non-connected paths of directly preceding knowledge points are marked. Combine the set of all knowledge point pairs that are directly preceding knowledge points with the set of knowledge points that are not related to the direct preceding knowledge points on non-connected paths into a knowledge point pair dependency gap set. Based on the set of knowledge point pair dependency gaps and the adjacent knowledge point pairs of knowledge point pair dependency gaps, calculate the nonlinearity of the teaching logic of knowledge point pair dependency gaps in the nonlinear structured path. The nonlinear feature vector of knowledge point pairs is obtained by additively concatenating and fusing the teaching logic nonlinearity of the knowledge point pair dependency gap in the nonlinear structured path, the topological nonlinear mutation degree index of the knowledge point pair in the nonlinear structured path, and the topological deviation contribution value of the knowledge point pair in the nonlinear structured path.

[0009] Preferably, step S2 further includes: Based on decision forest, a natural language teaching intention classifier is constructed. The knowledge graph of video education resources is used as the root node of the rule decision tree, the nonlinear feature vector of knowledge point pairs is used as the branch nodes of the rule decision tree, and the nonlinear topological structure type of knowledge point pairs is used as the leaf nodes. The normalized value of the ratio of the voting distribution of the nonlinear topological structure type of each leaf node of the decision forest to the total number of trees is calculated to determine the teaching rationality confidence between knowledge point pairs.

[0010] Preferably, step S3 specifically includes: Based on real-time query speech, a multimodal natural language processing network is used to obtain the knowledge point sequence of real-time query speech. The sequence is then fed into a natural language teaching intention classifier. According to the voting distribution of decision forest, the teaching rationality confidence of the knowledge point pairs of real-time query speech is determined, and a confidence sequence of real-time query speech knowledge point pairs is established. Based on the knowledge point sequence and the confidence sequence of knowledge point pairs in the real-time query speech, the Viterbi algorithm of weighted directed graph is used to perform maximum probability path search. The weights of all adjacent edges on the path are multiplied together to determine the maximum probability of the knowledge point pair path in the real-time query speech, thus obtaining the optimal matching video educational resource for the real-time query speech.

[0011] Furthermore, a video educational resource query system based on natural language processing is proposed to implement the video educational resource query method based on natural language processing described above, including: The system comprises a video education resource knowledge graph module, a knowledge nonlinear relationship module, and a knowledge intent recognition module. The knowledge nonlinear relationship module is electrically connected to the video education resource knowledge graph module, and the knowledge intent recognition module is electrically connected to the knowledge nonlinear relationship module. The education resource knowledge graph module is based on the video education resource database of the digital multimedia cloud platform. Following a multimodal natural language processing network, it extracts knowledge vectors from the video education resource data, analyzes the probability distribution of the pre- and post-relationships of the knowledge vectors, and constructs a video education resource knowledge graph. Knowledge Nonlinear Relationship Module: Based on the knowledge graph of video education resources, it analyzes the multidimensional nonlinear topological structure between adjacent knowledge points in the knowledge path sequence, quantifies the nonlinear feature vector of knowledge point pairs, trains a natural language teaching intention classifier, and obtains the teaching rationality confidence of knowledge point pairs. Knowledge Intent Recognition Module: Based on edge sensors, the module acquires the knowledge point sequence of real-time query speech and inputs it into a natural language teaching intent classifier. It then generates the teaching rationality confidence score of the real-time query speech knowledge point pair and performs a maximum probability path search on the video education resource knowledge graph to generate the optimal matching video education resource for the real-time query speech.

[0012] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention proposes a video education resource query scheme using natural language processing. S1 constructs a video education resource knowledge graph to explicitly model the multi-dimensional relationship structure between knowledge points. S2 quantifies and analyzes the multi-dimensional nonlinear topological structure between adjacent knowledge points and trains a teaching intent classifier to identify the teaching rationality of knowledge point pairs. S3 maps real-time query speech to a sequence of knowledge points, which is then judged by the classifier and subjected to a maximum probability path search. Compared to existing solutions that rely solely on single-point keyword matching or isolated vector retrieval, this invention proactively senses the knowledge point jumps and nonlinear relationships implicit in the user's query, automatically completing the skipped necessary prerequisite knowledge or connecting concepts. It reconstructs discrete, jumpy query intentions into continuous knowledge paths that conform to the progressive teaching logic, thereby accurately matching video education resource segments containing interwoven relationships of multiple knowledge points. This solves the problem of query results deviating from the user's true intention due to knowledge point relationship jumps in nonlinear, multi-relational educational scenarios. Attached Figure Description

[0013] Figure 1 Here is a flowchart of a video educational resource query method based on natural language processing; Figure 2 This is a flowchart of a video-based educational resource query system based on natural language processing. Detailed Implementation

[0014] The following description is intended to disclose the invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art.

[0015] Reference Figure 1 As shown, the video educational resource query method based on natural language processing includes: S1. Based on the video education resource database of the digital multimedia cloud platform, according to the multimodal natural language processing network, extract knowledge vectors from the video education resource data, analyze the probability distribution of the pre- and post-relationship of knowledge vectors in video education resources, and construct a knowledge graph of video education resources; Step S1 specifically includes: Based on the video education resource database, using a unit time as the observation window and video education resource data as the observation object, the audio segment sequence and image frame sequence of the video education resources are extracted. Based on the Transformer architecture, a multimodal natural language processing network is constructed using lightweight GLM-ASR automatic speech recognition as the auditory channel encoder and YOLOv8 target detection as the visual channel encoder. Specifically, the auditory channel encoder is: Using the audio segment sequence of video educational resources as input for lightweight GLM-ASR automatic speech recognition, the audio segment signal is segmented into overlapping audio frames. The Mel-frequency cepstral coefficients of the audio frames are calculated as acoustic features of the audio segment signal. The latent state sequence of the video educational resources is input to the encoder to extract the acoustic features. The latent state sequence of the video educational resources is decoded by an autoregressive decoder to generate the audio segment character sequence of the video educational resources. The latent state sequence of the video educational resources is input to a cross-attention layer to calculate the probability distribution of the characters to be decoded in the audio segment. The probability distribution of the decoded characters is searched according to the beam search to obtain the audio segment transcribed text sequence of the video educational resources. Specifically, the visual channel encoder is: Using the image frame sequence of video educational resources as the input for YOLOv8 target monitoring, image frames with pixel differences exceeding a threshold between adjacent frames are removed according to the mean square error sampling strategy. The pixel values ​​of the video educational resources' image frames are then normalized before being input into the backbone network. Convolutional layers are used for downsampling to obtain the feature images of the video educational resources. The CSP stage divides the video educational resources' images into two paths. The first path passes through several residual blocks to obtain deep, high-dimensional features of the video educational resources' feature images. The second path reaches the end of the CSP stage to obtain low-level detail features of the video educational resources' feature images. Additive concatenation combines the deep, high-dimensional features of the video educational resources' feature images with the low-level detail features. Convolutional layers output feature maps of the video educational resources at three scales. These are then input to a target bounding box regression detection head, a target category recognition detection head, and a target confidence detection head to obtain the image target bounding box offset, the image target category probability, and the image target confidence of the video educational resources. A three-dimensional tensor of the video educational resources' image is constructed and input to the CTC decoder for OCR text recognition monitoring, outputting a sequence of image frames transcribed into text from the video educational resources.

[0016] Step S1 also includes: The SemanticNet semantic network is embedded as an attention mechanism to extract the topic global semantic vector between the metadata topic text of video educational resources and the audio segment transcribed text sequence and image frame transcribed text sequence of video educational resources. The audio-video text sequence modality stacking matrix of video educational resources is established. The learnable parameters are randomly initialized as queries and the audio-video modality of video educational resources is used as keys. The attention scores of each audio segment transcribed text modality and image frame transcribed text modality under the global semantic vector are calculated. The scores are normalized by the Softmax function to obtain the global semantic vector weight, audio text vector weight and video text vector weight of video educational resources. The knowledge vector in the video educational resource data is generated by weighted fusion. As further content, an 8-second audio clip from a video educational resource is provided, in which the teacher explains that the core of backpropagation is the chain rule, serving as an example of the implementation of an auditory channel encoder: The 8-second audio file was divided into 25-millisecond frames with a 10-millisecond frame shift, generating approximately 800 audio frames. There is a 15-millisecond overlap between adjacent frames to ensure the continuity of acoustic information.

[0017] For each 25-millisecond audio frame, its Mel frequency cepstral coefficients are calculated: first, the time-domain signal is converted to the frequency domain through a fast Fourier transform, then mapped to 40 filter banks on the Mel scale, and finally the first 13-dimensional coefficients are extracted as the acoustic feature vector of the frame through a discrete cosine transform.

[0018] Output: An acoustic feature matrix with 800 rows and 13 columns, which serves as the input to the subsequent encoder.

[0019] The decoder inputs the currently generated character sequence "The core of backpropagation is" and the hidden state sequence output by the encoder into the cross-attention layer to calculate the probability distribution of the character to be decoded. The model predicts the probability of the character "chain" to be 0.7, the probability of the character "of" to be 0.2, and the probability of the character "calculate" to be 0.1.

[0020] The bundle width is set to 3, and the three candidate sequences with the highest probabilities are retained: "chain", "of", and "calculate". In the next decoding step, each candidate is expanded and sorted again, and finally, the "chain rule" with the highest overall probability is selected as the decoding result at the end of the sequence.

[0021] Output: A sequence of audio segments transcribed into text, with the core principle of backpropagation being the chain rule, along with a timestamp for each character.

[0022] Furthermore, the PPT slides remain still from the 5th to the 15th second, with the teacher only giving verbal explanations and no page turning. This serves as an implementation example of a visual channel encoder. The system extracts image frames at a frequency of 1 frame per second, resulting in 10 images from the 5th to the 15th second.

[0023] Calculate the pixel mean square error between the 5.0 second frame and the 6.0 second frame. If the difference is less than the preset difference threshold (e.g., 10%), it is determined that the two frames are highly similar.

[0024] Redundant frames at 6.0 seconds, 7.0 seconds, 8.0 seconds, and 9.0 seconds are removed, and only frames at 5.0 seconds and 10.0 seconds are retained as representative frames for this time window.

[0025] After being downsampled by the backbone network, the image enters the CSP stage. The first path passes through 4 residual blocks, gradually extracting deep, high-dimensional features of the image: the first residual block identifies edges and textures, the second identifies text regions, the third identifies curve shapes, and the fourth abstracts the curve shape into the semantic concept of a "loss function curve".

[0026] The second path maintains the original resolution, reaching the end of the stage and preserving the precise location of the text area and the detailed texture of the curve.

[0027] The outputs of the first path and the second path are concatenated along the channel dimension to form a feature map that simultaneously contains high-level semantics ("This is an explanation of the loss function") and low-level details ("The position of the title text, the coordinates of the curve inflection point").

[0028] After the target detection head locates the text area, it is sent to the OCR recognition module. The CTC decoder outputs the character probability sequence: position 1 predicts "d" with a probability of 0.9, position 2 predicts "y" with a probability of 0.8, position 3 predicts " / " with a probability of 0.7, position 4 predicts "d" with a probability of 0.6, and position 5 predicts "x" with a probability of 0.9.

[0029] The CTC decoder uses dynamic programming to merge duplicate characters and remove whitespace placeholders, decoding the above probability sequence as "dy / dx".

[0030] Output: The image frame is transcribed into the text sequence "dy / dx", and the normalized coordinates of the text in the image are recorded (e.g., top left corner 0.3, 0.2, bottom right corner 0.5, 0.3).

[0031] Furthermore, the video metadata provided the chapter title "Neural Network Basics: Backpropagation," and the audio transcribed text contained "chain rule" and "gradient calculation," which were then recognized by OCR.

[0032] Input the metadata title “Neural Network Basics: Backpropagation” into the SemanticNet semantic network to extract the topic semantic vector, which is close to “Deep Learning” and “Error Backpropagation” in its high-dimensional space.

[0033] Simultaneously, the audio-transcribed text and the OCR-recognized text are encoded separately to obtain their respective semantic vectors.

[0034] The dot product of the topic semantic vector and the audio text vector yields an attention score of 0.85; the dot product with the video text vector yields an attention score of 0.62. After Softmax normalization, the audio modality weight is 0.58, and the video modality weight is 0.42. These weights indicate that the audio content in the current segment is more thematically representative than the written text.

[0035] In the same video clip, the audio-to-text vector is V_audio, and the OCR whiteboard text vector is V_vision. After step 6, the audio weight is calculated to be 0.58 and the visual weight is 0.42.

[0036] Implementation process: Perform weighted fusion: V_knowledge = 0.58 × V_audio + 0.42 × V_vision.

[0037] The fused knowledge vector simultaneously embodies "the oral explanation of the chain rule in the audio" and "the formulaic expression of ∂L / ∂w in the blackboard" in the vector space.

[0038] Step S1 also includes: Based on the set time window constraint, the knowledge vectors in the video education resource data are sequentially traversed. The knowledge vectors in the video education resources whose time interval between the preceding and following relationships is less than the preset threshold time are retained. The cosine similarity between each knowledge point pair is calculated. The positive correlation similarity is used for secondary filtering to obtain the set of preceding and following knowledge point pairs in the video education resource data. Determine the prior probability distribution of the teaching logic type for known knowledge points; As a further explanation, the teaching logic types of known knowledge points include: prerequisite dependency type, where the parent node knowledge point is a necessary prerequisite for understanding the child node knowledge point; sequential explanation type, where the parent node and child node belong to the same knowledge module and appear in the teaching order, but are not strictly dependent; example and instantiation type, where the parent node is an abstract concept and the child node is a specific instance or application of that concept; and comparison and differentiation type, where the parent node and child node are two concepts that are compared and contrasted. Based on the set of pre- and post-knowledge point pairs in the video education resource data, according to the pre- and post-knowledge point dependencies, with the pre-knowledge point as the parent node and the post-knowledge point as the child node, and the prior probability distribution of the teaching logic type of the known knowledge point as the edge between the parent node and the child node, a directed graph of video education resource knowledge is obtained. Based on the directed graph of video education resources, and according to the prior probability distribution of the teaching logic type of known knowledge points in the knowledge point pair set, and the given observation of the preceding and following time order and the cosine similarity of the knowledge vector, the posterior probability of the preceding and following teaching relationship between each knowledge point pair is calculated as the weight of the corresponding directed edge by Bayes inference, thus constructing a knowledge graph of video education resources.

[0039] S2. Based on the knowledge graph of video education resources, analyze the multidimensional nonlinear topological structure between adjacent knowledge points in the knowledge path sequence, quantify the nonlinear feature vector of knowledge point pairs, train a natural language teaching intention classifier, and obtain the teaching rationality confidence between knowledge point pairs. Step S2 specifically includes: Based on any two knowledge point pairs in the knowledge graph of video education resources, linear structured paths and non-linear structured paths are marked according to the maximum prior path. Based on the linear structured paths in the knowledge graph of video educational resources, the expected paths of any adjacent knowledge points in the linear structured paths are verified, and the topological deviation contribution value of knowledge point pairs in non-linear structured paths is calculated, specifically: , in, This represents the contribution value of topological deviation of knowledge point pairs in the non-linear structured path of the video educational resource knowledge graph. Knowledge point nodes in the linear structured path of video educational resource knowledge graph To knowledge point node The shortest path length, Knowledge point nodes in the linear structured path of video educational resource knowledge graph To knowledge point node The expected path length of the teaching logic type. It is a very small positive number. The total number of knowledge points in the knowledge graph of video educational resources; As a further explanation, let's take the two knowledge point nodes "derivative" and "integral" in the high school mathematics knowledge graph as examples. In this knowledge graph, the expected path length of the standard teaching logic from "derivative" to "integral" is 1 (i.e., expected to be taught directly in sequence), while the actual shortest path length of the two knowledge point nodes in the graph is 2 (requiring "indefinite integral" as an intermediate node). Let the minimum positive number ε = 0.001, and the total number of knowledge points n be the number of knowledge point pairs in the current analysis path. Substituting into the formula, we can calculate the topological deviation contribution value NL_topo = (2 / 1.001) ≈ 1.998. This value is greater than 1, indicating that the actual knowledge point pair path deviates from the expected standard teaching path, exhibiting obvious topological nonlinear characteristics.

[0040] Based on second-order difference, this paper analyzes the angle between the instantaneous velocity vector and instantaneous acceleration vector of knowledge point pairs in the topic global semantic vector and the cosine similarity of the knowledge point pairs in the topic global semantic vector within the nonlinear structured path of the video educational resource knowledge graph. The topological nonlinearity abrupt change index of knowledge point pairs in the nonlinear structured path is calculated, specifically: , in, This is an index for the topological nonlinear mutation degree of knowledge point pairs in the nonlinear structured path of video educational resource knowledge graph. Let t be the instantaneous velocity vector of the t-th knowledge point pair in the global semantic vector of the topic. Let t be the instantaneous acceleration vector of the t-th knowledge point pair in the topic's global semantic vector. Let be the cosine similarity angle between the t-th knowledge point pair and the global semantic vector of the topic; As a further explanation, we will use the two knowledge point nodes "Convolutional Neural Network" and "Recurrent Neural Network" in the deep learning knowledge graph as examples. First, we obtain the embedding vectors of the two knowledge points in the pre-trained semantic space. We calculate the instantaneous velocity vector u_t from "Convolutional Neural Network" to "Recurrent Neural Network," and then combine it with the knowledge point pair from the previous time step to calculate the instantaneous acceleration vector a_t. Simultaneously, we calculate the cosine similarity angle θ_t between the embedding vectors of the two knowledge points. Setting ε=0.001, we substitute it into the formula to calculate the NL_sem value. The calculation shows that the cosine similarity between the two knowledge points is high (approximately 0.72), but the semantic displacement direction changes significantly, and the acceleration magnitude is large. The final calculated NL_sem value is approximately 0.83. This value is greater than the 0.5 threshold, indicating that there is a significant non-linear abrupt change in the semantic space of the knowledge point pair, belonging to the teaching jump type of comparative learning or analogical reasoning.

[0041] Based on the knowledge graph of video education resources, the first filter is the pre- and post-dependent relationship of knowledge point pairs. All sets of directly preceding knowledge points for knowledge point pairs are extracted. Random walks are performed on the association paths of the sets of directly preceding knowledge point pairs, and sets of irrelevant knowledge points on the non-connected paths of directly preceding knowledge points are marked. Combine the set of all knowledge point pairs that are directly preceding knowledge points with the set of knowledge points that are not related to the direct preceding knowledge points on non-connected paths into a knowledge point pair dependency gap set. Based on the set of knowledge point dependency gaps and the adjacent knowledge point pairs of the knowledge point dependency gaps, the nonlinearity of the instructional logic of the knowledge point dependency gaps in the nonlinear structured path is calculated, specifically as follows: , in, This refers to the nonlinearity of the teaching logic of the dependency gaps between knowledge points in the nonlinear structured path of video educational resource knowledge graphs. For knowledge point pairs, the prerequisite knowledge point pairs in the knowledge point pair dependency gap set With the The number of dependency gaps for each pair of prerequisite knowledge points. For the knowledge point in the dependency gap set, the first The size of each pair of prerequisite knowledge points As an indicator function, the current knowledge point pair With the The value is 1 if all prerequisite knowledge points are different, and 0 if they are the same. It is a positive constant; The nonlinear feature vector of knowledge point pairs is obtained by adding and splicing together the nonlinearity of the teaching logic of the knowledge point pair dependency gap in the nonlinear structured path, the topological nonlinear mutation index of the knowledge point pair in the nonlinear structured path, and the topological deviation contribution value of the knowledge point pair in the nonlinear structured path. As a further explanation, let's take the "loss function" and "gradient descent" knowledge nodes in the machine learning knowledge graph as examples. The direct prerequisite knowledge set for "gradient descent" includes three knowledge points: "derivative," "partial derivative," and "function extrema." The preset knowledge path from "loss function" to "gradient descent" only passes through the "differentiability condition" node and does not pass through any of the three prerequisite knowledge points mentioned above; therefore, the dependency gap set size is 3. The current knowledge point is different for "loss function" and "gradient descent," so the indicator function value is 1. Setting ε=0.001, substituting into the formula, we get the teaching logic nonlinearity NL_ped=3 / (3+0.001)≈0.9997. This value is close to 1, indicating that when a user jumps directly from "loss function" to "gradient descent," they completely lack all three prerequisite knowledge points necessary for the learning target knowledge point. This is a typical cognitive jump event, requiring system intervention to bridge the knowledge gap.

[0042] Step S2 also includes: Based on decision forest, a natural language teaching intention classifier is constructed. The video education resource knowledge graph is used as the root node of the rule decision tree, the nonlinear feature vector of knowledge point pair is used as the branch node of the rule decision tree, and the nonlinear topological structure type of knowledge point pair is used as the leaf node. The normalized value of the ratio of the voting distribution of the nonlinear topological structure type of each leaf node of the decision forest to the total number of trees is calculated to determine the teaching rationality confidence between knowledge point pairs. S3. Based on edge sensors, the knowledge point sequence of real-time query voice is obtained and substituted into the natural language teaching intention classifier to generate the teaching rationality confidence of the real-time query voice knowledge point pair. The maximum probability path search of the video education resource knowledge graph is then performed to generate the optimal matching video education resource for the real-time query voice. Step S3 specifically includes: Based on real-time query speech, a multimodal natural language processing network is used to obtain the knowledge point sequence of real-time query speech. The sequence is then fed into a natural language teaching intention classifier. According to the voting distribution of decision forest, the teaching rationality confidence of the knowledge point pairs of real-time query speech is determined, and a confidence sequence of real-time query speech knowledge point pairs is established. Based on the knowledge point sequence and the confidence sequence of knowledge point pairs in the real-time query speech, the Viterbi algorithm of weighted directed graph is used to perform maximum probability path search. The weights of all adjacent edges on the path are multiplied together to determine the maximum probability of the knowledge point pair path in the real-time query speech, thus obtaining the optimal matching video educational resource for the real-time query speech.

[0043] Step S4: A video educational resource query system based on natural language processing, including: The system comprises a video education resource knowledge graph module, a knowledge nonlinear relationship module, and a knowledge intent recognition module. The knowledge nonlinear relationship module is electrically connected to the video education resource knowledge graph module, and the knowledge intent recognition module is electrically connected to the knowledge nonlinear relationship module. The education resource knowledge graph module is based on the video education resource database of the digital multimedia cloud platform. Following a multimodal natural language processing network, it extracts knowledge vectors from the video education resource data, analyzes the probability distribution of the pre- and post-relationships of the knowledge vectors, and constructs a video education resource knowledge graph. Knowledge Nonlinear Relationship Module: Based on the knowledge graph of video education resources, it analyzes the multidimensional nonlinear topological structure between adjacent knowledge points in the knowledge path sequence, quantifies the nonlinear feature vector of knowledge point pairs, trains a natural language teaching intention classifier, and obtains the teaching rationality confidence of knowledge point pairs. Knowledge Intent Recognition Module: Based on edge sensors, the module acquires the knowledge point sequence of real-time query speech and inputs it into a natural language teaching intent classifier. It then generates the teaching rationality confidence score of the real-time query speech knowledge point pair and performs a maximum probability path search on the video education resource knowledge graph to generate the optimal matching video education resource for the real-time query speech.

[0044] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention. The scope of protection claimed by the appended claims and their equivalents is defined.

Claims

1. A video educational resource query method based on natural language processing, characterized in that, include: S1. Based on the video education resource database of the digital multimedia cloud platform, according to the multimodal natural language processing network, extract knowledge vectors from the video education resource data, analyze the probability distribution of the pre- and post-relationship of knowledge vectors in video education resources, and construct a knowledge graph of video education resources; S2. Based on the knowledge graph of video education resources, analyze the multidimensional nonlinear topological structure between adjacent knowledge points in the knowledge path sequence, quantify the nonlinear feature vector of knowledge point pairs, train a natural language teaching intention classifier, and obtain the teaching rationality confidence between knowledge point pairs. S3. Based on edge sensors, the knowledge point sequence of real-time query speech is obtained and fed into a natural language teaching intent classifier to generate the teaching rationality confidence of the real-time query speech knowledge point pair. The maximum probability path search of the video education resource knowledge graph is then performed to generate the optimal matching video education resource for the real-time query speech.

2. The video educational resource query method based on natural language processing according to claim 1, characterized in that, Step S1 specifically includes: Based on the video education resource database, using a unit time as the observation window and video education resource data as the observation object, the audio segment sequence and image frame sequence of the video education resources are extracted. Based on the Transformer architecture, a multimodal natural language processing network is constructed using lightweight GLM-ASR automatic speech recognition as the auditory channel encoder and YOLOv8 target detection as the visual channel encoder. Specifically, the auditory channel encoder is: Using the audio segment sequence of video educational resources as input for lightweight GLM-ASR automatic speech recognition, the audio segment signal is segmented into overlapping audio frames. The Mel-frequency cepstral coefficients of the audio frames are calculated as acoustic features of the audio segment signal. The latent state sequence of the video educational resources is input to the encoder to extract the acoustic features. The latent state sequence of the video educational resources is decoded by an autoregressive decoder to generate the audio segment character sequence of the video educational resources. The latent state sequence of the video educational resources is input to a cross-attention layer to calculate the probability distribution of the characters to be decoded in the audio segment. The probability distribution of the decoded characters is searched according to the beam search to obtain the audio segment transcribed text sequence of the video educational resources. Specifically, the visual channel encoder is: Using the image frame sequence of video educational resources as the input for YOLOv8 target monitoring, image frames with pixel differences exceeding a threshold between adjacent frames are removed according to the mean square error sampling strategy. The pixel values ​​of the video educational resources' image frames are then normalized before being input into the backbone network. Convolutional layers are used for downsampling to obtain the feature images of the video educational resources. The CSP stage divides the video educational resources' images into two paths. The first path passes through several residual blocks to obtain deep, high-dimensional features of the video educational resources' feature images. The second path reaches the end of the CSP stage to obtain low-level detail features of the video educational resources' feature images. Additive concatenation combines the deep, high-dimensional features of the video educational resources' feature images with the low-level detail features. Convolutional layers output feature maps of the video educational resources at three scales. These are then input to a target bounding box regression detection head, a target category recognition detection head, and a target confidence detection head to obtain the image target bounding box offset, the image target category probability, and the image target confidence of the video educational resources. A three-dimensional tensor of the video educational resources' image is constructed and input to the CTC decoder for OCR text recognition monitoring, outputting a sequence of image frames transcribed into text from the video educational resources.

3. The video educational resource query method based on natural language processing according to claim 2, characterized in that, Step S1 also includes: The SemanticNet semantic network is embedded as an attention mechanism to extract the topical global semantic vector between the metadata topic text of video educational resources and the audio segment transcribed text sequence and image frame transcribed text sequence of video educational resources. An audio-video text sequence modality stacking matrix of video educational resources is established. Learnable parameters are randomly initialized as queries, and the audio-video modality of video educational resources is used as the key. The attention score of each audio segment transcribed text modality and image frame transcribed text modality under the global semantic vector is calculated. The scores are then normalized by the Softmax function to obtain the global semantic vector weight, audio text vector weight, and video text vector weight of video educational resources. The knowledge vector in the video educational resource data is generated by weighted fusion.

4. The video educational resource query method based on natural language processing according to claim 3, characterized in that, Step S1 also includes: Based on the set time window constraint, the knowledge vectors in the video education resource data are sequentially traversed. The knowledge vectors in the video education resources whose time interval between the preceding and following relationships is less than the preset threshold time are retained. The cosine similarity between each knowledge point pair is calculated. The positive correlation similarity is used for secondary filtering to obtain the set of preceding and following knowledge point pairs in the video education resource data. Determine the prior probability distribution of the teaching logic type for known knowledge points; Based on the set of pre- and post-knowledge point pairs in the video education resource data, according to the pre- and post-knowledge point dependencies, with the pre-knowledge point as the parent node and the post-knowledge point as the child node, and the prior probability distribution of the teaching logic type of the known knowledge point as the edge between the parent node and the child node, a directed graph of video education resource knowledge is obtained. Based on the directed graph of video education resources, and according to the prior probability distribution of the teaching logic type of known knowledge points in the knowledge point pair set, and the given observation of the preceding and following time order and the cosine similarity of the knowledge vector, the posterior probability of the preceding and following teaching relationship between each knowledge point pair is calculated as the weight of the corresponding directed edge by Bayes inference, thus constructing a knowledge graph of video education resources.

5. The video educational resource query method based on natural language processing according to claim 4, characterized in that, Step S2 specifically includes: Based on any two knowledge point pairs in the knowledge graph of video education resources, linear structured paths and non-linear structured paths are marked according to the maximum prior path. Based on the linear structured path in the knowledge graph of video educational resources, verify the expected path of any adjacent knowledge points in the linear structured path, and calculate the topological deviation contribution value of knowledge point pairs in the nonlinear structured path. Based on second-order difference, this paper analyzes the angle between the instantaneous velocity vector and instantaneous acceleration vector of knowledge point pairs in the topic global semantic vector and the cosine similarity of knowledge point pairs in the topic global semantic vector in the nonlinear structured path of video education resource knowledge graph, and calculates the topological nonlinear mutation degree index of knowledge point pairs in the nonlinear structured path. Based on the knowledge graph of video education resources, the first filter is the pre- and post-dependent relationship of knowledge point pairs. All sets of directly preceding knowledge points for knowledge point pairs are extracted. Random walks are performed on the association paths of the sets of directly preceding knowledge point pairs, and sets of irrelevant knowledge points on the non-connected paths of directly preceding knowledge points are marked. Combine the set of all knowledge point pairs that are directly preceding knowledge points with the set of knowledge points that are not related to the direct preceding knowledge points on non-connected paths into a knowledge point pair dependency gap set. Based on the set of knowledge point pair dependency gaps and the adjacent knowledge point pairs of knowledge point pair dependency gaps, calculate the nonlinearity of the teaching logic of knowledge point pair dependency gaps in the nonlinear structured path. The nonlinear feature vector of knowledge point pairs is obtained by additively concatenating and fusing the teaching logic nonlinearity of the knowledge point pair dependency gap in the nonlinear structured path, the topological nonlinear mutation degree index of the knowledge point pair in the nonlinear structured path, and the topological deviation contribution value of the knowledge point pair in the nonlinear structured path.

6. The video educational resource query method based on natural language processing according to claim 5, characterized in that, Step S2 also includes: Based on decision forest, a natural language teaching intention classifier is constructed. The knowledge graph of video education resources is used as the root node of the rule decision tree, the nonlinear feature vector of knowledge point pairs is used as the branch nodes of the rule decision tree, and the nonlinear topological structure type of knowledge point pairs is used as the leaf nodes. The normalized value of the ratio of the voting distribution of the nonlinear topological structure type of each leaf node of the decision forest to the total number of trees is calculated to determine the teaching rationality confidence between knowledge point pairs.

7. The video educational resource query method based on natural language processing according to claim 6, characterized in that, Step S3 specifically includes: Based on real-time query speech, a multimodal natural language processing network is used to obtain the knowledge point sequence of real-time query speech. The sequence is then fed into a natural language teaching intention classifier. According to the voting distribution of decision forest, the teaching rationality confidence of the knowledge point pairs of real-time query speech is determined, and a confidence sequence of real-time query speech knowledge point pairs is established. Based on the knowledge point sequence and the confidence sequence of knowledge point pairs in the real-time query speech, the Viterbi algorithm of weighted directed graph is used to perform maximum probability path search. The weights of all adjacent edges on the path are multiplied together to determine the maximum probability of the knowledge point pair path in the real-time query speech, thus obtaining the optimal matching video educational resource for the real-time query speech.

8. A video educational resource query system based on natural language processing is proposed to implement the video educational resource query method based on natural language processing as described in any one of claims 1-7, comprising: The system comprises a video education resource knowledge graph module, a knowledge nonlinear relationship module, and a knowledge intent recognition module. The knowledge nonlinear relationship module is electrically connected to the video education resource knowledge graph module, and the knowledge intent recognition module is electrically connected to the knowledge nonlinear relationship module. The education resource knowledge graph module is based on the video education resource database of the digital multimedia cloud platform. Following a multimodal natural language processing network, it extracts knowledge vectors from the video education resource data, analyzes the probability distribution of the pre- and post-relationships of the knowledge vectors, and constructs a video education resource knowledge graph. Knowledge Nonlinear Relationship Module: Based on the knowledge graph of video education resources, it analyzes the multidimensional nonlinear topological structure between adjacent knowledge points in the knowledge path sequence, quantifies the nonlinear feature vector of knowledge point pairs, trains a natural language teaching intention classifier, and obtains the teaching rationality confidence of knowledge point pairs. Knowledge Intent Recognition Module: Based on edge sensors, the module acquires the knowledge point sequence of real-time query speech and inputs it into a natural language teaching intent classifier. It then generates the teaching rationality confidence score of the real-time query speech knowledge point pair and performs a maximum probability path search on the video education resource knowledge graph to generate the optimal matching video education resource for the real-time query speech.

Images