Personalized learning path recommendation system based on multi-modal data analysis

Abstract

This invention relates to the field of learning path recommendation and discloses a personalized learning path recommendation system based on multimodal data analysis. The system includes a multimodal data acquisition module, a state credit assessment module, a path topology reconstruction module, and a modal transcoding and recommendation module. By collecting environmental signals and operational behavior signals to extract features, the system quantifies the instantaneous attenuation of each sensory channel to generate instantaneous cognitive load impedance data. It calculates the Euclidean distance between the knowledge node content source impedance data and the cognitive load impedance data, establishes impedance mismatch penalty term data, and uses it as a negative feedback factor to generate a reconstruction transition probability matrix. The system solves for the sequence with the maximum global path gain in the reconstruction matrix as the prefetched path and triggers multimodal material decoupling transcoding in real time when impedance mismatch exceeds the limit. This invention can dynamically adjust the recommended path according to the individual's cognitive state and achieve sensory load balancing compensation, effectively avoiding cognitive overload and improving learning efficiency.

Description

Technical Field

[0001] This invention relates to the field of learning path recommendation, specifically a personalized learning path recommendation system based on multimodal data analysis. Background Technology

[0002] Currently, with the widespread adoption of digital education and online learning platforms, personalized learning path recommendation systems have become a core tool for improving user learning efficiency. Different learners differ in their knowledge reserves, cognitive habits, and learning abilities. Traditional static teaching models struggle to meet the customized resource acquisition needs. Relying on big data and algorithmic models, the system can construct a knowledge graph based on the user's objective state and plan a suitable sequence of learning nodes for the current user. This precise content distribution mechanism effectively reduces the information filtering costs for users.

[0003] Regarding the aforementioned issues, existing personalized learning recommendation systems typically rely on users' static profiles and historical interaction characteristics. When operating, such systems first collect explicit metrics such as users' historical answer records, click-through rates, and total learning time. Then, they use traditional recommendation algorithms such as collaborative filtering or knowledge graph reasoning to calculate the matching degree between target knowledge nodes and users' ability tags. The recommendation engine generates graph transition probabilities based on the inherent logical dependencies within the subject and a pre-set difficulty gradient, thereby outputting a sequence of knowledge points. Finally, external terminals present text, images, or video streams to the target audience according to the delivered sequence.

[0004] Existing learning path recommendations still have limitations. Most recommendations rely on macro-level statistics of historical behavior, failing to delve into real-time physiological and micro-level operational fluctuations. This lack of underlying data prevents the system from quantifying the instantaneous attenuation of various sensory channels such as vision and hearing. In the path planning process, the calculation of graph node transition probabilities largely ignores the crucial element of content presentation. Due to the lack of comparison between the multimodal proportion of knowledge sources and the recipient's cognitive impedance, nodes with high-energy-consuming presentation formats are easily forcibly pushed. Furthermore, the single-node presentation mechanism is too rigid. Once a user experiences a local sensory load exceeding its limit while lingering at a specific knowledge point, the system is often helpless; the underlying multimodal materials cannot trigger real-time decoupling and transcoding rendering based on load limits, significantly compromising learning continuity.

[0005] Therefore, this invention provides a personalized learning path recommendation system based on multimodal data analysis to address the shortcomings of existing technologies. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a personalized learning path recommendation system based on multimodal data analysis. This system solves the problem that existing learning path recommendation systems cannot dynamically reconstruct recommended paths and adaptively adjust content presentation formats based on the learner's instantaneous cognitive load across multiple sensory channels, which can easily lead to learner cognitive overload and discontinuous learning.

[0007] To achieve the above objectives, the present invention provides the following technical solution: A personalized learning path recommendation system based on multimodal data analysis includes: A multimodal data acquisition module is used to acquire environmental signals and operational behavior signals, and extract a first feature vector and a second feature vector. The state credit assessment module is used to combine the first feature vector and the second feature vector to quantify the instantaneous attenuation of each sensory channel to generate credit data, and to perform norm normalization processing to generate instantaneous cognitive load impedance data. The path topology reconstruction module is used to extract the content source impedance data of the pre-set target knowledge node from the multimodal knowledge graph data, calculate the Euclidean distance between the content source impedance data and the instantaneous cognitive load impedance data and establish it as the impedance mismatch penalty term data, and use the impedance mismatch penalty term data as a negative feedback factor to generate reconstruction transition probability matrix data. The modal transcoding and recommendation module is used to solve for the knowledge node sequence with the largest sum of global path gain in the reconstructed transition probability matrix data as a sensory compensation prefetch path. When the impedance mismatch penalty term data is greater than the preset modal transcoding trigger threshold parameter used to characterize the sensory channel load limit, it generates a real-time content modal transcoding data stream and sends it to an external terminal device for image and sound display, thereby realizing personalized learning path recommendation.

[0008] By adopting the above technical solution, the multimodal data acquisition module acquires environmental and operational behavior signals to extract underlying physiological and behavioral features; the state credit assessment module quantifies sensory channel attenuation to generate instantaneous cognitive load impedance; the path topology reconstruction module calculates the matching difference between knowledge content and cognitive load and generates a reconstruction transition probability matrix by combining impedance mismatch penalty terms; and the modal transcoding and recommendation module solves for the optimal prefetch path and triggers modal adaptive transcoding when a threshold is reached. Therefore, the effect of dynamically adjusting the map transition topology based on the individual's instantaneous cognitive state and compensating for the distribution of sensory load in real time is achieved.

[0009] Preferably, the multimodal data acquisition module extracting the first feature vector specifically includes: The acquired environmental signal is used as the received radio frequency signal, and the radio frequency signal is decomposed into multiple corresponding orthogonal frequency division multiplexing subcarriers; Extract the in-phase component values ​​and quadrature component values ​​under the time and frequency conditions corresponding to the orthogonal frequency division multiplexing subcarrier from the original complex data stream, and further calculate the amplitude characteristic data and phase characteristic data of the orthogonal frequency division multiplexing subcarrier; Principal component analysis algorithm is used to perform dimensionality reduction operation on the amplitude feature data and the phase feature data, and the principal component data stream with the largest variance contribution rate is extracted to filter static environmental noise caused by stationary objects in the environment. A bandpass filtering algorithm is used to separate and extract respiratory rhythm variability data sequences with a frequency range of 10 to 25 times per minute from the principal component data stream, and the respiratory rhythm variability data sequences are established as the first feature vector.

[0010] By adopting the above technical solution, the respiratory rhythm variability data sequence is separated by decomposition and dimensionality reduction filtering of wireless radio frequency signals, thereby achieving high-precision monitoring and feature extraction of non-contact physiological state.

[0011] Preferably, the multimodal data acquisition module extracting the second feature vector specifically includes: The dynamic data sequence generated by the external interactive device that produces the operation behavior signal during the interaction process is continuously recorded, and the dynamic data sequence includes a continuous trajectory coordinate data sequence in the two-dimensional coordinate system of the screen. The continuous trajectory coordinate data sequence is divided and truncated according to a sampling time window of fixed length; Based on the coordinate difference between adjacent trajectory coordinate data points and the fixed sampling time interval parameter, the instantaneous curvature value of each local trajectory coordinate data point is calculated sequentially, and the average curvature parameter is obtained by arithmetically averaging all instantaneous curvature values ​​within the current sampling time window. By calculating the sum of curvature deviations within a unit sampling time window, the high-frequency curvature jitter parameter of the mouse trajectory, which represents the stability of micro-operations, is extracted and established as the second feature vector.

[0012] By adopting the above technical solution, the total curvature deviation within a unit time window is calculated based on the curvature of the trajectory coordinates, thereby achieving a precise quantification of the microscopic operational stability of external interactive devices.

[0013] Preferably, the state credit assessment module generates credit data by quantifying the instantaneous decay of each sensory channel, specifically including: The Kalman filter algorithm is used to perform time-series alignment and data fusion calculations on the first feature vector and the second feature vector to filter out high-frequency observation noise in the first feature vector and the second feature vector. The state residual numerical parameters after fusion calculation are continuously extracted. When the state residual numerical parameters exceed the set benchmark threshold used to characterize the fatigue state, the credit decay calculation mechanism for each affected sensory channel is triggered. Extract the second feature vector at the current moment, combine it with the visual channel credit data from the previous moment and the pre-set attenuation coefficient, and perform exponential attenuation algebra calculation to obtain the visual channel credit data. Using a similar exponential decay calculation logic, the auditory channel credit data and kinesthetic interaction channel credit data at the current moment are calculated in parallel. The visual channel credit data, the auditory channel credit data, and the kinesthetic interaction channel credit data constitute a global sensory credit matrix for the current moment.

[0014] By adopting the above technical solution, a state feedback mechanism is established by using a filtering algorithm to fuse multi-source features and combining it with exponential decay algebra calculation, thereby achieving the effect of dynamically tracking the fatigue status and capacity consumption status of each sensory channel.

[0015] Preferably, the state credit assessment module performs norm normalization processing to generate instantaneous cognitive load impedance data, specifically including: The global sensory credit score matrix is ​​normalized by dividing it by the arithmetic sum of the credit score values ​​of each channel within the matrix. Generate reference data characterizing the instantaneous cognitive load impedance vector, and output the calculated instantaneous cognitive load impedance vector as the instantaneous cognitive load impedance data; Each element in the instantaneous cognitive load impedance vector corresponds precisely to the percentage of remaining data reception capacity of the three sensory channels—visual, auditory, and kinesthetic interaction—at the current moment.

[0016] By adopting the above technical solution, the global sensory credit matrix is ​​normalized by dividing it by the arithmetic sum of the channels, thereby obtaining a standardized measure of the percentage of the instantaneous remaining capacity of each independent sensory channel.

[0017] Preferably, the path topology reconstruction module extracts the content source impedance data of the target knowledge node, specifically including: Read pre-defined target knowledge node data from a multimodal knowledge graph database; Extract the data volume parameters of all visual content materials contained in the underlying layer of the target knowledge node, and calculate the percentage of the visual data volume to the total multimodal data volume of the node, which is defined as the visual sensory channel occupancy ratio. Extract the data volume parameter of the auditory content audio material corresponding to the target knowledge node, and calculate the percentage value of the data volume parameter of the auditory content audio material to the total data volume, which is defined as the auditory sensory channel occupancy ratio; Extract the amount of action instruction data required for the interactive operation issued by the target knowledge node to the external interactive device, and calculate the proportion of the action instruction data to the total data of the node as the proportion of the kinematic interaction sensory channel occupancy. The proportion of visual sensory channel occupancy, the proportion of auditory sensory channel occupancy, and the proportion of kinesthetic interactive sensory channel occupancy are integrated into a three-dimensional feature vector containing three elements and defined as the content source impedance vector of the target knowledge node, which serves as the content source impedance data.

[0018] By adopting the above technical solution, the percentage of heterogeneous materials at the bottom layer of the calculation node in the total data volume is used to generate a three-dimensional feature vector, thereby achieving the effect of accurately reflecting the distribution pattern of sensory resources occupied by a specific knowledge node during presentation.

[0019] Preferably, the path topology reconstruction module performs the calculation of the Euclidean distance between the content source impedance data and the instantaneous cognitive load impedance data and establishes it as the impedance mismatch penalty term data, specifically including: The received instantaneous cognitive load impedance vector and the content source impedance vector of the target knowledge node are matched in terms of vector dimension and time sequence to ensure that the two vectors involved in the numerical calculation are in the same three-dimensional feature space. In the three-dimensional feature space, the spatial Euclidean distance between the instantaneous cognitive load impedance vector and the content source impedance vector of each target knowledge node is calculated based on the vector subtraction operation. The calculated spatial Euclidean distance value is directly defined as the impedance mismatch penalty term for the target knowledge node, and stored as the impedance mismatch penalty term data in the cache of the computing unit.

[0020] By adopting the above technical solution, through vector dimension alignment and solving the spatial Euclidean distance in the feature space based on subtraction operation, the effect of obtaining the degree of absolute matching difference between the quantitative knowledge representation and the object's current reception capability is achieved.

[0021] Preferably, the path topology reconstruction module performs the process of generating reconstruction transition probability matrix data by using the impedance mismatch penalty term data as a negative feedback factor, specifically including: Read the pre-stored node transition relationship probability data in the multimodal knowledge graph database as the original knowledge logic transition probability; The impedance mismatch penalty term data in the system cache is called, the obtained impedance mismatch penalty term value is multiplied by a preset smoothing attenuation factor, and the negative number of the product result is taken to calculate the natural exponent value of the negative number to form a negative feedback adjustment term used to characterize the sensory broadband matching degree. By combining matrix multiplication and addition, the product of the original knowledge logic transfer probability and the pre-set dynamic adjustment coefficient is combined with the product of the negative feedback adjustment term and the difference between the pre-set dynamic adjustment coefficient and the negative feedback adjustment term. The result is then used as the reconstruction transfer probability for the corresponding node path. Traverse all the sets of starting knowledge nodes and target knowledge nodes that have established connections, and generate a global two-dimensional reconstruction transition probability matrix containing all reconstruction transition probability parameters, as the reconstruction transition probability matrix data.

[0022] By adopting the above technical solution, the impedance mismatch penalty term is transformed into a natural exponential negative feedback term and integrated into the matrix merging and summing process of the original Markov transition probability, thereby achieving the effect of real-time voltage reduction of the selection probability of high-conflict mode resource nodes.

[0023] Preferably, the modal transcoding and recommendation module performs the task of finding the knowledge node sequence that maximizes the sum of global path gains in the reconstructed transition probability matrix data as the sensory compensation prefetching path, specifically including: Read the reconstruction transition probability matrix data stored in the multimodal knowledge graph database, and based on the starting knowledge node where the target object currently resides, start the depth-first search algorithm to generate multiple candidate learning trajectory sequences starting from the starting knowledge node; The generated candidate learning trajectory sequence is extracted according to the set planning step parameters, and the reconstruction transition probability parameters of two adjacent knowledge nodes in the candidate learning trajectory sequence at the current time are extracted in sequence. The global path gain value of the candidate learning trajectory sequence is calculated by arithmetically summing the reconstruction transition probability parameters of all adjacent node pairs in the sequence. The global path gain values ​​in the cache are sorted in descending order, and the candidate learning trajectory sequence that is first in the sort and has the largest global path gain value is extracted and established as the sensory compensation prefetch path.

[0024] By adopting the above technical solution, based on the reconstructed transition probability matrix and the planned step count, the traversal and summation of the multi-step transition probabilities are performed, thereby achieving the effect of smooth optimization and pre-scheduling of coherent learning tasks across knowledge nodes.

[0025] Preferably, the process of generating a real-time content modal transcoding data stream when the modal transcoding and recommendation module determines that the impedance mismatch penalty term data is greater than the modal transcoding trigger threshold parameter specifically includes: During the process of the target object residing at the target knowledge node according to the sensory compensation prefetching path, the numerical changes of the impedance mismatch penalty term data are monitored cyclically at fixed time intervals. The impedance mismatch penalty term value read in real time is compared with the modal transcoding trigger threshold parameter by an algebraic comparison operation. When it is determined that the impedance mismatch penalty term value is strictly greater than the modal transcoding trigger threshold parameter, a control interrupt signal is generated and sent to the modal transcoding execution unit. Extract multimodal decoupled material data from the underlying layer of the target knowledge node, reduce the rendering priority of conflicting sensory channel materials, increase the rendering priority of surplus sensory channel materials, decouple the identified high-conflict content material data, and render it in real time into audio or interactive data files in the corresponding format of other low-load sensory channels, generating a real-time content modal transcoding data stream.

[0026] By adopting the above technical solution, the impedance mismatch penalty term and the preset transcoding trigger threshold are compared in a loop, and the high-conflict multimodal material is decoupled and rendered and replaced in real time when the limit is exceeded, so as to obtain the effect of adaptive allocation of content resources and balanced display of sensory compensation during the display of a single knowledge node.

[0027] This invention provides a personalized learning path recommendation system based on multimodal data analysis. It has the following beneficial effects: 1. This invention acquires feature vectors from environmental and operational behavior signals using a multimodal data acquisition module, and quantifies the instantaneous attenuation of each sensory channel using a state credit assessment module, generating normalized instantaneous cognitive load impedance data. It can collect and fuse objective physiological and operational behavior data in real time, quantitatively calculate the remaining data reception capacity of the object's current visual, auditory, and kinesthetic interaction channels, objectively reflecting the individual's instantaneous cognitive load distribution, and providing basic data support for subsequent path planning.

[0028] 2. This invention extracts the content source impedance data of target knowledge nodes through a path topology reconstruction module, calculates the Euclidean distance between the target knowledge node and the instantaneous cognitive load impedance data to generate impedance mismatch penalty term data, and uses this as a negative feedback factor to generate reconstruction transition probability matrix data. A matching calculation mechanism is established between the multimodal material composition ratio of knowledge nodes and real-time sensory reception capability. By introducing a negative feedback adjustment term, the transition probability between graph nodes is dynamically adjusted, reducing the probability of selecting nodes that are prone to causing specific sensory overload, thus improving the fit between the recommended path and the current cognitive state.

[0029] 3. This invention uses modal transcoding and a recommendation module to solve for the sequence with the maximum gain in the reconstructed transition probability matrix as the prefetch path, and generates a real-time content modal transcoding data stream when the impedance mismatch penalty term data exceeds the modal transcoding trigger threshold. Based on cross-node path optimization, it achieves dynamic monitoring during single-node dwell time. When sensory resource conflicts occur, it can adjust the rendering priority of underlying multimodal materials in real time and perform format decoupling and replacement, realizing compensatory display between sensory channels and ensuring the continuity of the learning interaction process. Attached Figure Description

[0030] Figure 1 This is a framework diagram of the personalized learning path recommendation system based on multimodal data analysis of the present invention; Figure 2 This is a flowchart of the personalized learning path recommendation system based on multimodal data analysis according to the present invention. Figure 3 This is a time series monitoring graph of the impedance mismatch penalty term and the mode transcoding trigger threshold of the present invention; Figure 4 The experimental curves show the comparison of the knowledge acquisition probability effect before and after the system reconstruction of this invention. Detailed Implementation

[0031] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] See attached document Figure 1 This invention provides a personalized learning path recommendation system based on multimodal data analysis, which may include a multimodal data acquisition module, a state credit evaluation module, a path topology reconstruction module, and a modality transcoding and recommendation module. The output of the multimodal data acquisition module is connected to the input of the state credit evaluation module via an internal system data bus. A data transmission link is established between the output of the state credit evaluation module and the input of the path topology reconstruction module. The output of the path topology reconstruction module is connected to the input of the modality transcoding and recommendation module.

[0033] The multimodal data acquisition module includes a wireless network communication interface and a peripheral monitoring and control board. The wireless network communication interface is used to connect to commercial wireless fidelity transceivers installed in the target environment. The peripheral monitoring and control board connects to external interactive devices, including keyboards and mice, via a universal serial bus interface. The multimodal data acquisition module is used to continuously acquire environmental signals and operational behavior signals without contact with the human body.

[0034] The wireless network communication interface is configured with a fixed sampling frequency for continuously acquiring radio frequency (RF) channel status information. This RF channel status information originates from the RF signals propagating in space from the wireless fidelity transceiver. The multimode data acquisition module decomposes the received RF signal into multiple orthogonal frequency division multiplexing (OFDM) subcarriers and extracts the amplitude and phase characteristic data of each OFDM subcarrier.

[0035] The peripheral monitoring control board is used to synchronously record the dynamic data generated by external interactive devices. This dynamic data includes key dwell time and key travel time data for the keyboard device, and trajectory coordinate data for the mouse device. The multimodal data acquisition module segments the trajectory coordinate data according to a preset sampling time window of 50 to 100 milliseconds and extracts the trajectory curvature deviation data within each time window segment. The multimodal data acquisition module then sends the acquired RF channel state information and dynamic data to the state credit assessment module.

[0036] The state credit assessment module includes a data fusion processing unit and an index calculation engine. The data fusion processing unit receives radio frequency channel state information and dynamic data from the multimodal data acquisition module. The unit uses principal component analysis to perform dimensionality reduction calculations on the amplitude and phase characteristic data in the radio frequency channel state information, separating the first feature vector corresponding to respiratory rhythm variability. Simultaneously, it transforms the trajectory curvature deviation data in the dynamic data into a second feature vector representing high-frequency curvature jitter parameters.

[0037] The index calculation engine has three state registers corresponding to the visual, auditory, and kinesthetic interaction channels. Combining the first and second feature vectors, the engine quantifies the instantaneous attenuation of each sensory channel. The engine maps the increase in the high-frequency curvature jitter parameter to an exponential decrease in the value in the visual channel state register, thereby generating credit data for the visual, auditory, and kinesthetic interaction channels at the current moment.

[0038] The index calculation engine performs norm normalization on the generated visual channel credit data, auditory channel credit data, and kinesthetic interaction channel credit data. The result of this norm normalization is defined as instantaneous cognitive load impedance data. Instantaneous cognitive load impedance data reflects the remaining data reception capacity of each sensory channel at the current moment. The state credit assessment module transmits the instantaneous cognitive load impedance data and the credit data of each sensory channel to the path topology reconstruction module.

[0039] The path topology reconstruction module includes a graph storage unit and a matrix update unit. The graph storage unit stores multimodal knowledge graph data, which includes multiple knowledge node data and transition edge data connecting these knowledge nodes. Each target knowledge node is configured with fixed content source impedance data, which consists of the proportion of visual sensory channel usage, the proportion of auditory sensory channel usage, and the proportion of kinesthetic interactive sensory channel usage.

[0040] The matrix update unit receives instantaneous cognitive load impedance data from the state credit assessment module. The matrix update unit calculates the Euclidean distance between the content source impedance data of the target knowledge node and the received instantaneous cognitive load impedance data. This Euclidean distance is established as the impedance mismatch penalty for the target knowledge node at the current moment, which measures the difference between the target knowledge node's default display format and the current channel capacity.

[0041] The matrix update unit uses the impedance mismatch penalty term data as a negative feedback factor to reconstruct the underlying transition probability matrix of the multimodal knowledge graph data in real time. The matrix update unit extracts the original knowledge logic transition probabilities between multiple knowledge nodes and superimposes the impedance mismatch penalty term data into the original knowledge logic transition probabilities in the form of a negative exponential function to generate the reconstructed transition probability matrix data. The path topology reconstruction module outputs the reconstructed transition probability matrix data to the modal transcoding and recommendation module.

[0042] The modal transcoding and recommendation module includes a path optimization engine and a resource scheduling board. The path optimization engine receives reconstruction transition probability matrix data from the path topology reconstruction module. Based on a set number of steps, the engine performs a traversal and cumulative calculation within the reconstruction transition probability matrix data to find the knowledge node sequence with the largest global path gain. This sequence is stored as a cross-node sensory compensation prefetch path.

[0043] The resource scheduling board is equipped with an impedance mismatch monitoring process. During the continuous display period of a specific knowledge node, the resource scheduling board continuously receives the impedance mismatch penalty data of the target knowledge node at the current moment. The resource scheduling board compares the impedance mismatch penalty data with a pre-set modal transcoding trigger threshold used to characterize the load limit of the sensory channel. When it is determined that the impedance mismatch penalty data is greater than the modal transcoding trigger threshold, the resource scheduling board generates a transcoding control command.

[0044] The resource scheduling board extracts multimodal decoupled material data associated with the current knowledge node based on transcoding control instructions. The resource scheduling board reduces the rendering priority of material data corresponding to the visual sensory channel and increases the rendering priority of material data corresponding to the auditory sensory channel or the kinesthetic interactive sensory channel, generating a real-time content modal transcoding data stream. The modal transcoding and recommendation module sends the generated sensory compensation prefetch path and the real-time content modal transcoding data stream to external terminal devices for visual and audio display.

[0045] See attached document Figure 2 This invention provides a personalized learning path recommendation method based on multimodal data analysis, which can be divided into four main execution stages: acquiring multimodal data and extracting low-level features, tracking knowledge state and quantifying sensory credit, reconstructing the graph topology based on impedance matching, and generating compensating path prefetching and dynamic transcoding. This method is run in the background by a personalized learning path recommendation system based on multimodal data analysis to realize the transformation of data into a recommended topology.

[0046] Step S100 involves acquiring multimodal data and extracting low-level features. This step is specifically executed by the multimodal data acquisition module in the system. During operation, the system continuously acquires environmental communication radio frequency signals from the target environment and operational behavior signals of the target object towards external interactive devices through network interfaces and hardware connection ports. This step is based on a non-intrusive continuous data monitoring mechanism and is used to construct the underlying data source for subsequent analysis.

[0047] The system utilizes commercial wireless fidelity transceivers connected to the target environment to receive radio frequency (RF) signals propagating in space. The system converts the RF signals into RF channel state information and uses a computing unit to separate the amplitude and phase characteristic data of multiple orthogonal frequency division multiplexing (OFDM) subcarriers. The system performs principal component analysis (PCA) dimensionality reduction on the amplitude and phase characteristic data to extract the first feature vector corresponding to the respiratory rhythm variability of the target object.

[0048] The system synchronously monitors the dynamic data generated by the keyboard and mouse devices. The dynamic data consists of trajectory coordinate data, key dwell time data, and key travel time data. The system segments the trajectory coordinate data according to a fixed millisecond-level sampling time window, calculates the trajectory curvature deviation data within the time window, and then extracts the high-frequency curvature jitter parameter, which represents the stability of micro-operations, as the second feature vector.

[0049] Step S200 involves tracking the knowledge state and quantifying sensory credit. This step is specifically executed by the state credit assessment module in the system. After acquiring the first and second feature vectors, the system transforms the collected underlying feature vector data into specific numerical indicators that measure the cognitive state and the capacity of each specific sensory channel. This step executes the state tracking calculation for the knowledge dimension and the credit quantification calculation for the sensory dimension in parallel within the system's logic processing unit.

[0050] The system extracts historical interaction log data of the target object during its operation. This historical interaction log data is then input into a pre-defined deep knowledge tracing algorithm model for forward propagation calculations, outputting the target object's current knowledge mastery probability data for each target knowledge node in the multimodal knowledge graph data. This knowledge mastery probability data forms the initial reference basis for the system to calculate node transition probabilities.

[0051] The system utilizes the Kalman filter algorithm to perform temporal alignment and data fusion calculations on the first and second eigenvectors. Based on the numerical changes in the second eigenvector, the system calculates the credibility data for the visual channel, auditory channel, and kinesthetic interaction channel using a negative exponential decay function. The system then performs norm-normalized algebraic operations on the credibility data for these three sensory channels to generate instantaneous cognitive load impedance data reflecting the overall remaining capacity of the sensory channels at the current moment.

[0052] Step S300 involves reconstructing the graph topology based on impedance matching. This step is specifically executed by the path topology reconstruction module in the system. The system reads the pre-configured content source impedance data for each target knowledge node in the multimodal knowledge graph data from the storage unit. The system performs numerical matching and difference calculation between the instantaneous cognitive load impedance data calculated in step S200 and the content source impedance data of each candidate target knowledge node.

[0053] The system calculates the Euclidean distance between the content source impedance data of the candidate target knowledge node and the instantaneous cognitive load impedance data generated at the current moment. The system establishes the calculated Euclidean distance as the impedance mismatch penalty data for the target knowledge node at the current moment. The impedance mismatch penalty data directly reflects the degree of difference between the preset multimodal presentation resource allocation ratio of the target knowledge node and the target object's current actual sensory data reception capability.

[0054] The system acquires the original knowledge logic transition probabilities between knowledge nodes in the multimodal knowledge graph data. The system injects impedance mismatch penalty term data as a negative feedback factor in the form of an exponential function into the calculation process of the original knowledge logic transition probabilities, generating a reconstructed transition probability matrix through multiplication and addition. This reconstructed transition probability matrix numerically reduces the selection probability of target knowledge nodes containing high-conflict modal resources.

[0055] Step S400 involves generating compensatory path prefetching and dynamic transcoding. This step is specifically executed by the modal transcoding and recommendation modules within the system. The system receives the reconstructed transition probability matrix data output from step S300 and plans the execution knowledge sequence for multiple future time steps within the multimodal knowledge graph data. This step enables coherent learning task scheduling across multiple knowledge node levels and real-time content rendering weight adjustment during the dwell time of a single knowledge node.

[0056] The system obtains the planning step parameters of the configured candidate learning trajectories. Based on these planning step parameters, the system performs multi-step transition probability traversal and cumulative optimization calculations in the reconstructed transition probability matrix data to find the knowledge node combination sequence with the largest global path gain. The system establishes this knowledge node sequence with the largest global path gain as the sensory compensation prefetch path and outputs it to the external terminal device for execution scheduling.

[0057] During the continuous display period of a specific target knowledge node on an external terminal device, the system cyclically monitors the changes in the impedance mismatch penalty data at fixed time intervals. When the impedance mismatch penalty data is determined to be greater than a pre-set modal transcoding trigger threshold, the system immediately extracts the multimodal decoupling material data from the underlying layer of the target knowledge node, reduces the rendering priority of conflicting sensory channel materials, increases the rendering priority of surplus sensory channel materials, and generates and outputs a real-time content modal transcoding data stream.

[0058] The method of the present invention may include sub-steps S101 and S102. The steps of acquiring multimodal data and extracting low-level features are specifically executed by the multimodal data acquisition module in the system.

[0059] Sub-step S101 involves collecting channel state information to extract physiological features. The multimodal data acquisition module utilizes a commercial wireless fidelity transceiver deployed in the target learning environment as both the signal source and receiver. The wireless fidelity transceiver continuously transmits and receives radio frequency signals in the target environment at a fixed millisecond-level sampling frequency. These radio frequency signals are modulated by reflection and scattering from the target object's chest and abdomen as they propagate through space.

[0060] The underlying hardware layer of the multimodal data acquisition module monitors the communication channel of the aforementioned wireless fidelity transceiver. Without contacting the target object, the multimodal data acquisition module intercepts radio frequency (RF) channel state information carrying spatial fading characteristics in real time by parsing the protocol header information of the received network data packets. This RF channel state information exists in the form of a data matrix, recording the channel frequency response status on multiple transmit-receive antenna data links.

[0061] The system decomposes the captured radio frequency channel state information data matrix into multiple corresponding orthogonal frequency division multiplexing (OFDM) subcarriers. For each OFDM subcarrier, the system extracts the in-phase component and quadrature component values ​​under the corresponding time and frequency conditions from the original complex data stream. Using these two component values, the system further calculates the amplitude and phase characteristic data of the subcarrier.

[0062] The mathematical calculation model for extracting amplitude and phase feature data of orthogonal frequency division multiplexing subcarriers is as follows: ; in, In time frequency Channel frequency response at the location; In time frequency In-phase components at the location; In time frequency Orthogonal components at the location; For time; For frequency; It is the imaginary unit.

[0063] After acquiring the channel frequency response of all orthogonal frequency division multiplexing subcarriers, the multimodal data acquisition module generates the corresponding time series data set. The system uses principal component analysis (PCA) to perform dimensionality reduction on this time series data set, extracting the principal component data stream with the largest variance contribution rate to filter static environmental noise caused by stationary objects. The system then uses a bandpass filtering algorithm to separate and extract the respiratory rhythm variability data sequence with a frequency range of 10 to 25 breaths per minute from the principal component data stream; this sequence is established as the first feature vector.

[0064] Sub-step S102 involves acquiring peripheral dynamics and extracting behavioral features. The multimodal data acquisition module establishes a synchronous monitoring data link with the external interactive devices used by the target object through a universal serial bus data interface. These external interactive devices include a keyboard and a mouse. This monitoring operation acquires the hardware input event stream at the operating system level, ensuring the continuity and seamlessness of behavioral data acquisition.

[0065] The system continuously records the dynamic data sequence generated during the interaction between the keyboard and mouse devices. The keyboard's dynamic data includes key press and release time, as well as key travel time between two consecutive keystrokes. The mouse's dynamic data primarily consists of a continuous trajectory coordinate data sequence in the screen's two-dimensional coordinate system. The multimodal data acquisition module is equipped with a fixed-length sampling time window to divide and truncate the continuous trajectory coordinate data sequence.

[0066] For each segmented sampling time window, the system extracts all mouse trajectory coordinate data points contained within that window. Based on the coordinate differences between adjacent trajectory coordinate data points and a fixed sampling time interval parameter, the system sequentially calculates the instantaneous curvature value for each local trajectory coordinate data point. Subsequently, the system performs an arithmetic mean calculation on all instantaneous curvature values ​​within the current sampling time window to obtain an average curvature parameter reflecting the overall motion trend of that time window.

[0067] The multimodal data acquisition module extracts high-frequency curvature jitter parameters of the mouse trajectory, representing the stability of micro-operations, by calculating the sum of curvature deviations within a unit sampling time window. Its mathematical calculation model is as follows: ; in, In time High-frequency curvature jitter parameters of mouse trajectory; This represents the total number of trajectory points within the sampling time window; Index for trajectory points; In time No. The curvature of a trajectory point; In time Average curvature within the sampling time window; For time.

[0068] The system outputs the aforementioned high-frequency curvature jitter parameters of the mouse trajectory as a direct numerical indicator to measure the smoothness of operation of external interactive devices. The multimodal data acquisition module establishes the continuously generated sequence of high-frequency curvature jitter parameters of the mouse trajectory as the second feature vector. Finally, the multimodal data acquisition module performs time-stamp alignment and synchronous output of the first and second feature vectors, providing a non-intrusive basic feature data source for the subsequent system to quantify the data reception capacity attenuation of each sensory channel.

[0069] The method of the present invention may include sub-steps S201 and S202. The steps of tracking knowledge state and quantifying sensory credit are specifically executed by the state credit assessment module in the system.

[0070] Sub-step S201 is knowledge state analysis. The state credit assessment module continuously reads the target object's historical interaction log data within the current learning session from the external data storage unit. The historical interaction log data records all operational events of the target object on the external interactive device, including resource access timestamps, page dwell time parameters, correct and incorrect answers to in-class exercises, and video playback control command data.

[0071] The status credit assessment module performs data cleaning and structured format conversion on the acquired historical interaction log data. The system removes invalid null values ​​and redundant request records from the log data, and uniformly encodes various heterogeneous operation events into a standard time-series feature matrix. Each row in this time-series feature matrix represents a specific sequence of interaction events, and each column corresponds to the extracted numerical parameters of the interaction behavior features.

[0072] The system is equipped with a pre-trained deep knowledge tracing algorithm model, which is built on a long short-term memory network architecture. The state credit assessment module takes the aforementioned standard time-series feature matrix as input data and feeds it into the input layer of the deep knowledge tracing algorithm model. The model iteratively updates the state vector in the internal hidden units through a forward propagation algorithm, calculating the changes in the implicit cognitive state of the target object during the temporal interaction process.

[0073] The output layer of the deep knowledge tracing algorithm model is mapped to each target knowledge node defined in the multimodal knowledge graph data. Based on the currently updated implicit cognitive state vector, the model calculates a predicted value for the mastery level of each target knowledge node using a pre-defined activation function. This predicted value is a continuous real number between 0 and 1, representing the target object's current depth of understanding of a specific knowledge concept.

[0074] The state credit assessment module defines the predicted knowledge level output by the model as the knowledge mastery probability data for each node. The system aggregates the knowledge mastery probability data corresponding to all target knowledge nodes into a state vector and temporarily stores it in a memory buffer. This knowledge mastery probability data serves as the evaluation basis for the node's basic transition probability and directly participates in the underlying multidimensional matrix calculation process of the subsequent topology reconstruction stage.

[0075] Sub-step S202 involves attenuation calculation and credit matrix generation. The state credit assessment module receives the first and second feature vectors extracted from the pre-module. The first feature vector is a sequence of respiratory rhythm variability data, and the second feature vector is a sequence of high-frequency curvature jitter parameters of the mouse trajectory. The system uses a Kalman filter algorithm to perform time-series alignment and data fusion calculations on these two feature vector sequences with different sampling frequencies.

[0076] During the data fusion calculation process, the state credit assessment module filters out high-frequency observation noise from the first and second eigenvectors by setting state transition matrices and observation matrices. The system continuously extracts the numerical parameters of the state residuals after fusion calculation. When the numerical parameters of the state residuals exceed the set benchmark threshold used to characterize fatigue state, the system determines that the target object has entered a fatigue state and triggers the credit decay calculation mechanism for each affected sensory channel.

[0077] The state credit assessment module, taking the visual sensory channel as an example, extracts the high-frequency curvature jitter parameters of the mouse trajectory at the current moment. Combined with the visual channel credit data from the previous moment and a pre-set attenuation coefficient, it performs exponential attenuation algebraic calculations. The mathematical model for extracting visual channel credit data is as follows: ; in, In time Visual channel credibility; In time Visual channel credibility; This is the visual channel attenuation coefficient; In time High-frequency curvature jitter parameters of mouse trajectory; For time.

[0078] The system employs a similar exponential decay calculation logic to calculate the auditory channel credit score data and the kinesthetic interaction channel credit score data at the current moment in parallel. The state credit assessment module integrates and arranges all the calculated sensory channel credit score data into a matrix to construct a global sensory credit score matrix for the current moment. The mathematical calculation model for constructing the global sensory credit score matrix is ​​as follows: ; in, In time Sensory credit matrix; In time Visual channel credibility; In time The credibility of the auditory channel; In time The credibility of the kinesthetic interaction channel; For time.

[0079] To generate relative proportional benchmark data that can be used for subsequent impedance matching calculations, the state credit assessment module performs norm normalization on the aforementioned global sensory credit matrix. The system divides the global sensory credit matrix by the arithmetic sum of the credit values ​​of each channel within the matrix to generate benchmark data representing the instantaneous cognitive load impedance vector. The mathematical model for generating the instantaneous cognitive load impedance vector is as follows: ; in, In time The instantaneous cognitive load impedance vector; In time Sensory credit matrix; In time Visual channel credibility; In time The credibility of the auditory channel; In time The credibility of the kinesthetic interaction channel; For time.

[0080] The state credit assessment module outputs the calculated instantaneous cognitive load impedance vector. Each element in this vector corresponds precisely to the remaining data reception capacity percentage of the three sensory channels (visual, auditory, and kinesthetic) at the current moment. This vector data is transmitted to subsequent processing modules via the system's internal data bus, serving as a direct numerical control input variable for adjusting the knowledge graph's topology.

[0081] The method provided by this invention may include sub-steps S301, S302, and S303. The aforementioned steps for reconstructing the spectral topology based on impedance matching are specifically executed by the path topology reconstruction module in the system.

[0082] Sub-step S301 defines the node impedance vector. The path topology reconstruction module reads pre-defined target knowledge node data from the system's multimodal knowledge graph database. The multimodal knowledge graph database stores a large number of knowledge nodes and the hierarchical and network connections between them.

[0083] Each target knowledge node is annotated with content modality distribution parameters when it is entered into the multimodal knowledge graph database. These content modality distribution parameters are generated by statistically analyzing the type composition of the underlying materials of the node, and directly reflect the proportion of data resources used by the target knowledge node in the three specific sensory channels of vision, hearing, and motion when it is presented to the target object.

[0084] The system extracts the data volume parameters of all visual content materials contained in the underlying layer of the target knowledge node, and calculates the percentage of visual data volume in the total multimodal data volume of the node. The system defines this as the proportion of visual sensory channel occupancy. Simultaneously, the system extracts the data volume parameters of the auditory content audio materials corresponding to the target knowledge node, and calculates the percentage of auditory content audio material data volume in the total data volume. This is defined as the proportion of auditory sensory channel occupancy.

[0085] The system extracts the amount of action command data required for interactive operations issued by the target knowledge node to external interactive devices, and calculates the proportion of action command data to the total data volume of the node. The system defines this as the proportion of occupancy of the kinematic interaction sensory channels. The arithmetic sum of the proportions of these three sensory channels equals 1.

[0086] The path topology reconstruction module integrates the calculated proportions of visual, auditory, and kinesthetic sensory channels into a three-dimensional feature vector containing three elements. This three-dimensional feature vector is defined by the system as the content source impedance vector of the target knowledge node. This feature vector generation process is executed in the background for each candidate target knowledge node in the multimodal knowledge graph database.

[0087] The mathematical model for generating the content source impedance vector of the target knowledge node is as follows: ; in, For target knowledge nodes The content source impedance vector; For target knowledge nodes The proportion of visual sensory channels occupied; For target knowledge nodes The proportion of auditory sensory channels occupied; For target knowledge nodes The proportion of kinesthetic and interactive sensory channels occupied; For target knowledge nodes.

[0088] Sub-step S302 calculates the impedance mismatch penalty term. After defining the content source impedance vector of the candidate target knowledge node, the system obtains the instantaneous cognitive load impedance vector generated in the previous stage through the internal communication bus. This instantaneous cognitive load impedance vector is a three-dimensional row vector, representing the proportion of remaining available receiving capacity of each sensory channel of the target object at the current specific time node.

[0089] The path topology reconstruction module performs vector dimension matching and temporal alignment operations on the received instantaneous cognitive load impedance vector and the content source impedance vector of the candidate target knowledge node. The system ensures that the two vectors participating in subsequent numerical calculations are both in the same reference three-dimensional feature space, and that the three dimensions of each vector strictly correspond in order to the three independent channels of visual, auditory, and kinesthetic interaction.

[0090] In the aforementioned three-dimensional feature space, the system calculates the spatial Euclidean distance between the instantaneous cognitive load impedance vector and the content source impedance vector of each candidate target knowledge node based on vector subtraction. This spatial Euclidean distance constitutes a scalar data, objectively reflecting the absolute value of the difference between the multimodal resource distribution ratio required by the target knowledge node and the sensory reception capability that the target object can currently provide.

[0091] The system directly defines the calculated spatial Euclidean distance value as the impedance mismatch penalty term for the target knowledge node. When the presentation mode of a target knowledge node heavily relies on the visual channel, resulting in a very large visual component in its content source impedance vector, while the instantaneous cognitive load impedance vector component of the target object's visual channel at the current moment is extremely low, the impedance mismatch penalty term value corresponding to the target knowledge node will be significantly amplified.

[0092] The mathematical model for calculating the impedance mismatch penalty term is as follows: ; in, In time Target knowledge nodes Impedance mismatch penalty term; For target knowledge nodes The content source impedance vector; In time The instantaneous cognitive load impedance vector.

[0093] The system stores the calculated impedance mismatch penalty terms, which include all candidate target knowledge nodes at the current moment, in the cache of the computation unit. These impedance mismatch penalty terms are directly input as negative suppression factors into the subsequent underlying transition probability matrix reconstruction calculation logic.

[0094] Sub-step S303 involves reconstructing the transition probability matrix. The system reads the pre-stored node transition probability data from the multimodal knowledge graph database. This transition probability data is represented by the underlying Markov recommendation probability matrix, which is the original transition probability of moving from an initial knowledge node to a candidate target knowledge node according to the knowledge structure logic under ideal conditions without any external physiological or behavioral state data intervention.

[0095] The path topology reconstruction module calls the impedance mismatch penalty term data in the system cache to numerically correct and overwrite the original transition probabilities. The system pre-sets two constant hyperparameters in the configuration file: a dynamic adjustment coefficient and a smoothing attenuation factor. The dynamic adjustment coefficient is used to allocate a fixed ratio of weights between the original knowledge logic transition probabilities and the current sensory state feedback variables, while the smoothing attenuation factor is used to control the severity and numerical boundaries of the negative feedback effect of the impedance mismatch penalty term.

[0096] The system performs an arithmetic multiplication operation, multiplying the acquired impedance mismatch penalty term value by a preset smoothing attenuation factor, and taking the negative form of the product. The system then calculates the natural exponent of this negative number, whose output range is strictly limited to a continuous real number interval between 0 and 1. This natural exponent constitutes a negative feedback adjustment term used to characterize the sensory broadband matching degree. The larger the impedance mismatch penalty term value, the closer the generated negative feedback adjustment term value is to 0.

[0097] The system uses a combination of matrix multiplication and addition to merge and sum the product of the original transition probability and the dynamic adjustment coefficient with the product of the negative feedback adjustment term and the difference between the two dynamic adjustment coefficients. The final output of this merged sum is the reconstructed transition probability for the corresponding node path. This reconstructed transition probability replaces the original static transition probability in the running instance of the multimodal knowledge graph database.

[0098] The mathematical model for calculating and generating the reconstruction transition probability is as follows: ; in, In time Starting from the knowledge nodes To the target knowledge node The reconstruction transition probability; This is a dynamic adjustment coefficient; In time Starting from the knowledge nodes To the target knowledge node The original transition probability; This is a smoothing attenuation factor; In time Target knowledge nodes Impedance mismatch penalty term.

[0099] The path topology reconstruction module utilizes the aforementioned execution logic to traverse all established connection sets of starting knowledge nodes and candidate target knowledge nodes, repeatedly executing the numerical calculation and data update process for reconstruction transition probabilities. The system ultimately generates a global two-dimensional reconstruction transition probability matrix containing all reconstruction transition probability parameters. This matrix is ​​synchronously transmitted to subsequent execution units as the basis for the mathematical model parameter set for cross-node optimization and task scheduling.

[0100] The method of the present invention may include sub-steps S401 and S402. The steps of generating compensatory path prefetching and dynamic transcoding are specifically executed by the modal transcoding and recommendation module in the system.

[0101] Sub-step S401 involves global path gain calculation and recommendation. The system reads the reconstruction transition probability matrix data stored in the multimodal knowledge graph database. This matrix, generated by the pre-execution unit based on impedance matching calculation, contains reconstruction transition probability values ​​between all nodes. Based on the starting knowledge node currently residing in the target object, the system initiates a depth-first search algorithm to generate multiple candidate learning trajectory sequences originating from the starting knowledge node.

[0102] The system sets the internal planning step count parameter to a fixed integer, for example, a number between 3 and 5. Based on this parameter, the system extracts candidate learning trajectory sequences, ensuring that the length of all sequences involved in the calculation is strictly equal to the planning step count parameter. Each candidate learning trajectory sequence consists of a series of target knowledge nodes arranged in chronological order.

[0103] For each selected candidate learning trajectory sequence, the system performs a numerical calculation of the global path gain. The system sequentially extracts the reconstruction transition probability parameters of two adjacent knowledge nodes in the candidate learning trajectory sequence at the current time step. The system then arithmetically sums the reconstruction transition probability parameters of all adjacent node pairs in the sequence to calculate the global path gain value for that specific candidate learning trajectory sequence.

[0104] The mathematical model for calculating the global path gain of candidate learning trajectory sequences is as follows: ; in, In time Candidate learning trajectory sequence Global path gain; For length is Candidate learning trajectory sequences; The number of steps to plan for the candidate learning trajectory; In time Starting from the knowledge nodes To the target knowledge node The reconstruction transition probability; The first in the candidate learning trajectory sequence One knowledge node; The first in the candidate learning trajectory sequence Each knowledge node.

[0105] After calculating the global path gain for all candidate learning trajectory sequences, the system stores all results in an internal cache. The system then uses a quicksort algorithm to sort the global path gain values ​​in the cache in descending order. The system extracts the candidate learning trajectory sequence that ranks first and has the largest global path gain value, and transmits this sequence as the final recommendation to the front-end interface to guide the target object to the next target knowledge node.

[0106] Sub-step S402 involves node-level state monitoring and transcoding triggering. While the target object resides in the target knowledge node according to the recommended sequence, the system continuously executes state monitoring tasks. The system reads impedance mismatch penalty data in real time via the internal data bus at a sampling frequency of 10Hz. This impedance mismatch penalty data reflects the absolute spatial distance between the target object's current instantaneous cognitive load impedance vector and the target knowledge node's content source impedance vector.

[0107] The system's internal control register contains a pre-set modal transcoding trigger threshold parameter, which characterizes the load limit of the sensory channels. This modal transcoding trigger threshold parameter is a specific real number, pre-stored in the system's non-volatile memory. During each data sampling period, the system performs an algebraic comparison between the real-time impedance mismatch penalty term value and this modal transcoding trigger threshold parameter.

[0108] The mathematical model for determining whether the modal adaptive transcoding mechanism is triggered is as follows: ; in, In time Target knowledge nodes Impedance mismatch penalty term; This is the threshold for triggering modal transcoding.

[0109] When the inequality conditions in the judgment model are not met, the system determines that the current sensory channel load is within an acceptable range, maintains the original multimodal content rendering ratio of the current target knowledge node, and enters the state monitoring process of the next sampling cycle. When the inequality conditions in the judgment model are met, that is, when the impedance mismatch penalty term is strictly greater than the modal transcoding trigger threshold, the system generates a control interrupt signal and sends it to the modal transcoding execution unit.

[0110] Upon receiving the control interrupt signal, the modal transcoding execution unit immediately initiates the content modal adaptive transcoding process. The system identifies the specific content material data occupying the currently high-load sensory channels within the target knowledge node. The system calls its internal transcoding engine to decouple the identified high-conflict content material data and render it in real time into audio or interactive data files in the corresponding formats for other low-load sensory channels. The system then pushes the converted low-load modal data files to the output terminal device, replacing the original content material for playback and display.

[0111] Specific application examples: In a specific application embodiment, the system runs within the learning environment of a specific target object to generate data validation results. The multimodal data acquisition module performs non-intrusive continuous data collection operations. The current sampling time node is set. For 10 seconds, the multimodal data acquisition module extracts the dynamic data sequence of the external interactive device within the current sampling time window, and calculates and outputs the high-frequency curvature jitter parameter of the mouse trajectory, representing the stability of micro-operations. It is 0.4500.

[0112] The status credit assessment module receives parameter data output from the multimodal data acquisition module. The system reads the pre-configured visual channel attenuation coefficient from its internal memory. Its value is 0.5000. The system synchronously acquires the previous sampling time node. Visual channel credibility at 9 seconds The value is 0.8000. The status credit assessment module performs exponential decay algebra calculations to determine the current time point. Visual channel credit score at 10 seconds The natural exponent, calculated as 0.8000 multiplied by the product of -0.5000 and 0.4500, is 0.6387. The system uses a parallel processing architecture to synchronously calculate the auditory channel credit score. The credit score is 0.9000 and the kinesthetic interaction channel credit score. It is 0.7000.

[0113] The status credit assessment module reconstructs the credit scores from each of the above channels to generate a sensory credit matrix for the current sampling time point. The vector element combinations are 0.6387, 0.9000, and 0.7000. The system uses this sensory credit matrix... Perform a norm-normalized mathematical calculation, which involves dividing the credit score of each channel by the arithmetic sum of the credit scores of all channels, to generate an instantaneous cognitive load impedance vector. The vector element combinations are 0.2853, 0.4020 and 0.3127.

[0114] The path topology reconstruction module reads candidate target knowledge nodes. The pre-configured data. This is the alternative target knowledge node. Internally configured source impedance vector The element combinations are 0.7000, 0.2000, and 0.1000, corresponding to the resource occupancy proportions of the visual, auditory, and kinesthetic interaction channels, respectively. The path topology reconstruction module calculates candidate target knowledge nodes in the three-dimensional feature space. Content source impedance vector With instantaneous cognitive load impedance vector The Euclidean distance between them is used to output the target knowledge node. In time Impedance mismatch penalty term at 10 seconds Its specific value is 0.5078.

[0115] The path topology reconstruction module uses impedance mismatch penalty terms. Initiate the reconstruction calculation process for the underlying Markov transition probabilities. The system reads data from the initial knowledge node. To the alternative target knowledge node The original transition probability The value is 0.8000. The system extracts the dynamic adjustment coefficient from the internal register. The value is 0.6000 and the smoothing attenuation factor is... The value is 2.0000. The path topology reconstruction module performs a weighted hybrid calculation, which involves multiplying the dynamic adjustment coefficient 0.6000 by the original transition probability 0.8000, adding the difference between 0.6000 and 2.0000 by the natural exponent of the product of the impedance mismatch penalty term 0.5078, and finally obtaining the value in time. When the time is 10 seconds, start from the initial knowledge node To the target knowledge node Reconstruction transition probability The value is 0.6249. This reconstructed transition probability data is stored in the database instead of the original parameters.

[0116] See attached document Figure 3 , Figure 3 The horizontal axis represents time, measured in seconds, and the total duration of the monitored data is 100 seconds. Figure 3 The vertical axis represents numerical parameters, with data ranging from 0.20 to 0.55. Figure 3 The image shows two data sequence lines, one of which represents the modal transcoding trigger threshold. The thick horizontal dashed line represents the system's internally set threshold for triggering this mode of transcoding. The value remains constant at 0.4500 throughout the entire time series. Figure 3The other line is a solid broken line with circular data point markers, used to represent the time series change trajectory of the impedance mismatch penalty term continuously calculated in the output during system operation.

[0117] Combination Figure 3 The solid line trend marked with circular data points shows that the impedance mismatch penalty term monitored by the system exhibits a periodic alternation of peaks and troughs. Within several consecutive peak segments in the running time intervals of 8 to 12 seconds, 28 to 34 seconds, 54 to 61 seconds, and 78 to 84 seconds, the solid line representing the impedance mismatch penalty term crosses the thick horizontal dashed line representing the modal transcoding trigger threshold upwards on the vertical axis. (Time...) Taking the system's judgment at 10 seconds as an example, the impedance mismatch penalty term calculated by the aforementioned module at this time... The value is 0.5078, corresponding to Figure 3 The highest circular marker point is located near the first peak region on the vertical axis. The hardware comparator inside the modal transcoding and recommendation module receives this value and performs an inequality comparison. It determines that the impedance mismatch penalty term value of 0.5078 at the current moment is greater than the set modal transcoding trigger threshold value of 0.4500. At this over-limit node, the system immediately generates an internal control interrupt signal instruction, schedules the transcoding engine to perform real-time decoupling of the high-load visual content in the target knowledge node, and converts it into corresponding spatial audio data for replacement rendering output.

[0118] Experimental verification and effect comparison section: See attached document Figure 4 , Figure 4 The horizontal axis also represents time, measured in seconds, with the total monitoring time set to 100 seconds; the vertical axis represents the probability parameter of knowledge mastery, representing the learning effect, with the data range set between 0.30 and 0.90. In an independent system performance verification environment, Figure 4 Two comparative curves were plotted, showing the execution of different scheduling logics. One curve, marked with solid square data points, represents the time series data of knowledge mastery probability when the system's full-process scheduling mechanism, including reconstruction transition probability and modal adaptive transcoding, is enabled. The other curve, using a dashed line, represents the time series data of the original system mastery probability when all system adjustment mechanisms are disabled and only the original multimodal presentation logic is executed.

[0119] observe Figure 4At the start of the two comparative curves, with the test time at 0 seconds, both the solid line with square markings representing the current system and the dashed line representing the original system begin at a position around 0.4000 on the ordinate. As the time series progresses to the right, the dashed line representing the knowledge acquisition probability of the original system, lacking adaptive intervention to the sensory channel bottleneck, indicates that its internal visual senses are continuously operating under overload. This underlying state is reflected in the curve trend, where the upward slope of the dashed line gradually weakens in the first 40 seconds, and multiple local numerical drops and fluctuations occur in the subsequent running period. When the test time reaches the end of 100 seconds, the knowledge acquisition probability parameter of the dashed line converges and stagnates within the data range of 0.6500 on the ordinate, unable to generate further performance gains.

[0120] In comparison, Figure 4 The solid line marked with a square, representing the probability mastered by this system, shows a clear technical improvement indicator. During the operation of this system, due to... Figure 3 The impedance mismatch penalty term over-limit monitoring mechanism demonstrated was successfully triggered multiple times. When the system detected high load on a local channel, it executed accurate cross-node compensatory prefetching and real-time modal transcoding strategies, effectively clearing blockages in the underlying sensory data receiving path. The implementation of this objective technical method ensured that the solid line marked with a square maintained a smooth and continuous exponential upward slope throughout the entire 100-second test period, without significant data oscillations. When the test time reached 100 seconds, the knowledge acquisition probability parameter of this solid line stably converged within the data range of 0.8600 to 0.8800 on the ordinate. The above... Figure 3 and Figure 4 The presented comparative experimental data trends directly and objectively verify that the multimodal data quantization tracking, knowledge graph topology reconstruction, and dynamic content compensation mechanism adopted in this invention can improve reception efficiency from the system's underlying layer.

Claims

1. A personalized learning path recommendation system based on multimodal data analysis, characterized in that, include: A multimodal data acquisition module is used to acquire environmental signals and operational behavior signals, and extract a first feature vector and a second feature vector. The state credit assessment module is used to combine the first feature vector and the second feature vector to quantify the instantaneous attenuation of each sensory channel to generate credit data, and to perform norm normalization processing to generate instantaneous cognitive load impedance data. The path topology reconstruction module is used to extract the content source impedance data of the pre-set target knowledge node from the multimodal knowledge graph data, calculate the Euclidean distance between the content source impedance data and the instantaneous cognitive load impedance data and establish it as the impedance mismatch penalty term data, and use the impedance mismatch penalty term data as a negative feedback factor to generate reconstruction transition probability matrix data. The modal transcoding and recommendation module is used to solve for the knowledge node sequence with the largest sum of global path gain in the reconstructed transition probability matrix data as a sensory compensation prefetch path. When the impedance mismatch penalty term data is greater than the preset modal transcoding trigger threshold parameter used to characterize the sensory channel load limit, it generates a real-time content modal transcoding data stream and sends it to an external terminal device for image and sound display, thereby realizing personalized learning path recommendation.

2. The personalized learning path recommendation system based on multimodal data analysis according to claim 1, characterized in that, The multimodal data acquisition module performs the extraction of the first feature vector, specifically including: The acquired environmental signal is used as the received radio frequency signal, and the radio frequency signal is decomposed into multiple corresponding orthogonal frequency division multiplexing subcarriers; Extract the in-phase component values ​​and quadrature component values ​​under the time and frequency conditions corresponding to the orthogonal frequency division multiplexing subcarrier from the original complex data stream, and further calculate the amplitude characteristic data and phase characteristic data of the orthogonal frequency division multiplexing subcarrier; Principal component analysis algorithm is used to perform dimensionality reduction operation on the amplitude feature data and the phase feature data, and the principal component data stream with the largest variance contribution rate is extracted to filter static environmental noise caused by stationary objects in the environment. A bandpass filtering algorithm is used to separate and extract respiratory rhythm variability data sequences with a frequency range of 10 to 25 times per minute from the principal component data stream, and the respiratory rhythm variability data sequences are established as the first feature vector.

3. The personalized learning path recommendation system based on multimodal data analysis according to claim 1, characterized in that, The multimodal data acquisition module performs the extraction of the second feature vector, specifically including: The dynamic data sequence generated by the external interactive device that produces the operation behavior signal during the interaction process is continuously recorded, and the dynamic data sequence includes a continuous trajectory coordinate data sequence in the two-dimensional coordinate system of the screen. The continuous trajectory coordinate data sequence is divided and truncated according to a sampling time window of fixed length; Based on the coordinate difference between adjacent trajectory coordinate data points and the fixed sampling time interval parameter, the instantaneous curvature value of each local trajectory coordinate data point is calculated sequentially, and the average curvature parameter is obtained by arithmetically averaging all instantaneous curvature values ​​within the current sampling time window. By calculating the sum of curvature deviations within a unit sampling time window, the high-frequency curvature jitter parameter of the mouse trajectory, which represents the stability of micro-operations, is extracted and established as the second feature vector.

4. The personalized learning path recommendation system based on multimodal data analysis according to claim 1, characterized in that, The state credit assessment module generates credit score data by quantifying the instantaneous decay of each sensory channel, specifically including: The Kalman filter algorithm is used to perform time-series alignment and data fusion calculations on the first feature vector and the second feature vector to filter out high-frequency observation noise in the first feature vector and the second feature vector. The state residual numerical parameters after fusion calculation are continuously extracted. When the state residual numerical parameters exceed the set benchmark threshold used to characterize the fatigue state, the credit decay calculation mechanism for each affected sensory channel is triggered. Extract the second feature vector at the current moment, combine it with the visual channel credit data from the previous moment and the pre-set attenuation coefficient, and perform exponential attenuation algebra calculation to obtain the visual channel credit data. Using a similar exponential decay calculation logic, the auditory channel credit data and kinesthetic interaction channel credit data at the current moment are calculated in parallel. The visual channel credit data, the auditory channel credit data, and the kinesthetic interaction channel credit data constitute a global sensory credit matrix for the current moment.

5. The personalized learning path recommendation system based on multimodal data analysis according to claim 4, characterized in that, The state credit assessment module performs norm normalization processing to generate instantaneous cognitive load impedance data, specifically including: The global sensory credit score matrix is ​​normalized by dividing it by the arithmetic sum of the credit score values ​​of each channel within the matrix. Generate reference data characterizing the instantaneous cognitive load impedance vector, and output the calculated instantaneous cognitive load impedance vector as the instantaneous cognitive load impedance data; Each element in the instantaneous cognitive load impedance vector corresponds precisely to the percentage of remaining data reception capacity of the three sensory channels—visual, auditory, and kinesthetic interaction—at the current moment.

6. The personalized learning path recommendation system based on multimodal data analysis according to claim 1, characterized in that, The path topology reconstruction module extracts the content source impedance data of the target knowledge node, specifically including: Read pre-defined target knowledge node data from a multimodal knowledge graph database; Extract the data volume parameters of all visual content materials contained in the underlying layer of the target knowledge node, and calculate the percentage of the visual data volume to the total multimodal data volume of the node, which is defined as the visual sensory channel occupancy ratio. Extract the data volume parameter of the auditory content audio material corresponding to the target knowledge node, and calculate the percentage value of the data volume parameter of the auditory content audio material to the total data volume, which is defined as the auditory sensory channel occupancy ratio; Extract the amount of action instruction data required for the interactive operation issued by the target knowledge node to the external interactive device, and calculate the proportion of the action instruction data to the total data of the node as the proportion of the kinematic interaction sensory channel occupancy. The proportion of visual sensory channel occupancy, the proportion of auditory sensory channel occupancy, and the proportion of kinesthetic interactive sensory channel occupancy are integrated into a three-dimensional feature vector containing three elements, which is defined as the content source impedance vector of the target knowledge node, and serves as the content source impedance data.

7. The personalized learning path recommendation system based on multimodal data analysis according to claim 1, characterized in that, The path topology reconstruction module calculates the Euclidean distance between the content source impedance data and the instantaneous cognitive load impedance data and establishes it as the impedance mismatch penalty term data, specifically including: The received instantaneous cognitive load impedance vector and the content source impedance vector of the target knowledge node are matched in terms of vector dimension and time sequence to ensure that the two vectors involved in the numerical calculation are in the same three-dimensional feature space. In the three-dimensional feature space, the spatial Euclidean distance between the instantaneous cognitive load impedance vector and the content source impedance vector of each target knowledge node is calculated based on the vector subtraction operation. The calculated spatial Euclidean distance value is directly defined as the impedance mismatch penalty term for the target knowledge node, and stored as the impedance mismatch penalty term data in the cache of the computing unit.

8. The personalized learning path recommendation system based on multimodal data analysis according to claim 7, characterized in that, The path topology reconstruction module performs the following specific steps to generate reconstruction transition probability matrix data using the impedance mismatch penalty term data as a negative feedback factor: Read the pre-stored node transition relationship probability data in the multimodal knowledge graph database as the original knowledge logic transition probability; The impedance mismatch penalty term data in the system cache is called, the obtained impedance mismatch penalty term value is multiplied by a preset smoothing attenuation factor, and the negative number of the product result is taken to calculate the natural exponent value of the negative number to form a negative feedback adjustment term used to characterize the sensory broadband matching degree. By combining matrix multiplication and addition, the product of the original knowledge logic transfer probability and the pre-set dynamic adjustment coefficient is combined with the product of the negative feedback adjustment term and the difference between the pre-set dynamic adjustment coefficient and the negative feedback adjustment term. The result is then used as the reconstruction transfer probability for the corresponding node path. Traverse all the sets of starting knowledge nodes and target knowledge nodes that have established connections, and generate a global two-dimensional reconstruction transition probability matrix containing all reconstruction transition probability parameters, as the reconstruction transition probability matrix data.

9. The personalized learning path recommendation system based on multimodal data analysis according to claim 1, characterized in that, The modality transcoding and recommendation module performs the task of finding the knowledge node sequence that maximizes the sum of global path gains in the reconstructed transition probability matrix data as the sensory compensation prefetch path, specifically including: Read the reconstruction transition probability matrix data stored in the multimodal knowledge graph database, and based on the starting knowledge node where the target object currently resides, start the depth-first search algorithm to generate multiple candidate learning trajectory sequences starting from the starting knowledge node; The generated candidate learning trajectory sequence is extracted according to the set planning step parameters, and the reconstruction transition probability parameters of two adjacent knowledge nodes in the candidate learning trajectory sequence at the current time are extracted in sequence. The global path gain value of the candidate learning trajectory sequence is calculated by arithmetically summing the reconstruction transition probability parameters of all adjacent node pairs in the sequence. The global path gain values ​​in the cache are sorted in descending order, and the candidate learning trajectory sequence that is first in the sort and has the largest global path gain value is extracted and established as the sensory compensation prefetch path.

10. The personalized learning path recommendation system based on multimodal data analysis according to claim 1, characterized in that, The modal transcoding and recommendation module generates a real-time content modal transcoding data stream when the impedance mismatch penalty term data is greater than the modal transcoding trigger threshold parameter. Specifically, this includes: During the process of the target object residing at the target knowledge node according to the sensory compensation prefetching path, the numerical changes of the impedance mismatch penalty term data are monitored cyclically at fixed time intervals. The impedance mismatch penalty term value read in real time is compared with the modal transcoding trigger threshold parameter by an algebraic comparison operation. When it is determined that the impedance mismatch penalty term value is strictly greater than the modal transcoding trigger threshold parameter, a control interrupt signal is generated and sent to the modal transcoding execution unit. Extract multimodal decoupled material data from the underlying layer of the target knowledge node, reduce the rendering priority of conflicting sensory channel materials, increase the rendering priority of surplus sensory channel materials, decouple the identified high-conflict content material data, and render it in real time into audio or interactive data files in the corresponding format of other low-load sensory channels, generating a real-time content modal transcoding data stream.

Images