Confidential propaganda and education system based on multi-modal interaction

Abstract

The invention relates to the technical field of data processing, in particular to a confidential propaganda and education system based on multi-modal interaction, which comprises a data acquisition module, a data processing module, a training module and an optimization module. The data collection module collects user eye movement tracks, voice instructions, operation behaviors and physiological signal data, noise is eliminated through time synchronization and self-adaptive filtering, and then a three-dimensional data cube is constructed through a tensor decomposition algorithm. And reversely adjusting knowledge graph nodes and scene parameters by using a genetic algorithm to form a closed-loop feedback link. And the cross-module cooperation unit aggregates a global cognitive feature model through a federated learning framework, and realizes privacy protection and resource dynamic allocation in combination with differential privacy protection and an ant colony optimization algorithm. Through multi-modal data fusion, dynamic strategy generation and closed-loop evaluation optimization, the problem of unidirectional interaction and feedback lag of a traditional system is solved, and user confidential knowledge internalization efficiency and complex scene coping capacity are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and particularly to a confidential education and publicity system based on multimodal interaction. Background Art

[0002] The confidential education and publicity system is an educational tool that enhances users' information security awareness and confidentiality skills through information technology means, and its core functions are realized based on data fusion and interactive learning mechanisms. Traditional confidential education mostly adopts a one-way content transmission mode, relying on static texts, video resources and standardized tests, and it is difficult to dynamically capture users' learning status and environmental interaction characteristics. Modern systems integrate multimodal interaction technologies, synchronously collect multi-dimensional behavioral data such as users' visual attention, voice input, touch operations and physiological feedback, analyze cognitive characteristics and knowledge blind spots through machine learning algorithms, and dynamically generate adaptive teaching strategies. The system constructs highly realistic confidential scenarios through virtual simulation technology, simulates information leakage risk situations, guides users to complete decision-making training and behavior correction in interactive operations, and at the same time constructs a knowledge mastery evaluation model based on multi-source heterogeneous information fusion, optimizes the learning path through a real-time feedback mechanism, and finally improves the efficiency of internalizing confidential knowledge and the risk response ability in complex scenarios.

[0003] Existing confidential education and publicity systems mostly adopt a one-way information transmission mode, mainly relying on learning management systems to integrate multimedia content libraries and online examination functions, and realizing one-way indoctrination of confidential knowledge through preset texts, images and video resources. Existing confidential education and publicity systems usually rely on users to independently select courses and complete standardized tests, and their interaction forms are limited to static content display and answer feedback in a fixed process, lacking dynamic perception and real-time response to users' behaviors, cognitive states and environmental information. Due to the single dimension of human-computer interaction, the system is difficult to provide adaptive teaching strategies according to individual learning characteristics, resulting in insufficient user participation and limited knowledge internalization efficiency. In addition, the evaluation of learning effects in existing confidential education and publicity systems mostly relies on single indicators such as examination scores, and fails to integrate multimodal behavioral data to construct a comprehensive evaluation model, restricting the accuracy and personalization degree of teaching feedback. Summary of the Invention

[0004] Aiming at the deficiencies of the prior art, the present invention provides a confidential education and publicity system based on multimodal interaction, which is used to solve the problems of insufficient user participation, inaccurate teaching feedback and low knowledge internalization efficiency caused by one-way information transmission and single interaction dimension in existing confidential education and publicity systems.

[0005] To solve the above technical problems, the specific technical solutions of the present invention are as follows: The confidential education and publicity system based on multimodal interaction provided by the present invention includes: a data acquisition module, a data processing module, a training module and an optimization module; The data acquisition module is configured to collect multi-source heterogeneous data through a multimodal sensor unit, wherein the multi-source heterogeneous data includes the user's eye movement trajectory data, voice command semantic data, operation behavior coordinate data, and physiological signal data, perform time synchronization processing on the multi-source heterogeneous data to generate a time-aligned multimodal data stream, construct a three-dimensional data cube through a cross-modal feature fusion algorithm, and transmit the data to the data processing module; The data processing module is configured to extract user cognitive features based on the three-dimensional data cube, generate teaching strategy constraints by matching knowledge graph nodes through a multi-objective optimization algorithm, and dynamically generate a teaching instruction set in combination with a deep reinforcement learning algorithm and transmit it to the training module; The training module is configured to load the confidential scenario model according to the teaching instruction set, adjust the scenario interaction logic based on the dynamic perturbation parameters, receive user multimodal operation data and generate compliance path comparison results, trigger real-time feedback and write the interaction data into the distributed database and push it to the optimization module; The optimization module is configured to integrate user operation response data, physiological feedback data and test data to build a capability evaluation model, generate a strategy parameter instruction set through an inverse optimization algorithm, and reversely update the reinforcement learning parameters of the data processing module to form a closed-loop feedback link.

[0006] Furthermore, in the confidentiality education system based on multimodal interaction described in the present invention, the data acquisition module is also configured to: perform noise elimination processing on the multimodal data stream through an adaptive Kalman filtering algorithm to generate filtered multimodal data; input the filtered multimodal data into a tensor decomposition algorithm to perform spatiotemporal feature fusion, construct a three-dimensional data cube including time dimension, space dimension and feature dimension, and transmit the three-dimensional data cube to the data processing module.

[0007] Furthermore, in the multimodal interaction-based confidentiality education system of the present invention, the data processing module is further configured to: extract a user attention distribution heat map, knowledge understanding probability, and decision-making tendency vector from the three-dimensional data cube using a graph convolutional neural network to generate a cognitive feature vector; The cognitive feature vector is matched with the knowledge graph node through a multi-objective optimization algorithm to generate a Pareto optimal solution set including teaching difficulty, cognitive load and knowledge coverage, so as to constrain the teaching strategy exploration scope of the deep reinforcement learning algorithm.

[0008] Furthermore, in the multimodal interaction-based confidentiality education system of the present invention, the training module is further configured to: optimize the dynamic perturbation parameters of the physics engine based on a simulated annealing algorithm, and adjust the file permission change frequency in the file processing scenario and the network attack gradient in the network attack and defense scenario in real time; According to the optimized dynamic perturbation parameters, a preset compliance path is generated through a real-time dynamic path planning algorithm, and the user operation path is compared with the preset compliance path to generate compliance path comparison data; Based on the compliance path comparison data, a differential feedback animation is triggered, and the user operation path and comparison data are written into a distributed time-series database.

[0009] Furthermore, in the multi-modal interaction-based confidentiality education system of the present invention, the optimization module is further configured to: generate multi-dimensional evaluation indexes by integrating response delay data, operation compliance data, physiological stress level data, and test score data in the user operation path based on the analytic hierarchy process; input the multi-dimensional evaluation indexes into a Gaussian process regression model constructed by a Bayesian optimization algorithm to quantify the deviation between the user ability curve and the target ability curve; reverse-optimize the association strength of the knowledge graph nodes and the dynamic perturbation parameters through a genetic algorithm to generate a set of policy parameter instructions, and transmit the set of policy parameter instructions back to the data processing module to update the reward function weight of the preset deep deterministic policy gradient algorithm.

[0010] Furthermore, the multi-modal interaction-based confidentiality education system of the present invention further includes a cross-module data collaboration unit, which is configured to: aggregate the multi-modal data streams collected by the data collection module through a federated learning framework, and perform distributed collaborative training with the data processing module to generate a global cognitive feature model; Perform noise injection desensitization processing on the user interaction data in the multi-modal data stream by using a differential privacy protection algorithm; Dynamically allocate the microservice instance computing resources of the training module and the optimization module through an ant colony optimization algorithm to adapt to the real-time load changes of the system.

[0011] Furthermore, in the multi-modal interaction-based confidentiality education system of the present invention, the data processing module is further configured to: input the generated Pareto optimal solution set into a preset deep deterministic policy gradient algorithm, and dynamically adjust the virtual scene trigger threshold and the knowledge point reinforcement sequence based on the user real-time operation data fed back by the training module; Transmit the adjusted teaching instruction set to the training module through a message queue to drive the physical engine to update the file permission change frequency and the network attack gradient parameters.

[0012] Furthermore, in the multi-modal interaction-based confidentiality education system of the present invention, the training module is further configured to: load virtual simulation models of file processing scenarios and network attack and defense scenarios optimized based on a simulated annealing algorithm through a three-dimensional modeling engine; Drive the decision-making logic of scene NPCs based on a behavior tree model, and generate comparison data with a preset compliance path according to user operation data; Trigger a differential feedback animation that matches the comparison data through a real-time rendering engine and write the comparison data into a distributed time-series database.

[0013] Further, in the confidentiality education system based on multi-modal interaction according to the present invention, the optimization module is further configured to: generate multi-dimensional evaluation indicators by fusing response delay data, operation compliance data, physiological stress level data, and test score data in the user operation path generated by the analytic hierarchy process; Input the multi-dimensional evaluation indicators into a genetic algorithm to reversely optimize the node association strength of the knowledge graph and the scene perturbation parameters, and generate a set of policy parameter instructions; Transmit the set of policy parameter instructions back to the data processing module through a microservice interface to update the policy parameters of the preset deep deterministic policy gradient algorithm.

[0014] Further, in the confidentiality education system based on multi-modal interaction according to the present invention, the cross-module data collaboration unit is further configured to: aggregate multi-modal data streams of multiple user terminals through a federated learning framework to construct a global cognitive feature model, and synchronize the global cognitive feature model to the data processing module; Inject noise into the user behavior data in the global cognitive feature model by using a differential privacy protection algorithm to generate desensitized global model data; Dynamically adjust the microservice instance computing resource allocation strategy of the training module through an ant colony optimization algorithm to respond to changes in the real-time concurrent request volume of the system.

[0015] Advantages of the present invention; Through the multi-modal data fusion and dynamic strategy generation mechanism, the present invention significantly improves user engagement and knowledge internalization efficiency. The data acquisition module integrates visual tracking, speech recognition, and physiological signal detection units to capture users' multi-dimensional interaction behaviors in real time. Through adaptive filtering and tensor decomposition, a three-dimensional data cube with spatio-temporal correlation is constructed, solving the problem of missing dynamic perception caused by one-way data acquisition in traditional systems. The cognitive feature modeling module extracts users' attention distribution and decision-making tendency features based on graph convolutional neural networks, and combines the Pareto optimal solution set to constrain the generation of deep reinforcement learning strategies, dynamically adapting to users' cognitive load and knowledge coverage requirements, and realizing the transformation from a fixed teaching process to personalized interactive training. The virtual simulation module optimizes scene perturbation parameters and compliance path planning through the simulated annealing algorithm, and combines a differential feedback animation triggering mechanism to enhance users' operation perception and risk response capabilities. The optimization module uses the analytic hierarchy process to fuse multi-dimensional evaluation indicators, and updates strategy parameters through reverse iteration of the genetic algorithm to form a closed-loop feedback link, accurately improving the adaptability of teaching strategies. The cross-module collaboration unit introduces a federated learning framework and differential privacy protection technology to protect users' privacy while aggregating the global cognitive feature model, and combines the ant colony optimization algorithm to dynamically allocate computing resources, adapt to real-time load fluctuations, and balance system robustness and response efficiency. The above technical solutions work together to systematically solve the defects of single interaction, lagging feedback, and one-sided evaluation in traditional confidentiality education systems, and effectively strengthen users' confidentiality awareness and behavioral capabilities in complex scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the present invention, the accompanying drawings required for use in the embodiments will be briefly introduced below. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on the accompanying drawings.

[0017] Figure 1 FIG. is a system architecture diagram of a confidentiality education system based on multi-modal interaction provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the specific embodiments of the present invention and the corresponding accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention. The following will describe in detail the technical solutions provided by each embodiment of the present invention in conjunction with the accompanying drawings. To better understand the objectives of the present invention, the present invention will be further described in detail below.

[0019] Please refer to Figure 1, the confidentiality education system based on multimodal interaction provided by the present invention includes: a data acquisition module, a data processing module, a training module, and an optimization module; The data acquisition module is configured to collect multi-source heterogeneous data through a multimodal sensor unit. The multi-source heterogeneous data includes the user's eye movement trajectory data, voice command semantic data, operation behavior coordinate data, and physiological signal data. The multi-source heterogeneous data is subjected to time synchronization processing to generate a time-aligned multimodal data stream, and a three-dimensional data cube is constructed through a cross-modal feature fusion algorithm and transmitted to the data processing module; The data acquisition module collects the user's eye movement trajectory data, voice command semantic data, operation behavior coordinate data, and physiological signal data through a multimodal sensor unit. The multimodal sensor unit integrates a visual tracking device, a voice acquisition device, a touch interaction interface, and a biological signal sensor, and captures the attention distribution, semantic input, operation trajectory, and physiological response characteristics of the user in different interaction scenarios respectively. The visual tracking device captures the eye movement trajectory based on an infrared light source and an image sensor, and generates an eye movement data stream including fixation point coordinates, saccade paths, and dwell times. The voice acquisition device collects voice commands through a microphone array, and combines a voice recognition engine to parse the semantic content and generate structured text data. The touch interaction interface captures the spatial coordinates of the user's operation behavior through a capacitive or optical sensor, and records the time series and spatial distribution of the touch operation. The biological signal sensor uses a non-invasive electrode to collect electrocardiogram, skin conductance, or electromyogram signals, and quantifies the physiological stress level of the user in a confidentiality scenario. Each sensor generates a timestamp based on a unified clock source, and realizes the preliminary synchronization of multi-source data through a hardware trigger signal or a software timestamp alignment mechanism.

[0020] The multi-source heterogeneous data is subjected to time synchronization processing to generate a time-aligned multimodal data stream. The time synchronization processing adopts the Network Time Protocol or a synchronization mechanism based on a hardware clock to eliminate the time deviation caused by the sampling frequency difference or transmission delay between sensors. For data segments with a large timestamp deviation, an interpolation algorithm compensates for the missing data to generate a continuous time series. The synchronized multimodal data stream is segmented according to a time window, forming parallel data channels including eye movement trajectories, voice commands, operation behaviors, and physiological signals. The data segments within each time window are mapped to a unified time reference through calibration parameters, retaining the temporal correlation between the data and providing time-consistent input for subsequent feature fusion.

[0021] The cross-modal feature fusion algorithm performs spatio-temporal association modeling on the time-aligned multi-modal data stream to construct a three-dimensional data cube. The feature fusion algorithm first extracts the spatial distribution features of fixation points from the eye movement trajectory data and generates an attention-semantic association matrix in combination with the semantic vectors of voice commands. The operation behavior coordinate data is converted into a continuous trajectory path through a spatial interpolation algorithm and dynamically aligned with the time series of physiological signals. The tensor decomposition algorithm expands the multi-modal data in the time dimension, space dimension, and feature dimension, and extracts cross-modal latent feature vectors through high-order singular value decomposition. The time dimension characterizes the dynamic characteristics of user behavior as the teaching scenario evolves, the space dimension reflects the spatial distribution law of different sensor data, and the feature dimension integrates the coupling relationship of visual, semantic, operation, and physiological signals. The fused three-dimensional data cube is transmitted to the data processing module through a distributed message queue as the input data for cognitive feature modeling. The structural design of the three-dimensional data cube supports multi-dimensional feature extraction of the subsequent graph convolutional neural network, providing a spatio-temporal association data basis for the dynamic generation of teaching strategies.

[0022] The data processing module is configured to extract user cognitive features based on the three-dimensional data cube, generate teaching strategy constraint conditions by matching knowledge graph nodes through a multi-objective optimization algorithm, and dynamically generate a teaching instruction set in combination with a deep reinforcement learning algorithm and transmit it to the training module; The data processing module extracts user cognitive features based on the three-dimensional data cube. The three-dimensional data cube contains multi-modal fusion data in the time dimension, space dimension, and feature dimension, and models spatio-temporal association features through a graph convolutional neural network. The graph convolutional neural network constructs a spatio-temporal graph model based on the topological structure of the data cube, analyzes the transfer law of user attention focus in the time dimension, correlates the physical distribution of visual fixation points, touch operation coordinates, and physiological signal detection areas in the space dimension, and analyzes the voice semantic weight, operation compliance mark, and physiological stress level in the feature dimension. The network outputs a user attention distribution heat map to quantify the fixation duration and visual focus switching frequency; generates a knowledge understanding probability vector to reflect the user's mastery of confidential knowledge points; and extracts a decision tendency vector to characterize the user's operation preference in a simulated risk scenario. These feature vectors together constitute a digital representation of the user's cognitive state, providing input data for teaching strategy generation.

[0023] The multi-objective optimization algorithm matches the cognitive feature vector with the knowledge graph nodes to generate teaching strategy constraint conditions. The knowledge graph nodes are pre-set with the logical associations of confidential knowledge points, teaching difficulty labels, and cognitive load parameters, and the nodes are mapped to the vector space through a semantic embedding model. The multi-objective optimization algorithm takes maximizing knowledge coverage, minimizing cognitive load, and adapting to teaching difficulty as optimization objectives, and establishes an objective function under multiple constraint conditions. The algorithm performs similarity matching between the cognitive feature vector and the knowledge graph node vector to generate a Pareto optimal solution set containing multiple non-dominated solutions. Each solution corresponds to a set of teaching strategy parameter combinations, balancing the dynamic relationship between the intensity of knowledge point reinforcement and the user's learning pressure. The Pareto optimal solution set is input into the deep reinforcement learning algorithm as boundary conditions, limiting the exploration range of the policy network in the action space and avoiding generating teaching strategies beyond the user's cognitive ability.

[0024] The deep reinforcement learning algorithm dynamically generates a teaching instruction set in combination with real-time operation data. The algorithm is based on the deep deterministic policy gradient framework, takes the Pareto optimal solution set as the initial policy constraint, and takes the user's real-time operation data as the environmental state input. The real-time operation data includes path deviation, response delay time, and compliance operation ratio, and evaluates the suitability of the virtual scene trigger threshold and the knowledge point reinforcement sequence through the policy network. The policy network adjusts the action value function according to the real-time feedback, updates the sensitivity coefficient of the scene trigger threshold and the knowledge point priority weight, and generates a teaching instruction set that dynamically matches the user's cognitive load. The adjusted instruction set is transmitted to the training module through a message queue in the publish-subscribe mode, driving the parameter update of the virtual scene interaction logic. The message queue enables asynchronous communication between modules, and the instruction set contains the adjustment coefficient of the file permission change frequency, the update step size of the network attack gradient, and the index of the knowledge point reinforcement sequence, which are used by the physics engine to synchronously update the scene interaction rules after parsing.

[0025] The training module is configured to load a confidential scene model according to the teaching instruction set, adjust the scene interaction logic based on dynamic perturbation parameters, receive the user's multi-modal operation data and generate a compliance path comparison result, trigger real-time feedback, and write the interaction data into a distributed database and push it to the optimization module; The training module loads a confidential scene model according to the teaching instruction set. The teaching instruction set contains the adjustment coefficient of the file permission change frequency, the update step size of the network attack gradient, and the index of the knowledge point reinforcement sequence, and drives the three-dimensional modeling engine to load the corresponding virtual simulation models of the file processing scene and the network attack and defense scene through the parsing module. The file processing scene constructs a multi-level permission management system to simulate the compliance process of file access, editing, and sharing operations; the network attack and defense scene is pre-set with vulnerability exploitation paths and a defense strategy library to dynamically generate network attack behavior sequences. During the loading process of the scene model, the physics engine initializes the dynamic perturbation rules according to the instruction set parameters, establishes the permission change trigger condition and the attack gradient calculation logic, and forms an interaction environment adapted to the current teaching strategy.

[0026] The dynamic perturbation parameter adjusts the scene interaction logic. The simulated annealing algorithm optimizes the physical engine parameters based on the teaching instruction set, and iteratively calculates the optimal combination of the file permission change frequency threshold and the network attack gradient step size. A random perturbation factor is introduced in the parameter adjustment process to balance the stability and challenge of scene interaction. The optimized dynamic perturbation parameters update the permission change trigger condition in real time, and control the frequency of permission upgrade requests in the file operation process; synchronously adjust the vulnerability exploitation frequency and attack path complexity of the network attack scenario, and simulate the dynamic threat evolution in the real risk environment. After the parameters are updated, the scene interaction rules and the user operation feedback mechanism are synchronously refreshed to form a teaching environment that dynamically adapts to the user's cognitive ability.

[0027] The user's multi-modal operation data is input into the training module through the touch interaction interface, the voice acquisition unit, and the eye movement tracking device. The touch operation coordinate data is mapped to the virtual scene space coordinate system to generate the spatio-temporal trajectory of the user operation path; the voice command is converted into an operation command by the semantic parsing engine to drive the state change of the scene object; the eye movement trajectory data extracts the visual focus distribution through the fixation point clustering algorithm to identify the area where the user's attention is concentrated. The multi-modal operation data is input into the behavior tree model after timestamp alignment to drive the decision-making logic of the scene NPC. The behavior tree pre-sets compliance operation judgment nodes and risk response rules, and triggers decision branches such as permission boundary interception, attack path blocking, or compliance operation guidance according to the spatio-temporal coordinates and operation types of the user operation path, generating operation compliance marks and path deviation degree indicators.

[0028] The compliance path comparison result is generated based on the real-time dynamic path planning algorithm. The algorithm constructs a permission operation reachability map and a network defense path topology in the virtual scene according to the optimized dynamic perturbation parameters. The rapidly-exploring random tree algorithm searches for the optimal compliance path under the current parameter constraints, and generates a preset path sequence including key operation nodes and time windows. The user operation path is compared with the preset path in terms of spatio-temporal trajectory, calculating the path deviation distance, operation delay time, and key node matching degree, generating multi-dimensional compliance quantification data. The comparison result reflects the matching degree between the user's operation behavior and the teaching goal, providing a quantitative basis for differential feedback.

[0029] The real-time feedback mechanism triggers differential animations according to the compliance path comparison data. The real-time rendering engine calls the preset animation resource library and matches the feedback type based on the path deviation degree level. The permission boundary crossing operation triggers a dynamic warning border and a high-frequency warning sound effect, and the compliance path matching operation generates a progressive guiding light effect and a positive incentive prompt. The animation rendering process integrates the real-time parameters of the physical engine, adjusts the light and shadow intensity and the response speed of the interface elements, enhancing the immersion of the user operation perception. The triggering logic of the feedback animation is embedded in the scene interaction loop to achieve millisecond-level synchronization between the user operation and the visual feedback, improving the real-time and interactivity of the teaching process.

[0030] Interaction data is written into the distributed time-series database and pushed to the optimization module. After the spatio-temporal coordinates, compliance marks, and feedback log data of the user operation path are marked with timestamps, data sharding is performed according to the scenario type and user ID. The distributed time-series database adopts a columnar storage structure, divides data partitions according to time windows, and supports high-concurrency writing and fast time-range queries. After the data writing is completed, the message middleware triggers an asynchronous push event, and transmits the interaction data packet in a standardized format to the message queue of the optimization module through the publish-subscribe mode. The data packet contains the original operation trajectory, compliance comparison metrics, and feedback trigger records, providing structured input for subsequent multi-dimensional evaluation and policy iteration. The data transmission process uses a lightweight serialization protocol to reduce network bandwidth occupancy and ensure the real-time requirements of the system.

[0031] The optimization module is configured to fuse user operation response data, physiological feedback data, and test data to build an ability evaluation model, generate a policy parameter instruction set through a reverse optimization algorithm, and update the reinforcement learning parameters of the data processing module in reverse, forming a closed-loop feedback link.

[0032] The policy parameter instruction set is transmitted back to the data processing module through the microservice interface. The interface encapsulates the instruction data using the RESTful protocol and defines a standardized parameter update format. The data processing module analyzes the instruction set, updates the reward function weight parameters of the deep deterministic policy gradient algorithm, and enhances the generation probability of teaching strategies that conform to the user's cognitive characteristics. The updated reward function drives the policy network to re-evaluate the virtual scenario trigger threshold and the priority of knowledge point reinforcement sequences, forming a dynamic adaptation mechanism for teaching strategies. Through continuous parameter iteration optimization, the closed-loop feedback link improves the tracking accuracy of the confidentiality education system for the evolution of user capabilities and the efficiency of policy adjustment.

[0033] The confidentiality education system based on multi-modal interaction provided by the present invention realizes a closed-loop teaching process through the collaborative work of multiple modules. The data acquisition module first synchronously acquires the user's eye movement trajectory data, voice command semantic data, operation behavior coordinate data, and physiological signal data through the multi-modal sensor unit. These multi-source heterogeneous data are aligned in time series through timestamp synchronization technology to eliminate the timing deviation between sensors and generate a time-aligned multi-modal data stream. Subsequently, the cross-modal feature fusion algorithm performs tensor decomposition on the time-aligned data stream, fuses the features of different modalities in the time, space, and feature dimensions, and constructs a three-dimensional data cube to represent the multi-dimensional interaction behavior characteristics of the user. The three-dimensional data cube is transmitted to the data processing module as the basic input for subsequent cognitive feature modeling.

[0034] Based on the three-dimensional data cube, the data processing module uses a graph convolutional neural network to extract the attention distribution heat map, knowledge understanding probability, and decision-making tendency vector of the user in the spatio-temporal dimension. The spatio-temporal feature vector is matched with the nodes of the knowledge graph through a multi-objective optimization algorithm to generate a Pareto optimal solution set including teaching difficulty, cognitive load, and knowledge coverage rate. The Pareto optimal solution set is input into a preset deep deterministic policy gradient algorithm as a constraint condition to dynamically adjust the virtual scene trigger threshold and knowledge point reinforcement sequence, generating a teaching instruction set. The teaching instruction set is transmitted to the training module through a message queue to drive the update of the subsequent scene interaction logic.

[0035] The training module loads the file processing scenario and network attack and defense scenario models in the security scenario library according to the teaching instruction set. The dynamic perturbation parameters of the physical engine are optimized through a simulated annealing algorithm to adjust the file permission change frequency and network attack gradient in real time. The operation data triggered by the user through the multi-modal interface is driven by the behavior tree model to generate the decision-making logic of the scene NPC, generating the comparison data between the user operation path and the preset compliance path. The real-time dynamic path planning algorithm generates a compliance path based on the optimized dynamic perturbation parameters and triggers a differential feedback animation matching the comparison data through the real-time rendering engine. The user operation path data and comparison data are written into the distributed time-series database and pushed to the optimization module for multi-dimensional evaluation.

[0036] The optimization module generates multi-dimensional evaluation indicators by integrating the response delay data, operation compliance data, physiological stress level data, and test score data in the user operation path based on the analytic hierarchy process. The evaluation indicators are input into the Gaussian process regression model constructed by the Bayesian optimization algorithm to quantify the deviation between the user ability curve and the target ability curve. The genetic algorithm reversely optimizes the node association strength of the knowledge graph and the scene perturbation parameters according to the deviation to generate a policy parameter instruction set. The policy parameter instruction set is transmitted back to the data processing module through the microservice interface to update the reward function weight of the deep deterministic policy gradient algorithm, forming a closed-loop optimization link for the teaching strategy.

[0037] The cross-module data collaboration unit aggregates multi-modal data streams through the federated learning framework and collaboratively trains the global cognitive feature model on the premise of protecting user privacy. The differential privacy protection algorithm injects noise into the interaction data for desensitization to ensure data security. The ant colony optimization algorithm dynamically allocates the computing resources of the microservice instances of the training module and the optimization module to respond to the real-time load changes of the system. The global cognitive feature model is synchronized to the data processing module to enhance the generalization ability of the personalized teaching strategy. The data flow and parameter update between modules form a self-iterative teaching system to achieve the dynamic adaptation of the user cognitive state and teaching strategy.

[0038] Specifically, in the confidentiality education system based on multimodal interaction according to the present invention, the data acquisition module is further configured to: perform noise cancellation processing on the multimodal data stream through an adaptive Kalman filtering algorithm to generate filtered multimodal data; input the filtered multimodal data into a tensor decomposition algorithm for spatio-temporal feature fusion, construct a three-dimensional data cube including a time dimension, a space dimension, and a feature dimension, and transmit the three-dimensional data cube to the data processing module.

[0039] The data acquisition module synchronously acquires the eye movement trajectory data, voice command semantic data, operation behavior coordinate data, and physiological signal data of the user through the multimodal sensor unit. These multi-source heterogeneous data include temporal information from different sensors and may have sensor noise and time deviation. The adaptive Kalman filtering algorithm is used to perform noise cancellation processing on the multimodal data stream, dynamically adjust the filtering parameters according to the noise statistical characteristics of each sensor, suppress signal drift and random interference, and generate filtered multimodal data. The filtered data retains the time synchronization and signal integrity of the user behavior characteristics, providing a preprocessing basis for subsequent feature fusion.

[0040] The filtered multimodal data is input into a tensor decomposition algorithm for cross-modal spatio-temporal feature fusion. The algorithm first slices the time dimension to align the time windows of different modal data and eliminate temporal misalignment. In the space dimension, a spatial correlation matrix is established according to the physical distribution of sensors or the spatial coordinates of user operations to extract the spatial consistency features of multimodal data. In the feature dimension, the visual attention features of eye movement trajectories, the semantic features of voice commands, the operation mode features of touch behaviors, and the stress response features of physiological signals are jointly extracted through high-order singular value decomposition. The tensor decomposition algorithm maps the multimodal data to a unified three-dimensional data space, constructs a three-dimensional data cube including time, space, and feature dimensions, and represents the spatio-temporal correlation and cross-modal feature coupling relationship of user behaviors.

[0041] The three-dimensional data cube is transmitted to the data processing module as the input for cognitive feature modeling. The time dimension of the data cube records the dynamic changes of user behaviors, the space dimension reflects the synergistic effect of multi-sensor data, and the feature dimension integrates cross-modal semantic and physiological information. Through structured data representation, it provides multi-granularity and multi-dimensional data support for the spatio-temporal feature extraction of the subsequent graph convolutional neural network, forming a complete preprocessing link from raw signal noise reduction to high-order feature fusion.

[0042] Specifically, in the confidentiality education system based on multimodal interaction according to the present invention, the data processing module is further configured to: extract the user attention distribution heat map, knowledge understanding probability, and decision-making tendency vector from the three-dimensional data cube by using a graph convolutional neural network to generate a cognitive feature vector; The cognitive feature vector is matched with the knowledge graph node through a multi-objective optimization algorithm to generate a Pareto optimal solution set including teaching difficulty, cognitive load and knowledge coverage, so as to constrain the teaching strategy exploration scope of the deep reinforcement learning algorithm.

[0043] The data processing module receives a three-dimensional data cube from the data acquisition module, and the data cube includes multimodal fusion information in time, space and feature dimensions. The graph convolutional neural network constructs a spatiotemporal graph model based on the topological structure of the data cube, captures the dynamic changes in the user's attention distribution in the time dimension, associates the physical positions of different sensors with the user's operation trajectory in the spatial dimension, and analyzes the visual focus of the eye movement trajectory, the semantic weight of the voice command, the compliance characteristics of the touch operation and the stress level of the physiological signal in the feature dimension. Through multi-layer graph convolution operations, the user's attention distribution heat map is extracted to quantify the frequency of visual focus shifts and the length of residence; the knowledge comprehension probability is generated to characterize the user's mastery of confidential knowledge points; and the decision-making tendency vector is output to reflect the user's risk response preference in the simulated scenario. The heat map, probability and vector together constitute a cognitive feature vector, which serves as a digital representation of the user's cognitive state.

[0044] The cognitive feature vector is input into the multi-objective optimization algorithm and dynamically matched with the knowledge graph node. The knowledge graph node is preset with confidential knowledge points, teaching difficulty labels and cognitive load parameters, and the cognitive feature vector is mapped to the graph vector space through the semantic embedding model. The multi-objective optimization algorithm solves the Pareto optimal solution set with the constraints of maximizing knowledge coverage, minimizing cognitive load and adapting teaching difficulty. The solution set includes multiple non-dominated solutions, each solution corresponds to a set of teaching strategy parameters, which are used to balance the dynamic relationship between knowledge reinforcement intensity and user learning pressure. The Pareto optimal solution set is input into the deep reinforcement learning algorithm as a boundary condition to constrain the exploration range of the strategy network in the action space to avoid excessive deviation from the user's actual cognitive ability. The deep reinforcement learning algorithm adjusts the virtual scene trigger threshold based on real-time interactive feedback, dynamically assigns the priority of the knowledge point reinforcement sequence, and generates a teaching instruction set adapted to the user's cognitive characteristics.

[0045] Specifically, in the multimodal interaction-based confidentiality education system of the present invention, the training module is further configured to: optimize the dynamic perturbation parameters of the physics engine based on a simulated annealing algorithm, and adjust the file permission change frequency in the file processing scenario and the network attack gradient in the network attack and defense scenario in real time; Based on the optimized dynamic disturbance parameters, a preset compliance path is generated through a real-time dynamic path planning algorithm, and the user operation path is compared with the preset compliance path to generate compliance path comparison data; Trigger a differential feedback animation based on the compliance path comparison data, and write the user operation path and comparison data into the distributed time series database.

[0046] The training module receives the teaching instruction set from the data processing module and optimizes the dynamic perturbation parameters of the physical engine based on the simulated annealing algorithm. The simulated annealing algorithm initializes the initial parameter combination of the file permission change frequency and the network attack gradient, introduces random perturbations through the iterative optimization process, and calculates the stability index of the scenario interaction logic under different parameter combinations. The algorithm evaluates the adaptability of the parameter combination according to the energy function, jumps out of the local optimum in a probabilistic way of accepting inferior solutions, and gradually converges to the global optimum parameter set. The optimized dynamic perturbation parameters adjust the triggering frequency of permission changes in the file processing scenario in real time, and control the gradient change intensity of attack behaviors in the network attack and defense scenario, forming scenario interaction rules that dynamically adapt to the user's cognitive ability.

[0047] According to the optimized dynamic perturbation parameters, the real-time dynamic path planning algorithm generates a preset compliance path in the virtual scene. The algorithm constructs a spatial reachability map of compliance operations based on the scenario physical constraints and the confidentiality rule library, combined with the permission change frequency and the attack gradient parameters. The optimal path is searched through the rapidly-exploring random tree algorithm, and a preset compliance path matching the current teaching strategy is generated. The operation data triggered by the user through the multimodal interface is captured in real time, compared with the preset path in terms of space-time trajectory, and the path deviation degree, operation delay time and key node compliance index are calculated to generate multi-dimensional compliance path comparison data. The comparison data reflects the matching degree between the user's operation behavior and the teaching goal, providing a quantitative basis for differential feedback.

[0048] The compliance path comparison data is input into the real-time rendering engine to trigger a differential feedback animation adapted to the user's operation behavior. The feedback system matches the preset animation template according to the path deviation degree level, triggers a red warning animation for permission overstep behaviors, and generates a green guiding animation for the compliance operation path. During the animation rendering process, the real-time parameters of the physical engine are integrated to dynamically adjust the light and shadow effects and the response speed of the interface elements, enhancing the immersion and feedback immediacy of the user's operation. The space-time coordinate data, comparison data and feedback log of the user operation path are marked with timestamps and written into the distributed time series database, and data sharding storage is performed according to the scenario type and time window, providing structured input data for the multi-dimensional evaluation of the optimization module.

[0049] Specifically, for the confidentiality education system based on multimodal interaction described in the present invention, the optimization module is further configured to: fuse the response delay data, operation compliance data, physiological stress level data, and test score data in the user operation path based on the analytic hierarchy process to generate multi-dimensional evaluation indicators; input the multi-dimensional evaluation indicators into the Gaussian process regression model constructed by the Bayesian optimization algorithm to quantify the deviation between the user ability curve and the target ability curve; reversely optimize the association strength of the knowledge graph nodes and the dynamic perturbation parameters through the genetic algorithm to generate a set of policy parameter instructions, and reversely transmit the set of policy parameter instructions to the data processing module to update the reward function weight of the preset deep deterministic policy gradient algorithm.

[0050] The optimization module receives the user operation path data from the training module, including response delay time, operation compliance mark, physiological stress level index, and test score data. The analytic hierarchy process constructs a multi-dimensional evaluation index system, and through expert weight assignment and consistency test, the data is standardized and weighted and fused. The response delay data is mapped to the user reaction speed index, the operation compliance data is converted into the rule compliance score, the physiological stress level data is associated with the cognitive stress level, and the test score data quantifies the knowledge mastery degree. After normalization processing, the above indicators generate a multi-dimensional evaluation vector, which represents the difference between the user's comprehensive ability state and the target ability model.

[0051] The multi-dimensional evaluation vector is input into the Bayesian optimization algorithm to construct a Gaussian process regression model. The model takes the user ability vector as the input, maps it to a high-dimensional feature space through a kernel function, and fits the non-linear relationship between the user ability curve and the preset target curve. Based on the posterior probability distribution, the ability deviation is calculated to quantify the ability gap of the user in the dimensions of knowledge application, risk decision-making, and stress response. The deviation is used as the optimization objective function to drive the subsequent parameter reverse optimization process.

[0052] Based on the ability deviation, the genetic algorithm initializes the population of the association strength of the knowledge graph nodes and the dynamic perturbation parameters. Through selection, crossover, and mutation operations, the population is iteratively optimized, and the deviation prediction value of the Gaussian process regression model and the stability constraint of the teaching strategy are combined when calculating the individual fitness. In the optimization process, the Pareto front solution set is retained, and the optimal combination of the association strength adjustment coefficient and the perturbation parameter is screened out to generate a set of policy parameter instructions. The instruction set is reversely transmitted to the data processing module through the microservice interface to update the reward function weight parameter of the deep deterministic policy gradient algorithm, strengthen the generation probability of the teaching strategy that conforms to the user ability characteristics, suppress the exploration direction of the strategy that deviates from the cognitive load threshold, and form a closed-loop optimization link of the teaching strategy and evaluation feedback.

[0053] Specifically, the confidentiality education system based on multi-modal interaction described in the present invention further includes a cross-module data collaboration unit, which is configured to: aggregate the multi-modal data streams collected by the data collection module through a federated learning framework, and perform distributed collaborative training with the data processing module to generate a global cognitive feature model; Perform noise injection desensitization processing on the user interaction data in the multi-modal data stream by using a differential privacy protection algorithm; Dynamically allocate the computing resources of the microservice instances of the training module and the optimization module through an ant colony optimization algorithm to adapt to the real-time load changes of the system.

[0054] The cross-module data collaboration unit aggregates the multi-modal data streams of the data collection module through a federated learning framework. The data streams include eye movement trajectories, voice commands, operation behaviors, and physiological signal data from different user terminals. The federated learning framework performs preliminary feature extraction and model training on the local device side, and only uploads the model parameters to the central server to avoid the transmission of raw data. The central server aggregates the model parameters of multiple terminals, generates a global cognitive feature model through weighted averaging, and synchronously updates it to the data processing module. The global model integrates the behavior patterns and cognitive features of multiple users, enhances the generalization ability of teaching strategies, and protects the privacy of user data at the same time.

[0055] The differential privacy protection algorithm performs desensitization processing on the user interaction data in the multi-modal data stream. The algorithm injects Gaussian noise during the local model training stage of federated learning to perturb sensitive information such as eye movement trajectory coordinates, voice command text vectors, and operation behavior sequences. The noise injection intensity is dynamically adjusted according to the data sensitivity. High-sensitivity fields are encrypted using the Laplace mechanism, and low-sensitivity fields are processed through random masking. The desensitized data retains the spatial correlation and temporal continuity of multi-modal features, meets the data requirements for cross-module collaborative training, and balances privacy protection and model training accuracy.

[0056] The ant colony optimization algorithm dynamically allocates the computing resources of the microservice instances of the training module and the optimization module. The algorithm initializes the virtual resource nodes as ant colony path nodes, and uses the real-time concurrent request volume of the system as the pheromone concentration index. By iteratively simulating the ant foraging path selection process, the allocation ratio of CPU, memory, and bandwidth resources of the microservice instances is optimized. During high-load periods, resources are preferentially allocated to the real-time rendering engine and the physical engine of the training module. During low-load periods, resources are tilted towards the genetic algorithm and the Bayesian optimization process of the optimization module. The dynamic allocation strategy adapts to the real-time load fluctuations of the system, maintains the balance of the interaction response speed and evaluation calculation efficiency in the teaching scenario, and supports the efficient operation of the closed-loop teaching system.

[0057] Specifically, in the confidentiality education system based on multi-modal interaction according to the present invention, the data processing module is further configured to: input the generated Pareto optimal solution set into a preset deep deterministic policy gradient algorithm, and dynamically adjust the virtual scene trigger threshold and the knowledge point reinforcement sequence based on the real-time operation data of the user fed back by the training module; Transmit the adjusted teaching instruction set to the training module through a message queue to drive the physics engine to update the file permission change frequency and the network attack gradient parameters.

[0058] The data processing module receives the Pareto optimal solution set generated by the multi-objective optimization algorithm. The solution set includes a non-dominated solution combination of teaching difficulty, cognitive load, and knowledge coverage. The deep deterministic policy gradient algorithm initializes the exploration boundary of the policy network based on the solution set, inputs the real-time operation data of the user as the environmental state, and dynamically evaluates the suitability of the virtual scene trigger threshold and the knowledge point reinforcement sequence. The real-time operation data includes the path deviation degree, response delay, and compliance operation ratio of the user in the current scene. The action value function is updated through online policy evaluation, and the sensitivity coefficient of the scene trigger threshold and the priority weight of the knowledge point reinforcement sequence are adjusted to generate a teaching strategy that dynamically matches the user's cognitive load.

[0059] The adjusted teaching instruction set is asynchronously transmitted to the training module through a message queue. The message queue adopts a publish-subscribe mode to achieve decoupling between modules. The instruction set includes the adjustment coefficient of the file permission change frequency, the update step of the network attack gradient, and the index list of the knowledge point reinforcement sequence. The physics engine analyzes the instruction set, dynamically adjusts the trigger frequency of the access control rule for the file processing scene based on the permission change coefficient, and updates the intrusion behavior complexity of the network attack and defense scene according to the attack gradient step. After the parameters are updated, the scene interaction logic is synchronously refreshed, and the compliance verification rule and the real-time feedback mechanism of the user operation path are adapted accordingly, forming a dynamic coordination between the teaching strategy and the user behavior.

[0060] The parameter update process is embedded in the online training loop of the reinforcement learning policy network. The policy network periodically transmits the policy gradient back to the data processing module according to the scene state change and the user operation data fed back by the physics engine. The gradient data drives the update of the policy network weights, optimizes the generation efficiency of the subsequent teaching instruction set and the scene adaptation accuracy, and constructs a closed-loop iterative link from policy generation, parameter execution to feedback optimization.

[0061] Specifically, in the confidentiality education system based on multi-modal interaction according to the present invention, the training module is further configured to: load the virtual simulation models of the file processing scene and the network attack and defense scene optimized by the simulated annealing algorithm through a three-dimensional modeling engine; Drive the decision logic of the scene NPC based on the behavior tree model, and generate comparison data with the preset compliance path according to the user operation data; Trigger the differential feedback animation matching the comparison data through the real-time rendering engine, and write the comparison data into the distributed time-series database.

[0062] The training module loads the virtual simulation models of the file processing scenario and the network attack and defense scenario optimized by the simulated annealing algorithm through the 3D modeling engine. The dynamic perturbation parameters generated by the simulated annealing algorithm include the file permission change frequency threshold and the network attack gradient adjustment coefficient, and the parameters are encoded into the physical engine configuration file of the scenario model. The 3D modeling engine analyzes the configuration file, constructs a virtual scenario including the permission level topology structure, the network node vulnerability distribution, and the attack path reachability map, and the dynamic perturbation parameters drive the real-time change of the scenario interaction logic to simulate the risk evolution process in the real confidentiality environment.

[0063] The user operation data is input into the virtual scenario through the multi-modal interface, and the behavior tree model analyzes the data and drives the decision-making logic of the scenario NPC. The pre-set decision nodes of the behavior tree include the permission overstep response rule, the attack path interception strategy, and the compliance operation guidance logic. The NPC triggers the corresponding decision branch according to the spatio-temporal coordinates and operation type of the user's current operation path. The decision-making logic outputs the comparison data of the spatio-temporal trajectories of the user operation path and the pre-set compliance path, and the comparison dimensions include the path deviation distance, the key node operation delay time, and the risk trigger times, generating multi-dimensional compliance quantification indicators.

[0064] The compliance quantification indicators are input into the real-time rendering engine to trigger the differential feedback animation matching the user operation behavior. The rendering engine calls the pre-set animation resource library according to the path deviation distance level. The permission overstep operation triggers the dynamic warning border and sound effect, and the compliance path matching operation generates the progressive guiding light effect. The animation rendering process integrates the real-time parameters of the physical engine, adjusts the light and shadow intensity and the interaction response speed of the interface elements, and enhances the authenticity of the user operation perception. The spatio-temporal coordinate data, compliance indicators, and feedback logs of the user operation path are marked with timestamps and stored in the distributed time-series database by slicing according to the scenario type and user ID, forming structured training data with time-series tags, providing data support for the multi-dimensional evaluation and strategy iteration of the optimization module.

[0065] Specifically, for the confidentiality education system based on multi-modal interaction described in the present invention, the optimization module is further configured to: generate multi-dimensional evaluation indicators by integrating the response delay data, operation compliance data, physiological stress level data, and test score data in the user operation path generated by the analytic hierarchy process; Input the multi-dimensional evaluation indicators into the genetic algorithm, inversely optimize the node association strength of the knowledge graph and the scenario perturbation parameters, and generate a set of strategy parameter instructions; The policy parameter instruction set is reversely transmitted to the data processing module through the microservice interface to update the policy parameters of the preset deep deterministic policy gradient algorithm.

[0066] The optimization module receives user operation path data from the training module, including response latency time, operation compliance mark, physiological stress level index, and test score data. The analytic hierarchy process constructs a multi-dimensional evaluation index system, and through expert weight assignment and consistency test, the data is standardized and weighted and fused. The response latency data is mapped to the user reaction speed index, the operation compliance data is converted into the rule compliance score, the physiological stress level data is associated with the cognitive stress level, and the test score data quantifies the knowledge mastery degree. After being normalized, the indexes generate a multi-dimensional evaluation vector, which characterizes the difference between the user's comprehensive ability state and the target ability model.

[0067] The multi-dimensional evaluation vector is input into the genetic algorithm to initialize the population of the association strength of the knowledge graph nodes and the dynamic perturbation parameters. The algorithm iteratively optimizes the population through roulette wheel selection, single-point crossover, and Gaussian mutation operations. When calculating the individual fitness, the deviation prediction value of the Gaussian process regression model and the stability constraint of the teaching strategy are combined. The Pareto front solution set is retained during the optimization process, and the optimal combination of the association strength adjustment coefficient and the perturbation parameter is selected to generate the policy parameter instruction set. The instruction set includes the knowledge graph node weight update coefficient, the file permission change frequency adjustment step, and the network attack gradient correction value, which are adapted to the user's real-time ability state.

[0068] The policy parameter instruction set is reversely transmitted to the data processing module through the microservice interface. The interface encapsulates the instruction data using the RESTful protocol to achieve cross-module asynchronous communication. The data processing module analyzes the instruction set and updates the reward function weight parameters of the deep deterministic policy gradient algorithm, strengthening the generation probability of the teaching strategy that conforms to the user's ability characteristics and suppressing the policy exploration direction that deviates from the cognitive load threshold. The updated reward function drives the policy network to re-evaluate the priority of the virtual scene trigger threshold and the knowledge point reinforcement sequence, forming a closed-loop optimization link of teaching strategy and evaluation feedback, and improving the dynamic adaptation ability of the confidentiality education system.

[0069] Specifically, for the confidentiality education system based on multi-modal interaction described in the present invention, the cross-module data collaboration unit is further configured to: aggregate the multi-modal data streams of multiple user terminals through the federated learning framework, construct a global cognitive feature model, and synchronize the global cognitive feature model to the data processing module; Inject noise into the user behavior data in the global cognitive feature model by using the differential privacy protection algorithm to generate desensitized global model data; The ant colony optimization algorithm is used to dynamically adjust the computing resource allocation strategy of the microservice instances in the training module to respond to changes in the real-time concurrent request volume of the system.

[0070] The cross-module data collaboration unit aggregates multimodal data streams from multiple user terminals through a federated learning framework. The data streams include heterogeneous data such as eye movement trajectories, voice commands, touch operations, and physiological signals. The federated learning framework performs preliminary feature extraction and model training on local terminals and only uploads model parameters to the central server, avoiding cross-device transmission of original user data. The central server performs weighted averaging and gradient aggregation on the model parameters uploaded by multiple terminals to generate a global cognitive feature model. The global model integrates the behavior patterns and cognitive feature distributions across user groups and is synchronized to the knowledge graph nodes of the data processing module to enhance the generalization ability and scenario adaptability of personalized teaching strategies.

[0071] The differential privacy protection algorithm performs noise injection desensitization processing on the user behavior data in the global cognitive feature model. The algorithm introduces Laplace noise in the parameter aggregation stage of federated learning to perturb the feature vectors strongly associated with user identities in the model parameters. An adaptive noise injection strategy is adopted for highly sensitive fields, and the noise intensity is dynamically adjusted according to the importance of the feature dimension. Low-sensitive fields are obfuscated through random masking. The desensitized global model data retains the statistical distribution characteristics of multimodal features, eliminates user individual identity information, and maintains the effectiveness of model training while meeting privacy protection requirements.

[0072] The ant colony optimization algorithm dynamically adjusts the computing resource allocation strategy of the microservice instances in the training module. The algorithm abstracts the CPU, memory, and bandwidth resources of the microservice instances as virtual path nodes, and uses the real-time concurrent request volume of the system as the path pheromone concentration index. By simulating the ant colony foraging path selection mechanism, the resource allocation ratio is iteratively optimized. During high-concurrency periods, resources are preferentially allocated to the real-time rendering engine and the physics engine to ensure the smoothness of virtual scene interactions. During low-load periods, resources are tilted towards compliance path comparison calculations and feedback animation generation processes to improve data processing throughput efficiency. The dynamic allocation strategy adapts to system load fluctuations and maintains resource balance for teaching training and evaluation optimization.

[0073] The explanations of the key technical features involved in the technical solution of this invention are as follows: The multimodal sensor unit refers to a composite sensor system integrating visual tracking, speech recognition, touch interaction, and physiological signal detection, which is used to synchronously collect multi-source heterogeneous data such as user eye movement trajectories, voice commands, operation behavior coordinates, and heart rate variability.

[0074] The adaptive Kalman filtering algorithm is a dynamic noise cancellation technology that suppresses sensor signal drift and random interference by adjusting filtering parameters in real time and retains data time series synchronization.

[0075] The tensor decomposition algorithm is used for cross-modal feature fusion. By jointly modeling the time, space, and feature dimensions, a three-dimensional data cube representing the spatio-temporal correlation of user behavior is constructed.

[0076] The graph convolutional neural network is a spatio-temporal feature extraction model. Based on the topological structure of the three-dimensional data cube, it extracts the user attention distribution heat map, knowledge understanding probability, and decision-making tendency vector, and generates a cognitive feature vector.

[0077] The Pareto optimal solution set refers to the set of non-dominated solutions output by the multi-objective optimization algorithm, which balances the teaching difficulty, cognitive load, and knowledge coverage rate, and constrains the generation boundary of the strategy.

[0078] The deep deterministic policy gradient algorithm is a reinforcement learning policy generation method. It dynamically adjusts the virtual scene trigger threshold and knowledge point reinforcement sequence in combination with real-time operation data, and generates a teaching instruction set.

[0079] The simulated annealing algorithm is used to optimize the dynamic perturbation parameters of the physical engine. By probabilistically accepting inferior solutions, it jumps out of the local optimum and generates the file permission change frequency and network attack gradient adapted to the user's ability.

[0080] The federated learning framework realizes cross-terminal data collaborative training. The local device extracts features and uploads model parameters, and the central server aggregates them to generate a global cognitive feature model, protecting user privacy.

[0081] The differential privacy protection algorithm injects Laplace noise into the model parameters to confuse sensitive user behavior data and achieve data desensitization.

[0082] The ant colony optimization algorithm dynamically allocates microservice resources. Taking the concurrent request volume as the pheromone index, it optimizes the allocation ratio of CPU, memory, and bandwidth to adapt to the real-time load of the system.

[0083] The above features cooperate to achieve multi-modal data fusion, dynamic strategy generation, closed-loop feedback optimization, and privacy protection, systematically solving the technical defects of one-way interaction, feedback lag, and one-sided evaluation of traditional systems, and improving the efficiency of internalizing confidential knowledge and the ability to handle complex scenarios.

[0084] The adaptive Kalman filter algorithm: Used for real-time noise cancellation of multi-modal data streams. It dynamically adjusts the filtering parameters according to the statistical characteristics of sensor noise, suppresses signal drift and random interference, and improves the signal-to-noise ratio of the data.

[0085] The tensor decomposition algorithm: Performs cross-dimensional fusion on time-aligned multi-modal data. By high-order singular value decomposition, it constructs a three-dimensional data cube including time, space, and feature dimensions to represent the spatio-temporal correlation of user behavior.

[0086] Graph Convolutional Neural Network: Extract spatio-temporal features based on the topological structure of three-dimensional data cubes, output an attention distribution heat map, a knowledge understanding probability, and a decision-making tendency vector to quantify the user's cognitive state.

[0087] Multi-objective Optimization Algorithm (NSGA-II): Generate a Pareto optimal solution set with the constraints of maximizing knowledge coverage and minimizing cognitive load, define the feasible domain of teaching strategies, and balance the user's learning pressure and knowledge reinforcement intensity.

[0088] Deep Deterministic Policy Gradient Algorithm (DDPG): Combine real-time interaction data with the Pareto optimal solution set, dynamically adjust the virtual scene trigger threshold and knowledge point reinforcement sequence, and generate a teaching instruction set.

[0089] Simulated Annealing Algorithm: Optimize the dynamic perturbation parameters of the virtual scene (such as file permission change frequency, network attack gradient), avoid local optima by probabilistically accepting inferior solutions, and generate highly realistic risk scenarios.

[0090] Real-time Dynamic Path Planning Algorithm (RRT): Generate a preset compliant path based on physical engine parameters, compare it with the user's operation path, and output path deviation and compliance indicators to support the feedback mechanism.

[0091] Bayesian Optimization Algorithm: Construct a Gaussian process regression model to quantify the deviation between the user's ability curve and the target curve, and identify weak links in knowledge application and risk decision-making.

[0092] Genetic Algorithm: Reverse-optimize the node association strength of the knowledge graph and the scene perturbation parameters, generate a strategy parameter instruction set through selection, crossover, and mutation operations, and drive the iterative update of teaching strategies.

[0093] Federated Learning Framework: Achieve collaborative training of multi-terminal data. After local devices extract features, they only upload model parameters, and the central server aggregates them to generate a global cognitive feature model to avoid the leakage of raw data.

[0094] Differential Privacy Protection Algorithm: Inject Laplace noise during the parameter aggregation stage of federated learning to obfuscate the sensitive features of user behavior data and meet the privacy protection requirements.

[0095] Ant Colony Optimization Algorithm: Dynamically allocate the computing resources of microservice instances, use the concurrent request volume as the pheromone index, optimize the allocation ratio of CPU, memory, and bandwidth, and adapt to the real-time load fluctuations of the system.

[0096] The confidentiality education system based on multimodal interaction provided by the present invention realizes a closed-loop teaching process through the collaborative work of multiple modules. The data acquisition module synchronously acquires the eye movement trajectory data, speech instruction semantic data, operation behavior coordinate data, and physiological signal data of the user through the multimodal sensor unit. These multi-source heterogeneous data are aligned in time series through the timestamp synchronization technology to eliminate the timing deviation between sensors, and a time-aligned multimodal data stream is generated. The adaptive Kalman filtering algorithm performs noise cancellation processing on the multimodal data stream, and dynamically adjusts the filtering parameters to suppress signal drift and random interference. The filtered data is input into the tensor decomposition algorithm to perform cross-modal feature fusion in the time, space, and feature dimensions, constructing a three-dimensional data cube to represent the spatio-temporal correlation of user behavior and the coupling relationship of multimodal features. The three-dimensional data cube is transmitted to the data processing module as the basic input for cognitive feature modeling.

[0097] Based on the three-dimensional data cube, the data processing module uses a graph convolutional neural network to extract the attention distribution heat map, knowledge understanding probability, and decision-making tendency vector of the user in the spatio-temporal dimension. The cognitive feature vector is matched with the nodes of the knowledge graph through a multi-objective optimization algorithm to generate a Pareto optimal solution set including teaching difficulty, cognitive load, and knowledge coverage. The solution set is input into the preset deep deterministic policy gradient algorithm, and the virtual scene trigger threshold and knowledge point reinforcement sequence are dynamically adjusted in combination with the user's real-time operation data to generate a teaching instruction set. The instruction set is transmitted to the training module through the message queue to drive the update of the scene interaction logic.

[0098] The training module loads the confidentiality scene model according to the teaching instruction set, optimizes the dynamic disturbance parameters of the physics engine based on the simulated annealing algorithm, and adjusts the file permission change frequency of the file processing scene and the network attack gradient of the network attack and defense scene in real time. The operation data triggered by the user through the multimodal interface is driven by the behavior tree model to determine the decision-making logic of the scene NPC, generating comparison data between the user operation path and the preset compliance path. The real-time dynamic path planning algorithm generates a compliance path based on the optimized parameters, and the real-time rendering engine triggers a differential feedback animation to enhance the immersion and feedback immediacy of the user operation. The user operation path data and comparison data are written into the distributed time series database and pushed to the optimization module for multi-dimensional evaluation.

[0099] The optimization module uses the analytic hierarchy process to fuse the response delay, compliance, physiological stress, and test data in the user operation path to generate multi-dimensional evaluation indicators. The Bayesian optimization algorithm constructs a Gaussian process regression model to quantify the deviation between the user ability curve and the target curve. The genetic algorithm reversely optimizes the association strength of the knowledge graph nodes and the scene disturbance parameters based on the deviation to generate a strategy parameter instruction set. The instruction set is reversely transmitted to the data processing module through the microservice interface to update the reward function weight of the deep deterministic policy gradient algorithm, forming a closed-loop iterative link from policy generation, execution to feedback optimization.

[0100] The cross-module data collaboration unit aggregates multi-terminal data streams through the federated learning framework, extracts features and trains models on the local device side, and only uploads model parameters to the central server to generate a global cognitive feature model, which is synchronized to the data processing module to enhance the policy generalization ability. The differential privacy protection algorithm injects Laplace noise during the parameter aggregation stage to desensitize sensitive feature vectors and balance privacy protection and model accuracy. The ant colony optimization algorithm dynamically allocates microservice resources for the training module, prioritizes resource requirements for real-time rendering and the physics engine during high-concurrency periods, and tilts resources to the evaluation calculation process during low-load periods to adapt to real-time load fluctuations of the system.

[0101] Each module solves the limitations of one-way transmission and single interaction in traditional systems through multi-modal data fusion, dynamic policy generation, and closed-loop evaluation and optimization. Users improve their ability to apply confidentiality knowledge through high-fidelity scenario training in multi-modal interaction. The system dynamically adjusts teaching strategies based on real-time feedback to quickly internalize the risk response ability in complex scenarios. The proposed solution ensures data security while improving user participation and feedback accuracy through data collaboration and privacy protection mechanisms, meeting the intelligent and personalized needs of modern confidentiality education systems.

[0102] The present invention solves the problem of insufficient user participation in existing systems through multi-modal data fusion and dynamic policy generation mechanisms. The data acquisition module integrates visual tracking, speech recognition, touch interaction, and physiological signal detection units to capture multi-dimensional data such as the user's eye movement trajectory, operation behavior, and physiological feedback in real time, and constructs a three-dimensional data cube with spatio-temporal correlation through the tensor decomposition algorithm. The cognitive feature modeling module extracts features such as the user's attention distribution, knowledge comprehension, and decision-making tendency based on the graph convolutional neural network, generates a Pareto optimal solution set adapted to the user's cognitive state in combination with the multi-objective optimization algorithm, and drives the dynamic adjustment of the virtual scene trigger threshold and knowledge point reinforcement sequence to achieve the transformation of the teaching mode from one-way indoctrination to multi-modal interaction and enhance the depth of user behavior participation.

[0103] To address the problem of inaccurate teaching feedback, the system improves feedback accuracy through a closed-loop evaluation and reverse optimization mechanism. The virtual simulation module optimizes dynamic perturbation parameters based on the simulated annealing algorithm to generate compliance path comparison data matching the real-time teaching strategy, and drives the NPC decision logic through the behavior tree model to trigger differential feedback animations. The optimization module uses the analytic hierarchy process to fuse user operation responses, physiological stress, and test data to construct multi-dimensional evaluation indicators, and reversely adjusts the node association strength of the knowledge graph and the scene perturbation parameters through Bayesian optimization and genetic algorithms to generate a policy parameter instruction set to update the reinforcement learning model in reverse, forming an accurate feedback link from user behavior collection, multi-dimensional evaluation to policy iteration.

[0104] To improve the efficiency of knowledge internalization, a cross-module collaboration and dynamic resource allocation system is systematically constructed. The federated learning framework aggregates multi-terminal data streams to generate a global cognitive feature model, combines differential privacy protection algorithms to achieve data desensitization, and enhances the generalization ability of teaching strategies. The ant colony optimization algorithm dynamically allocates microservice resources to adapt to real-time load changes, ensuring the balance between the interactive response speed of virtual scenarios and the evaluation calculation efficiency. The deep deterministic policy gradient algorithm continuously optimizes the weights of the reward function based on closed-loop feedback data, dynamically balances the knowledge coverage rate and cognitive load constraints, and drives users to complete knowledge application training in highly realistic scenarios through teaching instruction sets, realizing the rapid internalization of risk response capabilities in complex scenarios.

Claims

1. A confidentiality education system based on multimodal interaction, characterized by: include: Data acquisition module, data processing module, training module and optimization module; The data acquisition module is configured to collect multi-source heterogeneous data through a multimodal sensor unit, wherein the multi-source heterogeneous data includes the user's eye movement trajectory data, voice command semantic data, operation behavior coordinate data, and physiological signal data, perform time synchronization processing on the multi-source heterogeneous data to generate a time-aligned multimodal data stream, construct a three-dimensional data cube through a cross-modal feature fusion algorithm, and transmit the data to the data processing module; The data processing module is configured to extract user cognitive features based on the three-dimensional data cube, generate teaching strategy constraints by matching knowledge graph nodes through a multi-objective optimization algorithm, and dynamically generate a teaching instruction set in combination with a deep reinforcement learning algorithm and transmit it to the training module; The training module is configured to load the confidential scenario model according to the teaching instruction set, adjust the scenario interaction logic based on the dynamic perturbation parameters, receive user multimodal operation data and generate compliance path comparison results, trigger real-time feedback and write the interaction data into the distributed database and push it to the optimization module; The optimization module is configured to integrate user operation response data, physiological feedback data and test data to build an ability evaluation model, generate a strategy parameter instruction set through an inverse optimization algorithm, and reversely update the reinforcement learning parameters of the data processing module.

2. The multimodal interaction-based confidentiality education system according to claim 1, characterized in that: The data acquisition module is further configured to: perform noise elimination processing on the multimodal data stream through an adaptive Kalman filtering algorithm to generate filtered multimodal data; input the filtered multimodal data into a tensor decomposition algorithm for spatiotemporal feature fusion, construct a three-dimensional data cube including a time dimension, a space dimension, and a feature dimension, and transmit the three-dimensional data cube to the data processing module.

3. The multimodal interaction-based confidentiality education system according to claim 2, characterized in that: The data processing module is further configured to: extract a user attention distribution heat map, knowledge understanding probability, and decision tendency vector from the three-dimensional data cube using a graph convolutional neural network to generate a cognitive feature vector; The cognitive feature vector is matched with the knowledge graph node through a multi-objective optimization algorithm to generate a Pareto optimal solution set including teaching difficulty, cognitive load and knowledge coverage, so as to constrain the teaching strategy exploration scope of the deep reinforcement learning algorithm.

4. The multimodal interactive confidentiality education system according to claim 3 is characterized in that ； The training module is further configured to: optimize the dynamic perturbation parameters of the physics engine based on a simulated annealing algorithm, and adjust the file permission change frequency of the file processing scenario and the network attack gradient of the network attack and defense scenario in real time; Based on the optimized dynamic disturbance parameters, a preset compliance path is generated through a real-time dynamic path planning algorithm. The user operation path is compared with the preset compliance path to generate compliance path comparison data. Trigger differentiated feedback animation based on compliance path comparison data, and write user operation paths and comparison data into a distributed time series database.

5. The multimodal interactive confidentiality education system according to claim 4 is characterized in that ； The optimization module is further configured to: generate multi-dimensional evaluation indicators by fusing response delay data, operation compliance data, physiological stress level data and test score data in the user operation path based on the hierarchical analysis method; input the multi-dimensional evaluation indicators into the Gaussian process regression model constructed by the Bayesian optimization algorithm to quantify the deviation between the user ability curve and the target ability curve; reversely optimize the association strength and dynamic perturbation parameters of the knowledge graph nodes through the genetic algorithm to generate a strategy parameter instruction set, and reversely transmit the strategy parameter instruction set to the data processing module to update the reward function weight of the preset deep deterministic policy gradient algorithm.

6. The multimodal interaction-based confidentiality education system according to claim 5, characterized in that: It also includes a cross-module data collaboration unit configured to: aggregate the multimodal data streams collected by the data acquisition module through a federated learning framework, perform distributed collaborative training with the data processing module, and generate a global cognitive feature model; A differential privacy protection algorithm is used to perform noise injection and desensitization on user interaction data in multimodal data streams; The ant colony optimization algorithm is used to dynamically allocate computing resources of the microservice instances of the training module and the optimization module to adapt to the real-time load changes of the system.

7. The multimodal interaction-based confidentiality education system according to claim 6, characterized in that: The data processing module is further configured to: input the generated Pareto optimal solution set into a preset deep deterministic policy gradient algorithm, and dynamically adjust the virtual scene trigger threshold and knowledge point reinforcement sequence based on the real-time user operation data fed back by the training module; The adjusted teaching instruction set is transmitted to the training module through the message queue, driving the physics engine to update the file permission change frequency and network attack gradient parameters.

8. The multimodal interactive confidentiality education system according to claim 7 is characterized in that ； The training module is further configured to: load virtual simulation models of file processing scenarios and network attack and defense scenarios optimized based on simulated annealing algorithms through a three-dimensional modeling engine; Drive the scene NPC decision logic based on the behavior tree model, and generate comparison data with the preset compliance path based on user operation data; The real-time rendering engine triggers differentiated feedback animations that match the comparison data, and writes the comparison data into a distributed time series database.

9. The multimodal interactive confidentiality education system according to claim 8 is characterized in that ； The optimization module is further configured to: generate a multi-dimensional evaluation index by fusing the response delay data, operation compliance data, physiological stress level data and test score data in the user operation path generated by the analytic hierarchy process; Input multi-dimensional evaluation indicators into the genetic algorithm, reversely optimize the knowledge graph node association strength and scenario disturbance parameters, and generate a strategy parameter instruction set; The policy parameter instruction set is transmitted back to the data processing module through the microservice interface to update the policy parameters of the preset deep deterministic policy gradient algorithm.

10. The multimodal interaction-based confidentiality education system according to claim 9 is characterized in that ； The cross-module data collaboration unit is further configured to: aggregate multimodal data streams of multiple user terminals through a federated learning framework, construct a global cognitive feature model, and synchronize the global cognitive feature model to the data processing module; A differential privacy protection algorithm is used to inject noise into user behavior data in the global cognitive feature model to generate desensitized global model data. The computing resource allocation strategy of the microservice instances of the training module is dynamically adjusted through the ant colony optimization algorithm to respond to changes in the system's real-time concurrent request volume.

Images