31816 results about "Multi modality" patented technology

The invention discloses a power transmission and distribution production task cooperation system and method based on an intelligent agent, and relates to the technical field of power distribution production task scheduling, the system comprises six modules: a natural language input interaction module processes a user instruction and multi-modal information, and generates structured data; the electric power field knowledge enhancement analysis module establishes mapping from a natural language to business data; the dynamic interaction context memory module stores historical interaction data and generates a context feature vector through a bidirectional LSTM and an attention mechanism; the intelligent task scheduling and conflict resolution module is used for disassembling instructions into sub-tasks, dynamically evaluating priorities in combination with three-dimensional indexes and resolving resource conflicts; the agent task execution and cooperation module drives agents to execute tasks according to priorities and synchronize states in real time; the system closed-loop feedback optimization module analyzes the execution log and automatically updates model parameters; according to the system, the problems of term analysis deviation, strategy staticization and insufficient self-optimization capability of a traditional scheduling system are solved.

The invention provides an industrial environment monitoring and accident prediction method fusing multi-modal data, and relates to the technical field of data processing, and the method comprises the steps: carrying out the semantic collection and causal association preprocessing of multi-modal heterogeneous data collected in real time through constructing a dynamic industrial knowledge graph; a customized deep learning model is adopted to extract deep abstract features of each mode, and weak signals and potential risks are accurately represented and uncertainty is quantified; a high-fidelity digital twin model is utilized to drive a deep reinforcement learning algorithm, and dynamic optimization and verification are performed to generate a multi-level and multi-target preventive intervention strategy combination; an intervention strategy is executed through an edge-end-cloud three-layer collaborative intelligent architecture, and online learning and system sustainable evolution are realized by using a closed-loop data feedback mechanism. According to the method, the sensing and early warning capability of the early weak and complex abnormal state of the industrial environment can be remarkably improved, the accident evolution path is accurately predicted, and credible explanation is provided.

The invention discloses an auditing decision support system and method based on a dynamic knowledge graph, relates to the technical field of computers, and aims to solve the problems that auditing data are heterogeneous and complex, risk identification is not timely and causal interpretation is lacked. According to the system, multi-modal audit data is collected in real time through a streaming event processing framework, and a dynamic audit knowledge graph with timeliness weight is constructed. Based on a graph calculation engine and cross-domain rule mining, identifying a high-frequency risk mode, and generating a risk conduction path graph; further fusing a multi-modal graph attention network, identifying and positioning abnormal entities, and outputting abnormal nodes and risk links thereof; and finally, the abnormal node embedding representation is dynamically updated through the time sequence diagram attention network, an interpretable audit causal map is generated in combination with a structural causal model, and closed-loop support from data acquisition and risk identification to interpretive audit decision is realized. The intellectualization and transparency of audit decision making are improved, and an efficient and traceable decision making basis is provided for a complex audit scene.

A network mapping behavior anomaly detection method and system based on machine learning is provided. The method includes: collecting dual-source traffic data, generating a structured log data set through dual-source log fusion engine; performing subgraph matching calculation to obtain a mapping behavior deviation degree; generating communication data containing a watermark identifier in a session corresponding communication path; verifying whether attack events carry the watermark identifier; generating a network mapping behavior anomaly detection report. According to the disclosure, an adaptive attack behavior model is constructed through a multi-modal feature vector based on structured logs and a graph protocol mapping rule base, so that the cognitive robustness to protocol camouflage and path drift is fundamentally enhanced, a real-time verification chain of detection results is built, and traditional passive detection is transformed into self-proof active defense through cross verification of watermark carrying state and behavior trajectory.

The invention relates to the technical field of industrial automation control, and discloses a multi-machine collaborative industrial robot intelligent scheduling system and an application method.The system comprises a data acquisition module for acquiring real-time motion data through a multi-modal sensor; the feature fusion module is used for generating environment dynamic feature data based on graph neural network fusion data; the strategy generation module is used for generating collaborative strategy parameters by utilizing a collaborative reinforcement learning model; the trajectory optimization module is used for constructing a multi-target trajectory optimization model to plan an optimal synchronous trajectory; the motion control module is used for establishing a layered motion control model to realize cooperative motion control; a dynamic resource allocation module can also be included to optimize resource allocation. The application method sequentially executes corresponding steps according to the modules of the system. According to the method, the environment can be accurately sensed, the cooperation strategy can be optimized, the efficient track can be planned, accurate motion control and reasonable resource allocation can be realized, and the cooperative motion control capability and the production efficiency of the industrial robot in the complex environment can be effectively improved.

The invention provides a home abnormal state signal detection method and system based on multi-modal sensing, and relates to the technical field of detection, and the method comprises the steps: collecting human body motion, acoustics and environment parameters through a millimeter wave radar, an acoustic sensor and an environment sensor, inputting the parameters into a deep fusion network, a spatio-temporal attention mechanism and a multi-scale convolutional neural network are used to extract a spatio-temporal feature sequence, and joint probability features are formed in combination with an acoustic feature spectrum analysis result. And inputting the joint probability features into a time sequence knowledge graph, and obtaining scene adaptive features after processing by a graph attention network and a double-flow auto-encoder network. And finally, the multi-task learning network is combined with the risk propagation neural network and the recursive neural network to carry out anomaly prediction and risk level evaluation, and is combined with historical early warning information to output an early warning strategy. According to the invention, multi-modal data can be effectively fused, the accuracy and reliability of home abnormal state detection are improved, and a more accurate risk assessment and early warning strategy is provided.

The invention provides a notarization intelligent question and answer customer service method and system based on a knowledge graph, and relates to the technical field of information, and the method comprises the steps: collecting multi-source heterogeneous data in the notarization field, forming an entity set and a relationship set, combining to form an initial knowledge graph, carrying out a dynamic update mechanism based on time decay, and supplementing to form a dynamic knowledge graph. Carrying out vectorization expression to obtain an atlas vector; receiving a multi-modal problem, determining fusion features, performing deep semantic understanding, obtaining semantic features, determining a user intention, and forming a structured query expression; based on the user intention and the structured query expression, carrying out multi-hop reasoning in the dynamic knowledge graph, carrying out similarity calculation by utilizing graph vectors, determining a reasoning path in combination with the user intention, carrying out knowledge association reasoning along the reasoning path, obtaining a preliminary reasoning result, carrying out interpretability analysis on the preliminary reasoning result, and obtaining a final reasoning result; and generating a reasoning chain and a confidence score, forming a detailed reasoning result, and finally generating a personalized answer.

The invention belongs to the technical field of knowledge retrieval, and discloses a multi-modal large model-based dynamic vector knowledge base construction and retrieval method, which comprises the following steps of: obtaining a multi-source heterogeneous modal data set, and carrying out preprocessing and modal standardization processing on the multi-source heterogeneous modal data set to obtain a standardized multi-modal data set; performing feature extraction and semantic vector representation generation by using the pre-trained multi-modal large model, and constructing a multi-modal knowledge vector set; semantic association analysis and hierarchical clustering are carried out on the multi-modal knowledge vector set, and a structured vector knowledge base is constructed; performing semantic similarity calculation and relation modeling on the vector knowledge base to form a vector relation network; intention analysis and vector representation are performed based on mixed modal query information input by a user, and efficient similarity retrieval is realized in combination with a vector relation network; dynamic optimization is carried out through user feedback, personalized retrieval result adjustment is achieved, and the problem of limitation of a traditional retrieval system during multi-modal data processing is effectively solved.

The invention discloses a system and a method for task planning of an intelligent robot with a body based on multi-dimensional situation awareness, and particularly relates to the technical field of task planning of the intelligent robot with the body, space-time alignment is carried out on asynchronous heterogeneous data generated by a multi-modal sensor channel, and cross-modal space-time features are extracted through a cross-modal feature fusion network; a fusion situation matrix is generated, dynamic causal modeling is used to update association strength among the multi-modal data, an anti-factual reasoning engine is used to identify and trace abnormities, and an abnormities traceability result is output; dynamically adjusting the reliability weight of each sensing channel through a multi-modal credibility evaluation model by using the fusion situation matrix and an abnormal traceability result; and on the basis of the reliability weight, inputting the fusion situation matrix into a real robot dynamic model and a digital twin virtual model, executing collaborative predictive control, and starting an adaptive rule evolution mechanism when a safety score is lower than a threshold value, thereby solving the problem of fusion matrix distortion in dynamic obstacle avoidance and precise grabbing tasks.

The invention discloses a target identification tracking method and system based on multi-source fusion imaging, and the method comprises the steps: obtaining and preprocessing multi-modal environment monitoring data, carrying out the image enhancement based on the preprocessed multi-modal environment monitoring data, and obtaining the image-enhanced multi-modal environment monitoring data; performing multi-modal feature extraction on the image enhancement multi-modal environment monitoring data, and introducing a cross attention mechanism to fuse the extracted multi-modal features to construct cross-modal fusion features; constructing a cascade target recognition model, and inputting cross-modal fusion features to perform target recognition on the current monitoring scene; tracking a scene recognition target in the current monitoring scene to generate scene target tracking information, and storing the scene target tracking information in a preset track memory pool; and when a tracking target disconnection condition occurs during target tracking, obtaining a mismatched tracking trajectory, and performing tracking trajectory disconnection repair in combination with the trajectory memory pool. Therefore, the identification accuracy and tracking reliability of the target in the monitoring scene are improved.

The invention belongs to the technical field of visual special effects, and discloses an AI visual special effect dynamic generation system fusing multi-modal perception. Through cooperative work of six core modules of multi-modal input processing, importance analysis, parameter configuration, parameterized rendering, parameter optimization and rendering and output, deep fusion of special effects and contents is realized. An image sequence, audio data and scene parameters can be processed at the same time, a multi-modal feature data set is constructed, a scene key visual area is scientifically recognized, a visual importance distribution map is generated, and the special effect parameter configuration and rendering process is guided. An improved neural radiation field technology and a physical constraint model are adopted to ensure the consistency and reality of the special effect at different visual angles; and a multi-dimensional quality evaluation and parameter automatic optimization mechanism is introduced, so that the visual expressive force and the artistic value of the special effect are guaranteed. The creation threshold is remarkably reduced, the cooperative expression ability of the special effect and the content is improved, and the special effect becomes a powerful tool for enhancing the narration and enhancing the emotion.

The invention discloses a multi-modal AI-based power equipment anomaly detection method and system, and the method comprises the steps: synchronously collecting electrical, mechanical and thermal modal data of power equipment through an edge computing node, carrying out the load adaptive dynamic preprocessing, and uploading the data to a cloud end; the cloud constructs a multi-modal feature extraction network based on a structural causal model, analyzes a causal path between modals through a Bayesian network and performs weighted fusion on feature vectors; capturing device state mutation by using a gating attention mechanism, and updating the feature vector; executing time-space consistency verification of the equipment group to identify regional group abnormality and suppress single-point misinformation; generating an interpretable report containing an abnormal root cause analysis and priority ranking maintenance strategy; and establishing a closed-loop feedback mechanism to correct the cause and effect probability distribution of the Bayesian network model. The system comprises a multi-modal sensor array, an edge computing node and a cloud analysis platform, wherein the cloud analysis platform is integrated with a causal reasoning engine, a space-time consistency verification module and the like. According to the invention, by analyzing the multi-modal deep causal association, the method adapts to the dynamic change of the equipment, reduces the false alarm rate, generates an interpretable report, supports closed-loop self-optimization, and improves the anomaly detection accuracy and operation and maintenance decision efficiency of the power equipment.

The invention discloses an aerospace intelligent manufacturing large model construction method, which comprises the steps of collecting original data, performing preprocessing and data association, and constructing an aerospace intelligent manufacturing database; establishing a knowledge acquisition and structured conversion assembly line, a multi-dimensional associated domain knowledge graph, a knowledge quality control system and a dynamic updating mechanism, and constructing a professional knowledge base; aligning the cross-modal manufacturing data to generate a corpus; combining base general large model pre-training, injecting terminology semantics and multi-modal association capability, and completing knowledge migration; based on the pre-trained aerospace intelligent manufacturing large model, constructing an aerospace manufacturing cognitive agent, and forming a complex engineering problem solving framework; professional ability is optimized through a two-stage progressive multi-task training strategy, and dynamic adaptation of a production environment is realized in combination with an online learning and incremental updating mechanism. The intelligent level of aerospace intelligent manufacturing is remarkably improved.

The invention relates to the technical field of intelligent energy management, in particular to an energy consumption prediction and optimization system for energy-saving management and control, which comprises a data acquisition core module, a dynamic energy consumption prediction core module, an intelligent optimization control core module, a self-adaptive calibration core module, a user interaction core module and the like. The data acquisition module acquires energy consumption, equipment state and environment data from multiple sources; the dynamic energy consumption prediction module fuses improved time series decomposition and a multi-modal LSTM model to realize accurate prediction; the intelligent optimization control module is combined with strategies such as time-of-use electricity price and equipment linkage to generate an optimal instruction; the adaptive calibration module dynamically optimizes the model through Kalman filtering and incremental learning; the user interaction module supports visual display and strategy self-definition; in addition, the system is provided with an edge computing node to guarantee offline operation, an SM4 algorithm and a block chain technology are adopted to guarantee data security, and the system is compatible with various industrial protocols. The energy utilization efficiency is effectively improved, the operation cost is reduced, and the system safety and reliability are enhanced.

A multi-modal information tagging method, apparatus and device, and a storage medium and a product. An information tagging effect is effectively improved by means of determining visual feature information of visual information by means of a multi-modal tagging system, performing multi-modal fusion processing on the basis of the visual feature information, the visual information, recognized text information and descriptive text information to obtain image-text fusion features, determining image-text correlation information between the visual information and both the recognized text information and the descriptive text information, on the basis of the image-text fusion features and the image-text correlation information, determining a tagging result for information to be tagged, and performing multidimensional information tagging on the basis of multi-modal information fusion and cross-modal correlations.

The invention provides a network security penetration detection method and system based on artificial intelligence, and the method comprises the steps: firstly obtaining real-time network flow data containing multi-dimensional communication behavior records and protocol metadata of a target network system, and then extracting network behavior sequence features capable of describing communication behavior time relevance and a protocol interaction mode from the network flow data; inputting the network behavior sequence features into a pre-trained multi-modal threat prediction model, generating a potential attack vector prediction result containing an attack intention label, penetration path probability distribution and a threat level score, and dynamically verifying the potential attack vector prediction result based on a threat knowledge base, and finally, generating a self-adaptive defense strategy according to the penetration threat detection result, triggering defense interface deployment of a target network system, updating attack mode characteristics and a defense rule set of a threat knowledge base, and realizing intelligent and efficient network security penetration detection and defense.

The invention relates to the technical field of meteorological data analysis, and provides an aerosol prediction method and system based on artificial intelligence, which are used for accurately identifying an abnormal aerosol gathering mode in an extreme meteorological event. The method comprises the steps that an original multi-mode meteorological observation data set is acquired, and the original multi-mode meteorological observation data set comprises an earth surface aerosol concentration sequence, three-dimensional atmosphere state variables and boundary layer dynamic parameters; performing space-time heterogeneity correction on the original multi-modal meteorological observation data set to generate a space-time aligned target multi-modal meteorological observation data set; inputting the target multi-modal meteorological observation data set into a spatial-temporal feature fusion network to extract multi-scale spatial distribution features and cross-modal time sequence correlation features, and generating a spatial-temporal feature matrix based on the multi-scale spatial distribution features and the cross-modal time sequence correlation features; and performing multi-stage recursive optimization on the spatial-temporal characteristic matrix through a hierarchical time domain aggregation algorithm to generate an aerosol concentration prediction sequence of the target region.

The invention discloses an intelligent programming auxiliary method and system based on a multi-modal AI language model, and belongs to the technical field of programming auxiliary tools. The method comprises the following steps: a multi-modal input processing stage; a dynamic context modeling stage; a hierarchical semantic analysis stage; in the code generation stage, codes are generated in two stages by adopting a Codex-Plus large model; the reinforcement learning driven code optimization stage is used for carrying out multi-objective optimization and reward function design on the codes generated in the code generation stage; a multi-dimensional feedback stage; an interaction and visualization stage; code semantic deep analysis, dynamic context sensing, multi-target optimization generation and real-time interactive feedback are realized by fusing code texts, natural language description, developer behavior data and a domain knowledge graph, and programming efficiency and code quality can be remarkably improved.

The invention relates to the technical field of data security management, in particular to a self-adaptive data security management and risk early warning system based on intelligent analysis under a cloud platform. Comprising a multi-dimensional data acquisition module; an intelligent analysis module; a self-adaptive strategy generation module; a risk early warning module; and a user behavior portrait construction module. In the design, the security policy can be dynamically adjusted along with the risk situation of the cloud platform, the problem that a static policy cannot adapt to real-time change is solved, and dynamic mapping of risk characteristics-policy parameters is realized; according to the design, the one-sidedness of single-dimension analysis is broken through, multi-modal feature association modeling of user behaviors is achieved, an abnormal behavior triggering threshold value is accurately recognized, and the integrity and accuracy of risk feature analysis are improved; the security policy can be continuously optimized through historical event data, so that protection efficiency attenuation caused by long-term static operation is avoided, and an autonomous lifting link of data driving, algorithm optimization and policy evolution is realized.

The invention discloses a knowledge base system construction method and device for multi-source heterogeneous data and a medium, and relates to the technical field of artificial intelligence and natural language processing. The method comprises the following steps: integrating a dynamic graph convolutional network and a hierarchical attention mechanism to construct a multi-modal document analysis engine; performing semantic structure analysis on the original heterogeneous document on the basis of a multi-modal document analysis engine to extract document structure features and content semantic features, and constructing an original document relationship model on the basis of the document structure features and the content semantic features; based on the original document relationship model, performing classification fusion on heterogeneous data in the original heterogeneous document to obtain a to-be-stored heterogeneous data corpus, and processing the to-be-stored heterogeneous data corpus by using a graph neural network to establish a cross-modal semantic association index; and based on the cross-modal semantic association index, performing classified storage on the to-be-stored heterogeneous data corpora by utilizing a preset heterogeneous database so as to complete knowledge base system construction of the multi-source heterogeneous data.

The invention relates to the technical field of computers, provides an automatic test script dynamic generation method and system based on multi-modal AI recognition, and is used for remarkably improving the coverage dimension and execution robustness of test cases while reducing the manual maintenance cost. The method comprises the following steps: acquiring target test interface data, extracting dynamic interaction characteristics in the test interface data, calling a trained target detection network to carry out element positioning processing on the test interface data, generating an interface element coordinate set, and generating a target test interface based on the interface element coordinate set. Calling a character recognition network to carry out multi-modal feature analysis on the interface element distribution information, and generating a feature fusion result containing character semantic features and icon classification features; the feature fusion result is input into a script generation model for dynamic path planning processing, automatic test script elements are output, and the automatic test script elements comprise logic control instructions generated based on the element operation sequence and coordinate self-correction parameters.

The invention discloses a cross-modal information fusion and retrieval method and system driven by semantic comprehension, and the method comprises the steps: obtaining text, image and audio original data, and extracting an initial feature set of each modal through a deep neural network; dynamically distributing each modal weight coefficient based on an attention mechanism, and performing weighted fusion on the initial feature set to obtain cross-modal fusion feature representation; through a cross-modal semantic association analysis model, high-dimensional semantic association features are extracted from the fusion feature representation, and semantic enhancement feature vectors are generated; constructing a cross-modal semantic graph network based on the vector, complementing missing modal features, and generating an optimized multi-modal feature set; and inputting the optimized feature set and the query sample into a contrast learning model, calculating a semantic similarity score, and generating a cross-modal retrieval result sorting list according to the score.

The invention relates to the technical field of numerical control machining, in particular to a tool cutting path machining error compensation method based on online monitoring, which comprises the following steps: establishing a cutting path data model; constructing a multi-modal sensor assembly, and collecting error data in real time; establishing an error state equation to carry out error calculation; constructing a double-layer compensation mechanism; and executing the compensation instruction by adopting a real-time control system. Through real-time synchronous monitoring of the multi-mode sensor, a mixed error resolving framework and a double-layer intelligent compensation mechanism, the influence of dynamic error sources such as cutter abrasion and machine tool thermal deformation can be effectively overcome, the machining precision and the surface quality are remarkably improved, and the high-precision machining requirements in the fields of aerospace, precise instruments and the like are met.

The invention discloses a power plant intelligent maintenance method and system based on multi-modal dynamic graph learning. The method comprises the following steps: acquiring structured sensor data, unstructured data and equipment physical topology data of equipment operation in real time through a multi-source sensor cluster and an industrial terminal; the method comprises the following steps: preprocessing multi-modal data, and fusing multi-modal features by using a double-flow Transform architecture and a gated attention mechanism to generate a joint embedded representation; constructing a dynamic causal graph based on equipment physical topology data and sensor time sequence characteristics, updating an edge weight through a GraphSAGE algorithm, fusing domain rule constraints, and outputting equipment state information; generating a maintenance strategy through an improved near-end strategy optimization algorithm according to the state and the equipment health index; and finally, the maintenance strategy triggers third-level early warning of the DCS through an OPC UA protocol, and a maintenance instruction is accurately issued. According to the method, the defects of a traditional method in the aspects of data fusion, fault modeling and decision making are overcome, and the safety, the economical efficiency and the operation and maintenance intelligent level of power plant equipment are remarkably improved.

The invention provides a battery assembly process defect real-time detection and classification method and system, and relates to the technical field of intelligent manufacturing, and the method comprises the steps: carrying out the standardization preprocessing of collected battery surface images and three-dimensional point cloud data; a distributed edge computing architecture is adopted for feature extraction, the distributed edge computing architecture comprises a defect propagation prediction module based on a graph neural network, inter-process defect features are constructed into a dynamic graph structure, a defect evolution rule is learned through the graph network, and feature extraction parameters are dynamically adjusted; inputting the extracted features into a deep learning detection model, carrying out feature adaptive mapping and alignment through a multi-modal spatial-temporal feature coding layer, constructing a process association graph through a dynamic reasoning layer based on causal reasoning, and calculating a propagation influence degree; carrying out hierarchical precision detection on different defect evolution probability regions by a prediction perception detection layer with a self-adaptive threshold value, and outputting a defect detection result; and classifying defect detection results based on defect classification standards.

The invention discloses a Transform-based cross-modal fusion multi-modal emotion recognition method and device, which are used for solving the problems of modal isomerism, difficulty in time alignment and insufficient dynamic emotion modeling in a multi-modal emotion recognition task, and the method takes the accuracy and robustness of emotion recognition as performance evaluation indexes. Firstly, feature information of three modes of vision, voice and text is obtained, feature extraction is performed on each mode through a deep learning model, then features of different modes are fused by using a cross-mode Transform module, and a complex dependency relationship between the modes is dynamically modeled through a multi-head self-attention mechanism, so that more accurate emotion recognition is realized, and the emotion recognition efficiency is improved. And finally, performing emotion prediction on the fused features based on time sequence modeling and an emotion classification module. According to the method, the problems of modal isomerism, difficulty in time alignment and insufficient dynamic emotion modeling in multi-modal emotion recognition can be effectively solved.

The invention provides a cross-modal retrieval method for semantic and vector fusion in a data space, which belongs to the field of cross-modal information retrieval, and comprises the following steps: firstly, collecting and preprocessing multi-modal data; generating modal embedding and storing by utilizing the pre-training model; a shared semantic space is constructed, cross-modal vector alignment is optimized through comparative learning, and a modal mapping network is designed to enhance the embedding projection effect; storing the aligned embedding by using a Milvus database, and constructing an HNSW index; user text or image query is processed, text query analyzes limiting conditions to generate enhanced embedding, and image query extracts characters through OCR and fuses the characters with image features to generate embedding; in a database, through condition screening and semantic similarity calculation, a Top-K candidate item is retrieved; performing multi-modal correlation sorting on the candidate results and returning the results; according to the method, the shared semantic space is constructed, the alignment effect of different modal embedding is optimized, efficient storage and index management of multi-modal embedding are carried out, and real-time retrieval of large-scale cross-modal data is achieved.

According to the research and development document processing method and device provided by the embodiment of the invention, unified processing of text, voice and image information is realized by constructing the multi-mode document analysis engine. Different types of data are converted into standardized feature representations through cross-modal preprocessing and a semantic alignment network. The system adopts a deep learning model to identify research and development elements, establishes a multi-modal relation graph, and realizes intelligent extraction and correlation analysis of various types of information in research and development documents. According to the method, the defects of the traditional technology in the aspects of multi-modal information fusion and knowledge system construction are effectively overcome, powerful support is provided for research and development process management and knowledge asset accumulation, and the standardized management level of research and development documents is remarkably improved.

The invention belongs to the technical field of intelligent learning systems, and particularly relates to a generative and multi-modal perception integrated intelligent agent learning system, which comprises the following steps of: establishing a dynamic knowledge graph of a learner by collecting various daily learning data, physiological indexes and basic data of the learner; the teaching strategy planning module is used for establishing a teaching strategy plan of a learner based on a dynamic knowledge graph of the learner, evaluating a specified index of the learner and a specified index of the system after the learner executes a specified time period based on the teaching strategy plan, and optimizing the teaching strategy planning module based on an evaluation result of the specified index of the multi-scale evaluation module. The teaching strategy planning of the learner is optimized, and the modules are coordinated and optimized based on the evaluation result of the specified index of the system. The technical problems that an online education system in the prior art is low in learning efficiency, insufficient in suggestion correlation, not integrated with a teaching feedback mechanism, incapable of reflecting real engineering capability and lack of a privacy protection mechanism are solved.

The invention provides an industrial robot autonomous collaborative decision-making method and system based on multi-modal sensing and a medium, and belongs to the technical field of industrial robot intelligent control. The method comprises the steps of performing cross-modal space-time alignment to eliminate data space-time differences by collecting visual, tactile and auditory information, realizing multi-modal feature fusion in combination with a dynamic weight adjustment mechanism and an attention calculation model, and generating a joint decision strategy through a deep learning optimization model. The system dynamically allocates sensor weights according to task types, introduces a multi-objective optimization mechanism of energy consumption, precision and safety, and sets a fault-tolerant rule to automatically recover the weights or recalibrate the sensors. According to the method, the problems of rigid data fusion and single optimization dimension in traditional multi-modal decision making are solved, and the precision, the response speed and the environmental adaptability of collaborative operation of the industrial robot are remarkably improved.