11887 results about "Human machine interaction" patented technology

The invention relates to the technical field of multi-modal information processing, and provides a deep semantic collaborative fusion method for heterogeneous multi-modal data. The invention provides a dynamic adaptive fusion framework aiming at the problems that a modal interaction mechanism is rigid and semantic modeling is shallow in the prior art. The method comprises the following steps: carrying out feature coding and alignment on text, audio and video modal data to generate unified-dimension single-modal representation; dynamic interaction is realized through an enhanced multi-head gating fusion module, and double-path features are generated; and carrying out cross-modal depth modeling on the basis of a stacked Transform encoder, and outputting final fusion semantics. Wherein the multi-head attention path calculates cross-modal mapping by taking a text as a query vector and taking an audio / video as a key value vector; the gating path generates a dynamic weight through cosine similarity and a learnable temperature parameter; and the dual-path adaptive fusion adopts a balance factor alpha weighted combination. According to the method, the multi-modal data fusion precision and the system robustness are improved, and the method is suitable for government affair service, man-machine interaction and other scenes.

The invention discloses a cloud edge collaborative multi-source data fusion safety management and control system for intelligent power distribution equipment, relates to the technical field of intelligent power grids and power distribution automation, and is used for managing and controlling 10-35 kV power distribution equipment. The system comprises an edge data acquisition module, a preprocessing module, a cloud storage management module, a cloud edge collaborative scheduling module, a multi-source data fusion module, an intelligent risk assessment module, a safety control execution module, a safety protection module and a man-machine interaction module. The modules are interacted through a 5G / industrial Ethernet; the acquisition module obtains multiple parameters, the preprocessing module cleans standardized data, the cloud side performs hierarchical storage, the scheduling module allocates tasks, the fusion module integrates data, the evaluation module performs grading risk, the protection module guarantees safety, and the interaction module performs visual alarm. The method improves the power distribution data quality and risk assessment precision, optimizes the cloud edge cooperation efficiency, enhances the safety protection capability, achieves the preventive operation and maintenance of equipment, reduces the fault and power failure time, reduces the operation and maintenance cost, and provides support for the safe and efficient operation of a power distribution network.

The invention relates to a tower crane operation control system based on complex scene three-dimensional real-time modeling. According to the system, a lifting hook is coarsely positioned through a lifting hook positioning and state sensing unit, a real-time position is positioned by combining a laser radar point cloud clustering algorithm with historical pose data, and visual tracking is synchronously performed by means of a tower top camera AI; converting the real-time point cloud data into a 3D voxel grid map, generating a global path by using a 3DA algorithm, and outputting a hoisting track after smooth processing and track optimization; establishing a sling-lifting hook double-pendulum dynamic model, predicting a state sequence based on a model prediction control algorithm, and adjusting a control signal through a feedforward compensation item and a feedback correction item; and the man-machine interaction and monitoring unit is used for displaying the cantilever angle, the lifting hook height and the three-dimensional map of the tower crane in real time and remotely intervening the operation state of the tower crane. According to the system, multi-source data are fused to construct a high-precision three-dimensional map, lifting hook positioning and full-view tracking are achieved, and lifting safety and trajectory tracking precision are improved through path planning and dynamics control.

The invention discloses a mechanical arm natural language instruction control system and method based on a large language model, and belongs to the field of intelligent manufacturing. Aiming at the limitation that traditional mechanical arm control depends on pre-programming and a static rule library, a dynamic mapping mode from a natural language instruction to an atomic action sequence is designed, an atomic skill library including detection, grabbing, moving, placement and other operations is constructed, and semantic analysis and a multi-mode cooperation technology are combined, so that the atomic action sequence is obtained. And support is provided for man-machine cooperation of a flexible assembly task. The method specifically comprises the steps that a DeepSeek-Distil-Llam-8B large model and a LoRA fine tuning technology are adopted, and a natural language instruction is converted into an executable atomic action sequence; based on a transfer learning optimized YOLOv8 target detection technology and a binocular vision positioning technology, a sensing module adaptive to an assembly scene is constructed and is fused with a mechanical arm motion planning module, and positioning grabbing of parts and tools is achieved. And an interactive interface is built by combining a voice-to-text large model and a Gradio front-end framework, so that the convenience of man-machine interaction is improved. By optimizing large model reasoning and motion planning cooperation efficiency, response delay from instructions to execution is reduced, and an efficient and extensible solution is provided for man-machine cooperation in intelligent manufacturing.

The invention relates to the field of artificial intelligence, particularly discloses an intelligent optimal management system for urban traffic congestion based on artificial intelligence, and aims to solve the problems of congestion and low efficiency caused by response delay, local optimization and low data utilization rate of an existing traffic management system. The system comprises a data acquisition and fusion module, a traffic state perception and prediction module, a decision optimization module, an instruction issuing and execution module and a man-machine interaction and visualization module. Through multi-source data fusion, graph neural network prediction and multi-agent reinforcement learning, traffic flow real-time perception, accurate prediction and adaptive control are realized, congestion is effectively relieved, and the overall operation efficiency and toughness of a road network are improved.

The invention discloses a building robot multi-machine collaborative operation optimizing and monitoring method and system based on digital twinning, and belongs to the technical field of building construction intellectualization. The method comprises the following steps: constructing a digital twinborn body comprising an environment perception model and a kinematics / dynamics model based on a building information model and robot physical parameters; performing multi-machine task allocation and path planning containing static / dynamic obstacle avoidance in the virtual environment; synchronizing physical environment data in real time through a multi-mode sensor; dynamically optimizing operation parameters by adopting a genetic algorithm or a particle swarm algorithm and realizing closed-loop control; and generating a safety early warning and emergency scheme based on machine learning. According to the method, Markov decision path planning of reinforcement learning and multi-agent game task allocation are creatively fused, laser radar-vision-inertial navigation multi-source data fusion is adopted, the technical problems that in a traditional method, digital twinning precision is insufficient, and dynamic cooperation efficiency is low are solved, and the method is suitable for large-scale popularization and application. And the construction efficiency, the safety and the man-machine interaction experience are remarkably improved.

The invention discloses a virtual historical character dialogue method and system with role knowledge and context awareness, and relates to the technical field of man-machine interaction, and the method comprises the steps: constructing a multi-level role depth model; when a question of a current user is received, identifying information of a virtual scene where the current user is located, analyzing micro-expressions of the face of the user and voice rhythm characteristics of speech of the user, analyzing an emotional state and an interaction intention of the user based on a multi-modal fusion algorithm, and generating a user state vector; executing a dynamic Prompt construction program, extracting related information from the multi-level role depth model and the user state vector, and generating a structured Prompt; and inputting the structured Prompt into a large language model, generating a reply text conforming to role features based on questions of the current user, and driving a virtual character model. The method solves the problem that in the prior art, virtual historical figures cannot provide real immersion and credible interactive experience with emotional connection.

The invention relates to the technical field of natural language processing and artificial intelligence, and discloses a silk path knowledge base intelligent question answering system based on a large language model, comprising a silk path data acquisition module used for acquiring multi-source silk path knowledge data; the silk path knowledge base construction module is used for processing the multi-source silk path knowledge data to construct a structured knowledge base and an unstructured knowledge base; the data storage system module is used for storing a structured knowledge base and an unstructured knowledge base and providing a structured retrieval interface and an unstructured retrieval interface; and the man-machine interaction module is used for identifying a user intention according to the user question and outputting an answer corresponding to the user question by adopting a corresponding answer generation mode based on the user intention. According to the method, efficient and accurate silk path knowledge questions and answers are realized by constructing the multi-modal knowledge base and optimizing a retrieval algorithm and a prompt engineering technology.

The invention belongs to the technical field of logistics conveying, particularly relates to an automatic material conveying system for a door and window processing workshop, and aims to solve the problem of low efficiency caused by insufficient automation, flexibility and intelligence of an existing system. The system comprises a data acquisition and perception module, a digital twinning module, a task management and scheduling module, an intelligent path planning and obstacle avoidance module, a multi-agent cooperative control module, an execution and feedback module, a man-machine interaction and visualization module and the like. And multi-mode collaborative operation, real-time path optimization and dynamic task allocation of the material conveying equipment are realized. The production efficiency, the flexibility and the intelligent level of a door and window processing workshop are remarkably improved.

The invention discloses an industrial internet of things real-time monitoring and predictive maintenance system based on digital twinning, and relates to the technical field of industrial digital twinning operation and maintenance, the system comprises a multi-modal data acquisition module, a sensor network is deployed, and edge calculation preprocessing is carried out; the digital twinning construction module is used for constructing a high-precision model by fusing a physical law and deep learning; the real-time monitoring module is used for detecting abnormity by using a space-time diagram neural network; the predictive maintenance module is used for optimizing a maintenance strategy in combination with a probabilistic algorithm; and the man-machine interaction module supports AR / VR and brain-computer interface operation. In addition, the system integrates functions of block chain security, energy management and the like, and realizes full-life-cycle intelligent management of equipment. The operation and maintenance efficiency of the industrial equipment is greatly improved. The data acquisition precision reaches the nanoscale, and the early warning time is advanced to 72 hours; the maintenance cost is reduced, and the equipment availability is improved; the AR interaction enables the operation efficiency to be improved and the training period to be shortened. And meanwhile, energy consumption reduction is realized.

The invention discloses a bridge management and maintenance decision-making system and method based on multi-agent collaborative optimization, and the system comprises a monitoring agent which is deployed in a cloud server and is used for obtaining abnormal data in a bridge structure and environment data; the diagnosis agent is used for evaluating the health state of the bridge by utilizing a built-in knowledge base, a built-in machine learning model and a built-in physical model based on the abnormal data; the decision-making agent is used for generating a plurality of candidate maintenance schemes based on an evaluation result, and screening out an optimal scheme by integrating the comprehensive utility of each scheme and the resource matching degree score of the resource scheduling agent on each scheme; the resource scheduling agent generates a construction plan according to the optimal scheme; the coordination / communication agent is used for ensuring efficient cooperation among the agents through a communication protocol and a negotiation mechanism among the agents; the diagnosis report, the optimal scheme and the construction plan are integrated and presented to a bridge manager through a human-computer interaction interface; and collaborative optimization of bridge management and maintenance decisions is realized through continuous feedback and self-learning.

The invention belongs to the technical field of iron phosphate preparation, and discloses an energy-saving control system for iron phosphate preparation based on an energy consumption scheduling model. The system is composed of a data acquisition module, an energy consumption sensing module, a preparation process modeling module, an energy consumption prediction module, an energy-saving scheduling module, an intelligent execution module, a feedback correction module, a man-machine interaction module and a remote operation and maintenance module. The energy consumption sensing module intelligently senses an energy consumption state, the preparation process modeling and energy consumption prediction module accurately predicts energy consumption, the energy-saving scheduling module generates an optimal scheduling strategy, the intelligent execution module accurately executes an instruction, and the feedback correction module realizes closed-loop adaptive regulation and control; all the modules cooperatively operate, process parameters are adjusted in real time according to actual working conditions of iron phosphate preparation, energy consumption in the preparation process is remarkably reduced, the energy utilization rate is increased, and energy-saving optimization of iron phosphate preparation is achieved.

The invention belongs to the technical field of nerve regulation and control, and discloses a portable noninvasive brain deep electrical stimulation system based on time interference, and the system comprises an electroencephalogram signal collection unit which is used for obtaining an electroencephalogram signal in a high-quality resting state, and carrying out the signal preprocessing and feature extraction, thereby obtaining an optimal stimulation envelope frequency; the personalized finite element modeling unit is used for constructing a head personalized model and outputting optimal electrode configuration and stimulation parameters for a target brain region target spot; the electrode positioning and stimulation execution unit is used for realizing accurate positioning of an electrode position and implementing time interference electrical stimulation; the MCU unit is used for realizing data management, task scheduling and remote communication; the human-computer interaction unit is used for realizing visual treatment flow and parameter setting; and the power supply management unit provides stable working voltage for the system. According to the invention, the portable, intelligent and personalized modeling of the electrical stimulation treatment equipment is realized, and the precision and efficiency of nerve regulation and control are remarkably improved.

The invention relates to an automatic assembly system and method for multi-scale digital twin components, and the system comprises a semantic relationship construction module which is used for carrying out explicit definition on the structural features, functional attributes, spatial layout requirements and logic dependency relationships of the multi-scale components, and constructing semantic relationships among the three types of components; the semantic reasoning and constraint engine module is used for carrying out logical reasoning through the semantic relationship constructed by the semantic relationship construction module and judging whether the component combination meets the assembly constraint or not; the assembly generation and configuration module is used for generating an assembly topological structure and a connection sequence of a system shelf according to the component candidate set output by the semantic reasoning and constraint engine module; and the man-machine interaction and visualization module is used for supporting a feedback closed loop between the engineer and the system and providing a visual display interface. The problems that an assembly method depends on artificial experience and is difficult to support high-frequency and multi-scene production line reconstruction are solved, and the method has higher semantic interpretation capacity, automatic combination capacity and context adaptive capacity.

The invention discloses a multi-modal data fusion-based intelligent management system for a thoracic surgery monitoring unit, belongs to the technical field of medical information and artificial intelligence, and aims to solve the limitation of an existing thoracic surgery monitoring system in the aspects of multi-modal data fusion, heterogeneous data semantic alignment and intelligent deep analysis and prediction decision. The system is characterized by comprising a multi-modal data acquisition unit, a heterogeneous data fusion and semantic alignment module, an intelligent analysis and prediction decision module, a man-machine interaction and visual presentation module and a secure storage and management module. By the adoption of the technical scheme, comprehensive multi-modal data fusion, high real-time performance, deep intelligent analysis and prospective prediction can be achieved, intelligent decision support, resource optimization, continuous learning and self-adaptive optimization are provided, and the intelligent level and patient management efficiency of the thoracic surgery intensive care unit are remarkably improved.

The invention relates to a man-machine interaction dialogue method, system and device based on a natural language and a medium. The method comprises the steps that firstly, multi-modal interaction data is acquired and preprocessed, and segmented words and syntax are analyzed through a natural language processing technology to construct intention feature vectors; combining the intention feature vector with a historical dialogue record, and using a pre-trained language model to generate context semantic elements containing a semantic relationship; if the context semantic elements are matched with the preset scene feature library information, predicting a user intention change trend by adopting a reinforcement learning model to obtain an intention prediction result; and finally, evaluating user intention change based on an intention prediction result, extracting associated domain knowledge by utilizing a knowledge graph if significant change occurs, and inputting the associated domain knowledge into a dialogue generation model to obtain a natural language reply sequence. According to the method, the understanding precision of the user intention is improved, the dynamic prediction of the intention change is realized, the relativity and coherence of reply are guaranteed, and a more efficient processing path is provided for natural language man-machine interaction.

The invention relates to the technical field of artificial intelligence, in particular to a multi-modal emotion recognition and interaction adjusting system and method based on uncertainty evaluation, and solves the defects of uncertainty processing, robustness of interaction strategies, complementary information mining degree among modals and the like in the man-machine interaction process in the prior art. Depth application finiteness is caused by lack of modeling for feature uncertainty after fusion. The uncertainty of an emotion recognition result is quantified through technologies such as multi-modal feature fusion and Bayesian neural network / model integration, an interaction strategy is dynamically adjusted according to an uncertainty score, emotion clarification or conservative response is triggered in a high-uncertainty scene, the robustness of human-computer interaction and the user experience are improved, and the user experience is improved. The method is suitable for intelligent customer service, government affair consultation, medical inquiry and other scenes with high requirements for emotion interaction accuracy.

The invention belongs to the technical field of industrial automation and artificial intelligence crossing, and particularly relates to an AI industrial control intelligent master system and method based on deep learning. The system comprises an intelligent programming module, a real-time monitoring module, a fault management module, an optimization decision module and a man-machine interaction module. The system automatically generates a PLC program code through a deep learning model, monitors an equipment operation state through a multi-dimensional anomaly detection technology, analyzes a fault mode and a propagation path by using a knowledge graph and a graph neural network, optimizes process parameters and a maintenance strategy based on digital twinning and reinforcement learning, provides a 3D virtual interface and an AR remote cooperation function, and improves the reliability of the system. And intelligent management of the whole life cycle of the industrial control system is realized. The PLC programming efficiency is greatly improved, accurate anomaly detection and fault prediction are realized, system operation parameters are optimized, the service life of equipment is prolonged, the man-machine interaction experience is improved, and the problem of unified management of multiple brands of PLCs is solved.

The invention relates to the technical field of wireless communication, in particular to a production line wireless communication system and method. The system comprises a multi-dimensional feature sensing module, an interference intelligent cognition and decision-making module, a communication opportunity decision-making module and a channel resource arrangement module. An expected action sequence, transient current features and spatial position topology of an interference source are collected through a multi-dimensional feature sensing module, and an interference source multi-dimensional feature set is generated through fusion; the interference intelligent cognition and decision-making module performs space-time spectrum joint modeling of an interference field based on the feature set, generates a workshop electromagnetic environment situation map, constructs a whole network link toughness map in parallel, and generates a conflict-free communication scheduling table through fusion analysis and multi-dimensional cost optimization; the communication opportunity decision module converts the dispatch table into an executable instruction; and the channel resource arrangement module executes the instruction and realizes closed-loop feedback. According to the method, the problem of communication uncertainty caused by industrial electromagnetic interference is solved, and reliable transmission of key services such as automatic guided vehicle scheduling and man-machine interaction is ensured.

The invention discloses a knowledge model data management system based on artificial intelligence, and relates to the field of data management, and the system comprises the following components: a data collection and storage module, an intelligent prediction and analysis module, an abnormal comprehensive processing module, a knowledge graph and traceability module and a man-machine interaction management module. According to the invention, the intelligent prediction analysis module is combined with a time sequence prediction model and a causal reasoning algorithm, abnormal fluctuation and causal association thereof in data can be accurately judged, the accuracy of anomaly detection is effectively improved, and meanwhile, the knowledge graph and traceability module uses the constructed knowledge graph and reinforcement learning algorithm to improve the accuracy of anomaly detection. According to the method, the relation chain can be quickly traced from the abnormal data, the problem source can be positioned, the abnormal traceability efficiency is remarkably improved, and the functions act together, so that the system can more quickly and accurately discover and process problems when facing a complex data environment, and the stability and reliability of data management are guaranteed.

A computer implemented system and method for generating offer ranges for candidates in an interviewing process is disclosed. The system generates an AI-based interviewer simulating human-based interactions for conducting an ongoing interview with candidates. The system analyzes data associated with candidates obtained during ongoing interview. The system process analyzed responses of candidates to determine contextual attributes associated with responses using ML models. The system automatically generates follow-up interview questions to be delivered to candidates during ongoing interview based on analyzed responses from candidates, by applying AI model to contextual attributes associated with responses. The system generates recruitment scores for candidates based on analyzed responses, contextual attributes, and interpreted non-verbal cues, associated with candidates, using AI model. The system generates offer ranges for candidates based on recruitment scores using AI model. The system provides information associated with selected candidates, and offer ranges generated for selected candidates, to users.

The invention provides a user interface generation method, electronic equipment, a vehicle and a storage medium, and relates to the field of artificial intelligence and large language model application, and the method comprises the following steps: constructing a cue word according to a natural language description input by a user, inputting the cue word into a preset large language model, controlling the large language model to perform knowledge base query analysis based on the cue word; querying the target knowledge base based on a query decision output by the large language model, and returning a knowledge base query result to the large language model, so that the large language model generates a user interface description for the cue word based on the knowledge base query result; a user interface is generated in the electronic device in accordance with the user interface description. A large language model and knowledge base collaborative interface generation mechanism is introduced, by means of real-time knowledge base query, it is avoided that the model generates a non-compliant interface due to illusion, it is ensured that the generated interface only contains functions and components actually supported by electronic equipment, and the intelligent level of man-machine interaction is improved.

The invention relates to the technical field of supervision control and data acquisition, in particular to an intelligent monitoring system based on a multi-sensor fusion technology, comprising a multi-sensor acquisition and preprocessing module which acquires and self-calibrates at least two kinds of physical quantity data in real time and fuses the physical quantity data into a unified monitoring feature vector; and the fault diagnosis and prediction module is used for integrating interpretable semantic analysis in combination with a production plan and a maintenance record. And performing equipment state diagnosis, operation trend analysis and fault prediction in real time by utilizing electric inspection type monitoring, generating health assessment and early warning, and guiding optimization and adjustment of a control strategy. And the data storage module is used for storing semantic data model evaluation results, diagnosis reports and early warning information, and supporting fault root analysis and control effect evaluation based on processing batches and time sequences. And the remote man-machine interaction interface is used for displaying the equipment running state, the control parameters and the automatic process in real time, supporting voice instruction interaction and facilitating remote monitoring and intervention of an operator.

The invention belongs to the technical field of charging detection, and particularly relates to a charging pile extreme environment test method based on composite stress simulation, a terminal and a storage medium, and the method comprises the steps: inputting an identifier corresponding to a target test scene through a human-computer interaction interface of a test cabin, and executing a query operation based on the input identifier, retrieving a preset parameter combination from a preset scenarized stress library by using the mapping function; in a closed test cabin, environment parameter data are collected in real time through a distributed sensor array and fed back to a central controller, and the central controller drives an execution component to cooperatively regulate and control parameters such as temperature and humidity, so that the environment parameters are stabilized in a preset range; a sequential stress scheme is generated based on a pre-stored dynamic loading algorithm, an execution component is controlled to realize stress superposition or stress alternation, and a charging load is synchronously adjusted to simulate a composite stress effect under an actual working condition; simulation of multi-factor combined stress and dynamic loading is realized, an actual use fault can be found in advance, and the matching degree of a test result and a real environment is remarkably improved.

The invention discloses a truck-mounted crane group operation safety monitoring system and method based on multi-source perception and collaborative decision, and particularly relates to the field of safety monitoring, the truck-mounted crane group operation safety monitoring system comprises a distributed perception module, an edge calculation module, a graded early warning and execution module, a digital twin platform module and a man-machine interaction module; collected multi-source data are fused through Kalman filtering and then transmitted to an edge computing node; generating a dynamic safety envelope region based on an adaptive kinematics model, predicting a group conflict risk through a graph theory algorithm, and calculating collision time; the digital twin platform fuses a BIM model and real-time data, hoisting path rehearsal, load swing prediction and foundation bearing capacity visualization are carried out, an improved RRT * algorithm is adopted to plan a collision-free path, and a Lagrange dynamics model is adopted to predict a swing track; the grading early warning module triggers differential response according to the risk grade; and risk warning and path guidance are provided through the AR-HUD and the three-dimensional situation billboard.

The invention discloses a building heating, ventilating and air conditioning control method with dynamic energy efficiency optimization, and belongs to the technical field of heating, ventilating and air conditioning control. The method comprises the following steps: step 1, acquiring temperature, humidity, carbon dioxide concentration and personnel distribution data in real time, and performing preliminary data processing through an edge computing node; step 2, realizing accurate prediction of future load change of the building; step 3, generating an optimal control strategy by using a reinforcement learning algorithm, and realizing dynamic optimization control of equipment operation; 4, the compliance of equipment operation, network communication and control instructions is monitored in real time, and a closed-loop dynamic tuning mechanism is constructed to continuously optimize control strategy parameters; step 5, dividing a control task into a group control layer, a region layer and a terminal layer, and establishing communication among the layers through a standardized interface so as to realize accurate transmission and execution of an instruction; and step 6, designing a friendly human-computer interaction interface, displaying a building operation state, energy consumption analysis and a control strategy, and supporting multi-terminal access and remote monitoring.

The invention relates to the technical field of data processing, discloses a digital twin adaptive decision platform for TBM tunneling and a control method, and aims to solve the problems of single sensing dimension, mutual separation of physical and data models, disjunction of early warning and control links and low man-machine interaction efficiency in the prior art. According to the scheme, the method mainly comprises the steps that multi-source real-time data is collected through a hardware sensing layer, after the multi-source real-time data is processed through a data management and fusion layer, a digital twin model layer constructs a physical-data fusion model and completes risk assessment, finally risk visualization is achieved through an augmented reality interaction layer, and an intelligent control decision-making layer generates a control instruction and outputs the control instruction to a TBM main control system. According to the integrated intelligent decision-making platform and the control method, multi-source perception deep fusion, physical and data model online coupling, risk assessment and equipment control second-level closed loop can be realized, and visual interaction can be realized through augmented reality.

The invention discloses a ship state monitoring system and method based on multi-source data, and relates to the technical field of ship engineering, the system comprises a bionic flexible sensing module, a multi-source data acquisition module, an anti-interference communication module, a multi-source data fusion module, an edge calculation processing module, an intelligent fault diagnosis module and a man-machine interaction and decision support module; wherein the bionic flexible sensing module acquires broadband vibration signals, the multi-source data acquisition module accesses multi-source data, the anti-interference communication module realizes data transmission based on a metamaterial antenna and the like, the multi-source data fusion module generates a standardized data set, and the edge calculation processing module completes feature extraction and abnormity pre-screening. The intelligent fault diagnosis module locates a fault and generates a report, and the man-machine interaction and decision support module visualizes data; the bottleneck of curved surface detection is broken through, data transmission is guaranteed, and accurate diagnosis is realized.

The invention relates to the technical field of man-machine interaction, in particular to a dialogue Agent interaction method based on multi-modal intention understanding, which comprises the following steps: S1, collecting multi-modal data in a user interaction process in real time, and calculating a time synchronization deviation value of each modal data source; s2, constructing a space-time fusion feature vector; s3, analyzing a dominant action instruction and a recessive behavior clue in the space-time fusion feature vector; s4, generating a multi-level intention analysis tree; s5, when the corrected confidence coefficient of any node in the intention analysis tree is lower than a set threshold value, activating a targeted sensor to complementarily collect data; and S6, analyzing a tree drive response decision according to the finally confirmed intention. According to the method, high-precision identification and response control of the dialogue Agent on the user intention in a complex scene are realized by constructing a multi-modal interaction method with space-time consistency fusion capability, an explicit and implicit intention analysis mechanism and an adaptive modal clarification strategy.