1129 results about "Feature code" patented technology

The invention relates to the technical field of water quality monitoring, in particular to an automatic water quality monitoring method and system.The method comprises the steps that multiple pieces of collected water quality monitoring data are combined pairwise, dynamic coupling strength is calculated, a topological network atlas is generated, and automatic extraction and structural characterization of the dynamic coupling relation among complex water quality parameters are achieved; the limitation of dependence on manual feature recognition traditionally is overcome; secondly, matching the topological network atlas with a preset pollution mode feature library, dynamically determining a newly added abnormal mode, and outputting an abnormal feature code set, thereby solving the key defect that a static model cannot recognize an unknown pollution mode; and finally, a water quality monitoring and early warning signal is output by fusing the pollution diffusion prediction result and the abnormal feature code set, so that bidirectional verification of data driving and a mechanism model is realized, and the early warning accuracy of a water quality abnormal phenomenon is remarkably improved.

The invention provides a cross-department data collaborative modeling method and system based on federated learning in the cross technical field of federated learning and privacy computing, and the method comprises the steps: S1, carrying out desensitization operation on department data by each edge computing node, and coding the desensitization data to obtain a feature coding vector; s2, constructing a layered federated learning framework based on a department private feature extractor, a cross-department parameter collaborative aggregation layer and a global task header; s3, extracting data features from the feature coding vectors through a department private feature extractor so as to train the model and extract model parameters; s4, performing hierarchical quantization coding, encryption and compression on the model parameters to obtain compression parameters, and sharing the compression parameters by a cross-department parameter collaborative aggregation layer; and S5, the global task head reads the shared compression parameters to carry out collaborative modeling on the model. The cross-department data collaborative modeling method has the advantages that the precision and efficiency of cross-department data collaborative modeling are greatly improved on the premise that the safety is guaranteed.

The invention discloses an MES work order dynamic scheduling and resource self-adaption system based on real-time event triggering, and relates to the technical field of intelligent manufacturing and industrial automation, equipment states and material events are monitored in real time through interfaces such as a PLC and an OPCUA, event types and validity are analyzed through hardware filtering, feature code recognition and double verification, and the real-time event triggering is realized. Event data, MES batch information and equipment load data are integrated to generate standardized scheduling input, a preset strategy is matched, an optimal dispatching scheme is generated through a priority algorithm, material, equipment and logistics resource allocation and dispatching operation are completed, synchronization and MES interaction ensure that page logic is consistent, full-process automation of dynamic work order scheduling is achieved, and work order scheduling efficiency is improved. The equipment state and the material event are responded in real time, the production cooperation efficiency is improved by more than 30%, the resource adaptation accuracy rate reaches 95% or above, manual intervention is reduced, and the consistency of data and MES is guaranteed.

The invention discloses a feature attention and bilinear gating fused speech emotion recognition method and device, and the method comprises the following steps: 1, collecting an audio file, obtaining corresponding label information, generating audio waveform and time frequency representation data through preprocessing, and constructing an audio waveform mask and a time frequency mask to mark an effective information region; 2, constructing a dual-path feature extraction module which comprises a time-frequency feature extraction module and a pre-training acoustic feature coding module; wherein the time-frequency feature extraction module models emotion correlation through local convolution and a multi-dimensional attention mechanism, and performs global time sequence modeling based on a bidirectional gated loop network; the pre-training acoustic feature coding module extracts high-level speech representation with high expression ability for emotion distinguishing by using a pre-training model; and step 3, constructing a feature fusion module and an emotion classification module, and combining with a dual-path feature extraction module to form a speech emotion recognition model.

The invention relates to the technical field of network communication, and discloses a vehicle-mounted edge computing data recording system and method based on protocol adaptive analysis, and the system comprises a multi-protocol network access controller which is used for capturing an original data frame and extracting a physical feature triple; the identification analysis engine is used for executing hash operation on the triple to generate a physical feature code and retrieving a physical logic address mapping table; the direct memory access controller is used for responding to a target memory address pointer hit by retrieval and directly writing a data load into an input buffer area of the functional operation module, and by constructing a direct addressing mechanism based on Hash mapping, thorough decoupling of vehicle-mounted heterogeneous network physical topology and edge computing logic is achieved.

The invention discloses an equipment fault early warning system and method based on the Internet of Things, and relates to the technical field of equipment fault early warning, and the method comprises the steps: collecting mechanical operation parameters and production process parameters of equipment, and setting each piece of equipment as an independent intelligent agent; establishing a three-level topological structure; based on the dual-threshold model, judging the running state of the equipment; when the equipment intelligent body detects that the equipment is abnormal, the equipment intelligent body immediately marks the state of the equipment intelligent body as abnormal and generates a feature code; the equipment intelligent agent receiving the feature code analyzes associated parameters in a process associated parameter atlas, and sends an analysis detection result to an adjacent equipment intelligent agent and a central management node; the central management node summarizes analysis results of all the equipment intelligent agents and judges the influence of faults on the fly ash recycling production process; and according to the severity and the influence range of the fault, performing graded early warning, and sending early warning information.

The invention discloses a contract filing method, system and device based on automatic recognition and a medium, and the method specifically comprises the steps: taking a contract document uploaded by a user as input, and extracting contract key data; inputting the signature area coordinate, the signature date metadata and the clause keyword into a space-time correlation model to generate a context enhanced candidate contract number set; inputting the candidate contract number set into a preset knowledge graph, and outputting a target contract number; inputting the distribution characteristics of the clause keywords into a bidirectional long-short-term memory network classifier for processing, and constructing a multi-dimensional metadata matrix in combination with the target contract number; and inputting the desensitized contract text content and the multi-dimensional metadata matrix into a file DNA atlas generator, and generating a unique feature code by fusing a semantic embedding vector and a document structure fingerprint. The efficiency bottleneck and accuracy problems of a traditional archiving mode are solved, and the efficiency, accuracy and compliance of contract archiving are remarkably improved.

The invention provides an oil and gas pipeline defect three-dimensional contour determination method and device. Acquiring a three-axis magnetic flux leakage detection signal of a to-be-detected target oil and gas pipeline and corresponding space coordinate information of the three-axis magnetic flux leakage detection signal; constructing multi-channel input data according to the three-axis magnetic flux leakage detection signal and the space coordinate information; determining a defect contour prediction result of the target oil and gas pipeline according to the multi-channel input data by using a pre-trained defect contour inversion model; wherein the pre-trained defect contour inversion model comprises a multi-axis feature extraction and fusion module, a feature coding module and a multi-task decoding module; the defect contour prediction result is used for indicating whether the target oil and gas pipeline has defects or not, and the defect contour prediction result is further used for indicating the three-dimensional contour shape of the defects under the condition that the target oil and gas pipeline has the defects. Therefore, high-precision visual reconstruction of the defect position and the three-dimensional form of the oil and gas pipeline is realized, and the accuracy of defect identification and the reliability of evaluation are improved.

The invention discloses an equipment fault diagnosis method and system based on a large electric power model, and the method comprises the steps: collecting and fusing voltage and current waveform data and alarm log information in a short time window before and after a fault moment, achieving the context integration of a multi-dimensional feature vector through multi-level feature coding and semantic embedding, and achieving the fault diagnosis of a large electric power model. And relevant knowledge fragment retrieval is carried out in combination with a power field knowledge base. On the basis, relevant knowledge fragments, voltage-current waveform fusion feature coding vectors and fault alarm key information semantic embedding coding feature vectors are jointly embedded into a preset Prompt template and then are input into a diagnosis engine based on large model driving, so that intelligent analysis and report generation of fault types, reasons and key features are realized. Through the mode, the limitation of dependence on single data or rules traditionally is broken through, the accuracy, automation and interpretability of fault diagnosis under complex working conditions are remarkably improved, and a solid support is provided for intelligent and high-reliability equipment operation and maintenance.

The invention relates to the field of copyright authentication of non-perpetual culture elements, and discloses a block chain copyright authentication method for non-perpetual culture elements, which comprises the following steps: S1, performing multi-dimensional data acquisition through a distributed storage system, performing structured data acquisition and feature coding on the non-perpetual culture elements, and storing the structured data in a database; comprising the following steps: multi-modal data fusion acquisition, dynamic metadata standardization processing, and generation of unique digital identifiers and cultural feature codes; and S2, constructing a hybrid block chain architecture, constructing a hierarchical network architecture comprising a permission chain layer and an open chain layer, and realizing heterogeneous chain data synchronization through a cross-chain interoperation protocol. Through multi-modal data acquisition and dynamic metadata structured processing, the spatial form, the process time sequence and the oral context of non-residual elements are completely recorded, digital twin bodies with microscopic details and semantic association are formed, the problem of fragmentation of traditional archiving is solved, and a unique block chain identifier and two-way index mapping are combined, so that the method is simple and convenient to implement. And the originality of the cultural heritage in virtual restoration and cross-domain propagation is ensured.

The invention discloses a three-dimensional scene reconstruction method based on an intelligent LED street lamp multi-mode sensor. The three-dimensional scene reconstruction method comprises the following steps that RGB images are obtained and preprocessed; the information is input to a visual feature coding module, two-dimensional bounding box information is extracted, and an object segmentation module is guided to output a two-dimensional segmentation mask; obtaining point cloud data, projecting the point cloud data to the standardized RGB image, and screening target points in combination with the two-dimensional segmentation mask; complementing the preliminary segmentation result of the point cloud, mapping the result to a standardized RGB image, and extracting a pixel region; carrying out joint coding, implicit representation and neural decoding processing on the object-level RGB image and the point cloud complete segmentation result; and fusing into an original three-dimensional scene, and completing spatial restoration through point cloud registration, attitude optimization and semantic constraint. The invention provides an efficient three-dimensional scene reconstruction method in combination with a multi-mode sensor of an intelligent LED street lamp, and the method has high precision, real-time performance and dynamic target processing capability.

The invention belongs to the technical field of computers, and particularly relates to a software fault repairing method and system fused with intelligent analysis, which comprises the steps of collecting a multi-level running log and performing structured preprocessing, constructing a dynamic calling graph through a time sequence encoder and a graph neural network, inferring a fault root cause in combination with a Bayesian causal inference model, and repairing a fault fault according to the fault root cause. And matching the repair strategy to generate an atomization instruction sequence, and deploying the atomization instruction sequence to a production system after sandbox environment verification. The system comprises a log acquisition module, a feature coding module, a graph construction module, a causal reasoning module, a strategy matching module, an instruction generation module, a sandbox verification module, a deployment feedback module and the like. Through end-to-end intelligent analysis and a closed loop verification mechanism, the fault positioning precision and the repair safety are remarkably improved, system self-evolution is supported, and operation and maintenance are promoted to be transformed from passive response to active autonomy.

The invention provides a panoramic segmentation method and device in a complex scene and a storage medium, and the method comprises the steps: inputting an initial image into a panoramic segmentation network, and obtaining a panoramic segmentation result of the initial image; the panoramic segmentation network comprises: a backbone network; the feature coding layer comprises a DLK module and a DFF module which are cascaded, and the DLK module and the DFF module cooperate with each other to generate enhanced features E1 '-E4'; the edge guiding layer comprises an edge guiding fusion module and a feature pyramid, the EGF module fuses the C1 feature of the backbone network with the enhanced feature E4'to generate an edge enhanced feature F, and then performs channel splicing with E1 '-E3' to construct a four-level feature pyramid F1-F4; the dual-path decoding layer comprises a learnable query and a pixel updating path; the dual-path feature uses a cross attention mechanism to realize interaction, and a semantic category and an instance mask required by panoramic segmentation are output; the method can meet the adaptability in a complex scene and improve the segmentation quality.

The invention discloses an air-ground cooperation system and an air-ground cooperation method based on intelligence, and relates to the field of intelligent unmanned system control. The system comprises an intelligent unit, a multi-element sensing unit, a communication unit and a task execution unit. Based on a Vision-Language-Action (VLA) framework, an intelligent unit with a body realizes end-to-end analysis of a natural language instruction and a visual instruction, fuses sensing data of an unmanned aerial vehicle and an unmanned vehicle, generates a joint feature code with time-space alignment, and dynamically allocates a cooperative task. The communication unit supports low-delay end-to-end communication and real-time data synchronization between clusters, and ensures efficient interaction between agents. The multi-element sensing unit is integrated with a visual camera, a laser radar and high-precision positioning equipment, and air-ground environment multi-mode sensing is achieved. And the task execution unit supports the unmanned aerial vehicle to complete coordination actions in a complex scene. According to the scheme, the problems of insufficient multi-mode perception, limited natural language interaction and low dynamic cooperation efficiency of the existing system are solved, and the cooperation capability and system robustness of the unmanned aerial vehicle and the unmanned aerial vehicle in a complex environment are remarkably improved.

The invention provides a face image recognition method and system, and relates to the technical field of data processing, and the method comprises the steps: carrying out the fusion of a topological relation feature vector and a biological feature contour, calculating the curvature manifold of the concave region of five sense organs, and generating a local curvature feature vector through differential geometric transformation; inputting the topological relation feature vector, the local curvature feature vector and the micro feature coding sequence into an evolvable biological feature library, and fusing a space-time weight to update a comparison template; fusing the micro-feature coding sequence, the topological relation feature vector and the local curvature feature vector to obtain a biological feature set, and calculating the multi-scale similarity between the biological feature set and the updated comparison template; and if the multi-scale similarity exceeds an adaptive threshold, generating a face recognition result. According to the invention, the stability and accuracy of face recognition in a complex illumination environment are improved.

The invention discloses a document table extraction method and device, equipment and a medium, and relates to the technical field of computer information processing. The extraction method comprises the following steps: performing OCR (Optical Character Recognition) on a to-be-processed document table image to obtain a text block; performing visual feature coding on the document table image to obtain deep visual features; performing semantic feature coding on the text sequence of the text block to obtain a semantic feature vector; performing spatial feature coding on the bounding box of the text block to obtain a spatial feature vector; performing feature fusion processing on the deep visual features, the semantic feature vectors and the spatial feature vectors to obtain multi-modal guide features; and performing structured decoding processing on the multi-modal guide features to obtain structured representation of the table. According to the method, the text and the position information pre-recognized by the OCR are fused with the visual features of the document table, so that the visual features are guided to be expressed again and are actively aligned to the logic structure defined by the prior information, and the extraction accuracy of the table logic structure is improved.

The invention relates to the technical field of industrial robot control, and particularly discloses an intelligent foreign matter sorting robot control system based on AI recognition, which comprises a dynamic spatial feature extraction module, a manipulator motion state coding module, a collaborative conflict detection module, a dynamic trajectory optimization module and an execution control adjustment module, constructing a three-dimensional dynamic space model through multi-sensor fusion, and extracting spatial topological features by utilizing continuous coherence analysis; manipulator motion parameters are converted into topological space representation, and a track feature coding matrix is established; detecting interaction conflicts among the manipulators in real time by adopting a multi-scale coherence analysis method, and generating graded early warning signals; a collision avoidance track is optimized based on topological constraints and a virtual rejection field technology; precise execution is achieved through inverse kinematics of the Lie group theory and self-adaptive control.

The invention relates to the technical field of industrial monitoring, and discloses a factory real-time digital monitoring method and system based on the Internet of Things, and a storage medium. The method comprises the following steps: carrying out encryption access on heterogeneous Internet of Things equipment through dual identity authentication to obtain a security authentication equipment identifier pool; performing multi-dimensional data acquisition on the equipment identification pool based on Hash verification to obtain an encrypted time sequence data chain; performing dynamic identification on the factory visual image by adopting a confidence adaptive threshold to obtain a multi-task fusion feature code; performing heterogeneous fusion on the time sequence data chain and the feature code by using a sliding window weight distribution algorithm to obtain a factory digital state fingerprint; and performing knowledge graph reasoning on the state fingerprint through three-level early warning threshold judgment to obtain a real-time risk level early warning code. The technical problems that heterogeneous Internet of Things equipment cannot be uniformly accessed and authenticated, multi-source data lacks safety fusion processing, and the factory state cannot be intelligently reasoned and pre-warned are solved.

An apparatus comprises at least one processing device configured to select a test case associated with a test script comprising functional code for testing an information technology asset and a test case description. The at least one processing device is also configured to generate, utilizing a first machine learning model, first pseudocode for the test script based at least in part on the functional code of the test script, and to generate, utilizing a second machine learning model, second pseudocode for the test script based at least in part on the test case description. The at least one processing device is further configured to determine a stability of the test script based at least in part on a similarity between the first and second pseudocode, and to automatically control one or more characteristics of the functional code of the test script based at least in part on the determined stability.

The invention relates to the field of augmented reality, and provides a display system and method based on the augmented reality technology, and the method comprises the steps: firstly obtaining a virtual scene image and a real scene image, carrying out the semantic-level feature fusion of the scene images, synchronously obtaining the natural language description of the interaction intention of a user, and carrying out the semantic understanding; then, semantic-level feature coding representation of virtual and real scenes is extracted, multi-granularity semantic embedding of natural language description is combined, user intention representation is enhanced layer by layer by means of a recursive cross-modal attention mechanism, and finally an interaction picture highly matched with the context of the scene is generated. Thus, through a cross-modal hierarchical interaction recursion mechanism, the limitation of traditional single-modal intention recognition is broken through, deep coupling of dynamic scene semantics and user language intentions is realized, a new technical path is provided for high-immersion AR interaction, and the sense of participation of the user is improved.

The invention discloses a cross-domain privacy protection method and system for advertisement recommendation, equipment and a medium, and the method specifically comprises the steps: carrying out the encryption matching of user behavior data and anonymized equipment data, and generating an initial cross-domain joint feature vector; fusing the initial cross-domain joint feature vector and the disturbance feature vector to form a target cross-domain joint feature vector; splitting a pre-trained advertisement recommendation model into a feature coding sub-module and a reasoning sub-module, deploying the feature coding sub-module to a user adjacent edge node, and retaining the reasoning sub-module in a user local device; and based on the target cross-domain joint feature vector, performing calculation of the feature coding sub-module and calculation of the reasoning sub-module, and uploading the encrypted hidden layer feature vector to a federated learning aggregation server for global model updating. According to the method, cross-domain data utilization and user privacy protection in an advertisement recommendation process are realized, and effective feature vectors are generated for personalized advertisement recommendation while data are guaranteed not to be out of a domain.

The embodiment of the invention discloses a spinal metastatic tumor treatment scheme generation system. According to one specific embodiment, the system comprises a data processing server, an information fusion server and a scheme generation server which are in communication connection with one another, and the data processing server is used for preprocessing multi-source patient data to obtain standard multi-source patient data; the information fusion server is used for executing the following steps: performing feature code fusion on standard multi-source patient data to obtain a multi-source patient feature vector; performing feature enhancement on the multi-source patient feature vector to obtain a joint patient characterization vector; generating an initial therapeutic schedule result based on the joint patient characterization vector; and the scheme generation server is used for performing feature decision processing on the initial treatment scheme result to obtain a final treatment report. According to the embodiment, waste of computing resources can be reduced, and system response time can be shortened.

The invention discloses a printed circuit board withstand voltage detection and evaluation method, which comprises the following steps: in the whole withstand voltage detection process, aiming at different process batches and environments, acquiring electrical signals, process parameters, environment variables and multi-modal visual data, and carrying out time sequence synchronization marking and standardized preprocessing; extracting multi-source features by using a feature coding algorithm, and establishing a multi-mode distribution model covering signal mutation, visual defects and working condition anomalies; through cross-modal feature fusion and a time sequence causal inference model, an abnormal event causal relationship graph is automatically constructed, and a voltage withstanding failure root cause chain is identified and positioned; according to the method, the efficient structured management of the detection data, the accurate traceability of the abnormal reasons and the cross-batch analysis are realized, and the capability of diagnosing and preventing the withstand voltage abnormality of the printed circuit board is remarkably improved.

The invention provides an atmospheric precipitable water quantity inversion method and system based on multi-source satellite remote sensing data, and relates to the technical field of remote sensing data. After clear sky pixels are screened, radiation brightness temperature data, digital elevation model data and spatio-temporal feature coding data are processed through three feature extraction branches, obtained feature representations are spliced and input into a feature fusion network, a spatial attention module is introduced in the feature fusion stage, finally, a composite loss function is adopted for training, and the feature fusion network is constructed. And high-precision atmospheric precipitable water quantity inversion is realized. According to the method, the multi-source data characteristics are fully utilized, and the inversion precision and the space continuity are improved.

The invention provides a power marketing archive intelligent management method, which comprises the steps of receiving archive data through a power data acquisition interface, matching an electricity price feature code in an archive based on a real-time electricity price policy identifier, and injecting a version calibration mark into unmatched data; constructing an electricity consumption abnormity prediction model based on the user historical load mode and the environmental factors; when a query request is responded, dynamic penalty parameters are generated based on the user behavior portrait and the rule base; a quantum key distribution path is generated based on a power grid topological structure, and cascade encryption is implemented along physical connection nodes; dynamically associating user operation authority with a power grid operation risk signal, and automatically disabling core operation in a high-risk state; and analyzing archive data in a security isolation environment to drive line loss optimization, electric charge tracing and fault pre-judgment service flow in real time, and outputting feedback archive update based on the service. According to the invention, the safety, reliability and operation efficiency of power marketing archive management are improved.

The invention provides an emergency event identification method based on a neural network model and computer equipment, and the method comprises the steps: obtaining a multi-source perception data set covering a target region, carrying out the feature coding processing of the multi-source perception data set, obtaining a multi-dimensional feature representation set of perception data units, and carrying out the recognition of an emergency event. Calling a pre-trained emergency event recognition neural network model to perform event recognition processing on the multi-dimensional feature representation set, generating an event recognition result of the sensing data unit, and determining the type of an emergency event existing in the target area and spatial-temporal feature information of the emergency event in the sensing data unit according to the event recognition result; and based on the emergency event type and the time-space feature information, generating an emergency early warning instruction containing an event positioning identifier, and sending the emergency early warning instruction to the target emergency response terminal to start a response operation. According to the invention, the comprehensiveness and accuracy of emergency event identification are improved, and the conversion efficiency of the identification result to the actual response is enhanced, so that the timely early warning of the emergency event is effectively supported.

The invention relates to the technical field of gift packaging three-dimensional modeling, and discloses a construction method of a gift box packaging design three-dimensional model. The method comprises the following steps: collecting natural language description, a freehand sketch image and a physical material attribute parameter; analyzing a natural language to construct a design intention map, and analyzing a sketch to generate a geometric feature code; performing knowledge alignment fusion on the two to form enhanced design semantic representation; generating an initial three-dimensional design framework by using a generative design algorithm; material parameters are integrated, compatibility simulation test is carried out, parameter self-adaptive adjustment circulation is started according to a feasibility evaluation report, and component parameters are dynamically optimized; and rendering and outputting a final model. Through deep fusion of multi-modal design information and preposed dynamic optimization of physical attributes, the accuracy of design intention expression and the producibility of a three-dimensional model are improved, and automation and intelligentization of a design process are realized.

The invention relates to the technical field of wind power operation and maintenance management, and particularly provides an abnormal state detection method and system for a wind driven generator system. Joint feature extraction is carried out on the multi-source monitoring data flow through a feature coding network, a state evolution feature sequence and a cross-source association feature set are generated, a pre-trained anomaly diagnosis model is called to carry out fusion analysis on the features, and an anomaly diagnosis result containing anomaly confidence distribution information is generated; and after the abnormal type, the time dimension development track and the space dimension influence component identifier are analyzed, an equipment early warning instruction containing time and space positioning information is generated and sent to the target management terminal, so that the abnormal state of the wind driven generator system can be comprehensively and accurately detected, detailed abnormal information is provided in time, and the safety of the wind driven generator system is improved. And the equipment fault risk and the operation and maintenance cost are effectively reduced.

The invention relates to the technical field of data processing, and discloses a construction monitoring method and system based on artificial intelligence. The method comprises the following steps: collecting construction operation behaviors through a high-speed camera, and performing frequency division filtering on data according to a vibration spectrum to obtain standardized data; inputting the data into a construction behavior feature genetic algorithm, and optimizing a recognition network through behavior feature coding to obtain a lightweight model; carrying out probability calculation on the abnormal behavior according to the model; the probability value is transmitted to edge equipment, and model parameters are compressed through weight pruning to obtain a reasoning result; and early warning signals are obtained through three-level early warning threshold grading judgment. The technical problems of unstable data quality, lack of construction scene adaptability of an AI algorithm, delay of cloud processing and insufficient early warning decision reliability of an existing construction monitoring technology in a complex environment are solved.

The invention relates to the technical field of computers, and discloses an interaction control method of intelligent glasses. The method comprises the following steps: synchronously acquiring multi-modal data such as eye movement, voice, gestures and head postures and environment and application context information; carrying out independent time sequence feature coding on each modal data; generating a modulation vector in combination with the context, and outputting probability distribution of user intentions through a cross-modal attention fusion network; and a unique execution instruction is determined through an instruction arbitration module based on rules and a state machine. The system comprises corresponding function modules. According to the method, the accuracy, robustness and naturalness of interaction are improved through multi-modal synchronous fusion and a context self-adaption mechanism.