5178 results about "Feature generation" patented technology

The invention provides a network attack and defense decision support method and system based on artificial intelligence, and relates to the technical field of network attack and defense, and the method comprises the steps: obtaining network topology information and equipment configuration information, constructing a network environment digital twinborn model, collecting network flow data and security logs, and extracting features through a deep learning model. Generating a network security situation assessment report, analyzing the attack graph by using a graph neural network, identifying a target node and generating an initial defense strategy set, generating a plurality of candidate defense schemes by using a multi-target optimization algorithm based on a validity assessment result of a defense strategy, and performing security and feasibility analysis on the candidate defense schemes, the method comprises the steps of selecting an optimal defense scheme, refining the optimal defense scheme into specific implementation steps, generating alternative schemes for coping with different attack conditions, presenting the optimal defense scheme and the alternative schemes through a visual interface, and providing a decision basis and a risk assessment report.

The invention discloses a domain speech recognition method and system based on RAG. The method comprises the steps that an original domain document set is acquired, a hierarchical document structure is constructed through subject classification, paragraph segmentation and sentence segmentation, vectorization processing is executed to generate a multi-granularity vector matrix set, and a hierarchical knowledge vector index structure is established; performing multi-scale time-frequency analysis on the real-time audio stream, extracting and enhancing acoustic features, and generating a multi-scale feature tensor; performing adaptive blocking and progressive attention processing on the feature tensor, constructing a probability estimation model, and generating a candidate text sequence; executing multi-layer retrieval in the knowledge index based on the multi-dimensional query vector to generate related knowledge fragments; and carrying out space mapping and dynamic attention processing on the features, and executing probability optimization decoding to obtain a final recognition text. According to the invention, through knowledge enhancement and multi-level feature processing, the accuracy and robustness of voice recognition in the professional field are improved.

The invention provides an intelligent real-time interactive question-answering system based on a virtual digital human, and belongs to the technical field of voice signal processing and voice recognition, and the system comprises a data acquisition module which receives a voice or text interaction request input by a user, collects the expression dynamic parameter sequence and limb movement sequence data of the user in real time, and transmits the data to a user interaction module; obtaining a standardized voice feature vector and structured text data; the cross-modal fusion module is used for constructing an interactive feature matrix; the behavior decision module outputs a decision instruction set; the knowledge retrieval module is used for generating an answer text with emotional adaptability and voice features; and the voice generation module is used for generating a mouth shape animation key frame, a micro expression parameter sequence and a limb action track of the virtual digital human, generating a voice response in combination with the answer text and the voice characteristics, and pushing the voice response to the user terminal. According to the method, the interaction experience and adaptability of the virtual digital human are remarkably improved.

The invention relates to the technical field of meteorological data analysis, and provides an aerosol prediction method and system based on artificial intelligence, which are used for accurately identifying an abnormal aerosol gathering mode in an extreme meteorological event. The method comprises the steps that an original multi-mode meteorological observation data set is acquired, and the original multi-mode meteorological observation data set comprises an earth surface aerosol concentration sequence, three-dimensional atmosphere state variables and boundary layer dynamic parameters; performing space-time heterogeneity correction on the original multi-modal meteorological observation data set to generate a space-time aligned target multi-modal meteorological observation data set; inputting the target multi-modal meteorological observation data set into a spatial-temporal feature fusion network to extract multi-scale spatial distribution features and cross-modal time sequence correlation features, and generating a spatial-temporal feature matrix based on the multi-scale spatial distribution features and the cross-modal time sequence correlation features; and performing multi-stage recursive optimization on the spatial-temporal characteristic matrix through a hierarchical time domain aggregation algorithm to generate an aerosol concentration prediction sequence of the target region.

The invention discloses an underground facility vibration monitoring method and system based on a distributed sensor network. The method comprises the following steps: acquiring initial data of the distributed sensor network; constructing an initial network topology structure; collecting sensor node state data; dynamic topology adjustment is executed; network performance is evaluated and optimized; an optimized network topology structure is obtained; receiving original data acquired by the sensor and performing data cleaning and standardization by using a network topology structure; a filtering algorithm is adopted to remove noise, and features are analyzed and extracted; and generating a preprocessed feature data set. A multi-source data fusion algorithm module is adopted to integrate information; constructing an anomaly prediction model and optimizing parameters, and performing anomaly detection by adopting the optimized anomaly prediction model; outputting the fused data and an anomaly detection result; and judging whether to trigger early warning or not according to a preset threshold value for an abnormal detection result. Efficient and reliable real-time vibration monitoring is realized, and the anomaly detection accuracy and the system adaptability are improved.

The invention discloses a cross-modal information fusion and retrieval method and system driven by semantic comprehension, and the method comprises the steps: obtaining text, image and audio original data, and extracting an initial feature set of each modal through a deep neural network; dynamically distributing each modal weight coefficient based on an attention mechanism, and performing weighted fusion on the initial feature set to obtain cross-modal fusion feature representation; through a cross-modal semantic association analysis model, high-dimensional semantic association features are extracted from the fusion feature representation, and semantic enhancement feature vectors are generated; constructing a cross-modal semantic graph network based on the vector, complementing missing modal features, and generating an optimized multi-modal feature set; and inputting the optimized feature set and the query sample into a contrast learning model, calculating a semantic similarity score, and generating a cross-modal retrieval result sorting list according to the score.

The disclosed embodiments include centralized, computer-implemented processes for feature generation and management within web-based computing environments. By way of example, an apparatus may transmit first data characterizing a plurality of features to a device, the first data causes the device to present interface elements associated with the features within a digital interface. The apparatus may receive second data that identifies at least a subset of the features from the device, and based on the second data, the apparatus may generate, for each of the subset of the features, elements of executable code associated with a calculation of a corresponding feature value. The apparatus may transmit third data that includes the elements of executable code to the device, which may present the elements of executable code within one or more additional portions of the digital interface.

The invention discloses a turbofan engine operation monitoring method and system based on digital twinning, belongs to the technical field of turbofan engine monitoring, and aims to solve the problems that weak fault signals such as early cracks and abrasion are difficult to extract and the prediction precision of a multi-source fault propagation path is low under a strong noise background. An original operation signal is collected through a sensing array, and is processed by an adaptive resonance demodulation chain to generate a demodulation signal. The method comprises the following steps: carrying out time-frequency transformation on a demodulation signal, constructing an initial candidate feature set by combining feature frequency prior matching actual measurement and theoretical feature frequency, and generating an independent feature set by fusing multi-scale decoupling network separation features of digital twin constraints; for independent features, effective causal pairs are screened by adopting a physical coupling relationship combining Granger causal analysis and digital twinborn simulation, a dynamic Bayesian network is constructed to simulate fault propagation, a posterior probability is calculated through digital twinborn verification and Monte Carlo simulation, early warning is triggered, and a maintenance decision is generated. And weak signal extraction and accurate fault prediction under strong noise are realized.

The invention discloses a real-time video quality optimization and enhancement method based on deep learning. The method comprises the following steps: S1, collecting a video stream and extracting continuous frames; s2, extracting motion area features by using an optical flow estimation algorithm, and generating a multi-modal feature matrix; s3, carrying out normalization processing; s4, outputting a de-noising optimization frame by using an adaptive multi-dimensional residual network and a frequency domain analysis technology; s5, generating a detail enhancement frame by using a progressive generative adversarial network; s6, constructing an inter-frame relation graph through locality sensitive hashing, correcting the frame by using a double-flow time sensing network, optimizing the frame sequence based on a dynamic time warping method, and outputting the corrected frame sequence; s7, inputting the corrected frame sequence into a dynamic feedback optimization module to generate an optimized frame sequence; and S8, recombining the optimized frame sequence into a continuous video, and transmitting the continuous video to a target platform. According to the method, the video definition and the time consistency are comprehensively optimized through methods of multi-modal feature fusion, dynamic optimization, progressive network generation and the like.

The invention provides a real-time deviation analysis method and system for grid-connected operation data of a new energy station, and relates to the technical field of data deviation analysis, and the method comprises the steps: collecting the real-time grid-connected operation data of the station, and carrying out the adaptive decomposition based on a data fluctuation trend, and obtaining a plurality of groups of original feature sequences; screening features through a recursive support degree model and information entropy, and generating a reconstructed feature matrix by using a manifold embedded network; the double-branch verification network analyzes the features from the time sequence and the frequency domain, and generates time sequence feature vectors and frequency domain feature vectors; the topological continuous coherence analyzer extracts the topological relation between the features; the dynamic coupler constructs a multi-dimensional feature space, a deviation source is tracked, analyzed and positioned through a feature propagation link, and a propagation path is established; constructing a fractional order differential operator based on a propagation path, depicting a long-range dependency relationship among parameters, generating a dependency intensity matrix, performing causality test, constructing a dynamic causality graph, quantifying contribution degrees of propagation nodes, analyzing a deviation evolution mechanism, and outputting a deviation analysis result.

The invention relates to the technical field of artificial intelligence, in particular to an AI-driven multilingual intelligent document abstract and mind map generation system, which realizes unified representation of data such as multilingual texts, images and tables through a multi-modal semantic feature generation module, and constructs a dynamically optimized causal semantic map; generating a semantic flow path with clear logic based on a dynamic semantic flow modeling method, wherein the semantic flow path is used for multi-language abstract generation; the structure of the mind map is optimized by using the generative adversarial network, and the content of the map is further perfected through user interaction adjustment and logic consistency verification. The system supports a user to efficiently extract document core information, a logic structure is displayed in a visual mode, the accuracy, logicality and efficiency of document generation are remarkably improved, and the system is suitable for generation and decision support of complex multi-modal documents.

The invention discloses an accurate advertisement putting method and system based on artificial intelligence, and relates to the technical field of information, and the method comprises the steps: collecting user behavior data and real-time situation data, and carrying out the preprocessing of the user behavior data; analyzing the preprocessed user behavior data to generate a user interest label; combining the real-time situation data with the user interest label, constructing a situation interest model by adopting a deep learning technology, and predicting a user interest change mode; using an interest-driven recommendation method to obtain an initialized advertisement putting strategy; generating feedback information based on the analysis of the initialized advertisement putting strategy; and optimizing the feedback information through a reinforcement learning algorithm to generate an optimized advertisement display strategy. By vectorizing the user interest and combining the advertisement strategy characteristics, the generator model can meet the user preference and follow the advertisement materials of the optimal display strategy, so that the visual experience and the emotion resonance of the user are improved, and finally the click rate and the conversion rate of the advertisement are improved.

The invention relates to an unmanned aerial vehicle intelligent dynamic path guidance method and system based on deep reinforcement learning. The method comprises the steps of obtaining three-dimensional terrain point cloud and obstacle information of a flight area, constructing an environment topological structure diagram, extracting features through a three-layer diagram convolutional network, and generating high-dimensional environment feature representation. And a deep reinforcement learning state space is constructed, and a multi-target reward function is designed. And initializing the deep reinforcement learning network by using [-0.1, 0.1] uniformly distributed random parameters to generate an initial path scheme. Optimizing through a dynamic planning algorithm, adjusting a node sequence according to a path length and a task dynamic threshold value, and calculating a target function value. And judging whether the current path scheme reaches the optimal balance point, if not, adjusting the weight of the reward function by using a gradient descent method, and regenerating the path scheme until the path scheme is optimal. And finally, converting the scheme into a waypoint coordinate and speed instruction, performing real-time monitoring during flight, and triggering re-planning when an obstacle exceeds a safety threshold, so as to achieve the purposes of safe flight of the unmanned aerial vehicle in a complex environment and the like.

The invention discloses a water turbine state intelligent prediction method which comprises the following steps: collecting multi-source sensor data of a water turbine, including vibration, temperature, pressure, flow and electrical parameters; performing space-time alignment preprocessing on the multi-source sensor data to generate a space-time associated data set; extracting and fusing the features of the space-time associated data through a multi-modal space-time diagram network, and generating a joint feature vector; performing health state prediction on the joint feature vector based on a dynamic digital twinborn model, and outputting a health score, a fault probability and a confidence interval; analyzing a fault propagation path by using a causal reasoning module, positioning a fault root cause and generating an interpretable report; and triggering an early warning or maintenance decision according to the prediction result. According to the technical scheme, the technical problems that multi-source heterogeneous data fusion is difficult, fault coupling and propagation are uncertain, real-time performance and computing resources are contradictory, and interpretability and reliability are insufficient are solved.

The present disclosure relates to the technical field of artificial intelligence, and provides a multimodal large language model training method, a correlation calculation method, and a label generation method. The multimodal large language model training method comprises: on the basis of a sample text feature vector, a sample image feature vector, and a sample first multimodal feature vector which are obtained by processing image information and text description information of a sample commodity by a pre-trained multimodal large language model, training the pre-trained multimodal large language model to obtain a multimodal large language model; and processing a sample search word, the image information and the text description information on the basis of the multimodal large language model to obtain a sample search word feature vector, a first multimodal feature vector and a sample second multimodal feature vector, and training the multimodal large language model to obtain a trained multimodal large language model. The trained multimodal large language model of the present disclosure can simultaneously learn image features related to the search word and the text description information, and thus the generated second multimodal feature vector is more accurate.

The invention discloses a multi-modal bill processing method based on dynamic knowledge enhancement. The multi-modal bill processing method comprises the following steps: S1, constructing a dynamic knowledge base containing an aging weight; s2, synchronously processing text, image and format features of the bill by adopting a multi-modal feature fusion network to generate a composite feature vector; s3, semantic-level, format-level and timeliness three-level fusion retrieval is carried out based on the composite feature vector, and a three-level fusion retrieval engine comprises dynamic weighted sorting with timeliness attenuation, a difference degree triggered artificial review mechanism and a policy sensitive slope adjustment algorithm; s4, setting a multi-expert cooperative verification system, wherein the multi-expert cooperative verification system comprises cooperative work of a rule engine, a large language model and a logical reasoning module; s5, implementing a dynamic knowledge updating mechanism, and automatically triggering incremental learning of the knowledge base when policy change or format update is detected; and S6, outputting structured data, and synchronously generating an auditing traceability chain containing a decision path. According to the method, the key field identification accuracy can be improved, and auditing traceability and non-perceptual increment updating in the whole process are realized.

The embodiment of the invention discloses an outer wall thermal insulation defect diagnosis method and system based on artificial intelligence, and the method comprises the steps: firstly obtaining an infrared thermal imaging and visible light image sequence of a target building outer wall, the former comprising continuous temperature distribution data, and the latter comprising textural feature data in time-space alignment with the latter; performing dynamic temperature gradient analysis on the infrared thermal imaging image sequence to generate a three-dimensional heat conduction abnormal map, extracting surface deformation characteristics from the visible light image sequence to generate a structure deformation distribution map, and performing multi-modal characteristic fusion on the two to obtain a joint defect characteristic matrix; performing defect type classification and region positioning on the matrix based on a pre-trained deep residual neural network model, outputting a defect type identifier and a corresponding region boundary coordinate, and finally generating a diagnosis report containing a repair priority score and a material matching suggestion according to the defect type identifier and the corresponding region boundary coordinate, and sending the diagnosis report to a user terminal for visual display. And efficient and accurate external wall thermal insulation defect diagnosis is realized.

The invention relates to a remote sensing target detection method and device and a medium, and the method comprises the steps: inputting a feature map to a backbone network for feature extraction, inputting an extracted feature tensor into a multi-branch expansion convolution structure, and extracting multi-scale features through convolution kernels with different expansion rates. Then, multi-scale features are fused through a space and channel double-path attention mechanism, and enhanced features are generated; and the enhanced features are further input into a cascade pooling module to generate multi-level reconstruction features, and weight coefficients are calculated through a gating fusion network to carry out weighted fusion, so that multi-scale fusion features are obtained. Next, these features are input into an asymmetric decomposition convolutional layer for downsampling, and dynamic channel attention calibration is performed to generate channel enhanced features. And finally, inputting the feature map processed by the backbone network and the neck network into a detection head network, and outputting a target bounding box and category prediction. According to the method, high-precision detection of multi-scale rotating targets and high-density small targets is realized in a complex remote sensing scene.

The present disclosure relates to a multimodal computing-based early intelligent graded screening system for a brain disease. The system comprises: a multimodal data acquisition unit, configured to acquire multimodal data of a target patient under a specified screening grade to form a screening data set; a multimodal feature generation, completion, fusion and calculation unit, configured to generate and complete feature data of modal features in the screening data set to obtain complete modal features, and extract pathological features for calculation to obtain a first screening result; a knowledge-based intelligent screening unit, configured to encode the feature data of the modal features in the screening data set into corresponding graph structure data features, use a multimodal association graph optimized by expert knowledge to match the graph structure data features to obtain knowledge-based association features, and obtain a second screening result on the basis of the knowledge-based association features; and an intelligent graded screening unit, configured to carry out weighted calculation on the first screening result and the second screening result to obtain a final screening result. The present disclosure achieves accurate early screening of brain diseases of patients.

The invention discloses an adaptive teaching real-time feedback method based on multi-modal fusion, and the method comprises the following steps: synchronously collecting text modal information, voice modal information and image modal information generated by students in a teaching process, forming multi-modal original data information, and extracting historical student interaction behavior data; preprocessing the multi-modal original data information, and respectively generating corresponding text, voice and image sequence features; a visual feature encoder and a sequence feature encoder are adopted to encode each modal sequence feature to obtain a high-dimensional feature; inputting the modal high-dimensional features into a cross-modal fusion network for deep fusion; parameters of the feedback model are optimized through a model-independent element learning feedback regulation and control algorithm, and a personalized feedback strategy is generated; generating comprehensive feature representation according to the fusion features, and outputting personalized teaching feedback; and the interaction information is updated based on the feedback behavior data to realize closed-loop optimization.

The invention relates to the technical field of traffic management, in particular to a smart city intelligent traffic management system based on big data, which comprises a multi-source data acquisition module, a data fusion module, a dynamic feature generation module, a situation prediction module, a multi-target optimization module and an instruction execution module, the multi-source data acquisition module acquires multi-source data in real time; the data fusion module generates a standardized traffic data flow and outputs a three-dimensional data cube; the dynamic feature generation module outputs a traffic feature matrix; the situation prediction module generates a dynamic situation map; the multi-objective optimization module outputs an optimization instruction set; and the instruction execution module issues a control instruction to the traffic signal controller, the variable information board and the vehicle-mounted terminal. According to the invention, traffic jam can be relieved, road traffic efficiency can be improved, bus priority scheduling can be optimized, vehicle delay can be reduced, and emergency response speed can be improved.

The invention relates to the technical field of wind turbine generator data analysis, and discloses an offshore wind turbine generator operation data analysis and prediction system. The system comprises a marine environment data integration module for collecting data to generate a multi-source time-space synchronization data set; the multi-modal feature fusion module is used for extracting cross-modal correlation features to generate a high-dimensional fusion feature tensor; the dynamic fault prediction module is used for constructing a two-way gating circulation network model to predict the degradation probability and the residual life of key components of the equipment; and the self-adaptive optimization control module is used for constructing a multi-target dynamic programming model to optimize a fan operation strategy. In addition, the system is further provided with a feedback correction module for correcting prediction model parameters, and a virtual sensor module based on a physical information neural network is used for monitoring tower stress and diagnosing sensor faults. According to the system, comprehensive monitoring, accurate fault prediction and optimal control of the offshore wind turbine generator are realized, the operation efficiency, reliability and safety of the wind turbine generator are effectively improved, and the operation and maintenance cost is reduced.

The present invention provides a method and intelligent agent for data collection and drug safety signal mining for PMS, which relates to the field of drug monitoring technology, including standardizing PMS data using a medical text pre-training model, and integrating drug molecular characteristics and clinical medication information to construct a multi-layer drug knowledge graph. A heterogeneous graph neural network with a time series feature extraction function is used to represent the graph, and the drug action mechanism rules are integrated into the training process to obtain a node vector representation of multi-dimensional information fusion. Based on the node vector representation, a multi-task learning framework is constructed to simultaneously complete the drug-adverse reaction association probability prediction and risk level prediction, and combine time series analysis to detect dynamic change characteristics to generate candidate signals. Finally, a hierarchical evaluation system is constructed to verify the candidate signal, and its reliability is confirmed in combination with clinical data, the target signal is output, the warning level is determined, and differentiated monitoring cycles and trigger thresholds are set to perform real-time risk warnings.

The invention relates to the technical field of intelligent manufacturing, in particular to a manufacturing industry digital twinning three-dimensional visual management and control method and system, and the method comprises the following steps: collecting manufacturing equipment operation data, converting the data into a time sequence, analyzing the trend characteristics of each dynamic subsequence in the time sequence, and generating a dynamic operation state sequence set. According to the method, the time sequence characteristics in the operation data of the manufacturing equipment are dynamically analyzed, so that the trend change of the operation state of the equipment can be accurately captured in multiple time dimensions, and potential abnormal states are deeply mined and predicted. In combination with prediction of future operation state parameters and comparison of a normal state range, an abnormal trend region and a corresponding fault level can be clearly marked, and the risk assessment capability of the equipment health state is effectively improved. And carrying out three-dimensional space mapping on the marked risk area, and accurately positioning an abnormal part and a deviation area in combination with gradient change and deviation distribution analysis of three-dimensional coordinates of equipment.

The invention relates to a PCB defect detection method and system based on a YOLOv8 network, and belongs to the technical field of target detection. Collecting and preprocessing a PCB sample image data set for training a PCB defect detection model; inputting a to-be-detected image into the trained defect detection model to obtain a detection result; the to-be-detected image is generated by collecting and processing image data of the to-be-detected PCB; wherein the defect detection model comprises a trunk network, a neck network and a head network, and the trunk network extracts features through a mobile reverse bottleneck convolution module and a C2f module to generate an intermediate feature map; and the neck network performs splicing fusion on the middle feature map to generate a spliced feature map, the head network performs re-weighting on the spliced feature map through a convolution kernel attention fusion module to generate a third feature map, and then the detection module outputs a detection result. The method not only improves the precision of the defect detection model in detecting the features of the tiny target, but also improves the detection speed and generalization ability, and reduces the calculation cost and complexity at the same time.

The invention discloses a multi-screen voice interaction system and method applied in an automobile cabin, and relates to the technical field of vehicle-mounted intelligent interaction, and the system comprises a voice collection unit which is used for carrying out sound source positioning and collection, carrying out the noise reduction processing of a collected voice signal, extracting features from the processed voice signal, and generating a voice feature vector; the multi-modal sensing unit is used for collecting behavior information and physiological state data of a driver and passengers through multi-sensor fusion, so as to extract multi-modal information; the man-machine interaction unit is used for carrying out space-time modeling on the multi-modal information to generate a scene state vector; and the cooperative scheduling unit is used for realizing multi-screen intelligent distribution and cooperative control. According to the invention, voice instructions of a driver and passengers can be accurately identified, the safety and experience of the driver are improved, changes in different driving environments are dynamically adapted, the operation efficiency and user experience of a vehicle-mounted system are improved, and the intelligence and adaptability of the system are improved.

The invention discloses an access anomaly analysis method and system based on multi-dimensional features and user behaviors, and relates to the technical field of dynamic access anomaly detection, and the method comprises the steps: based on a dynamic hypergraph structure, extracting high-order correlation features of the user behaviors through a multilayer hypergraph convolutional network, and generating a high-order feature matrix; based on the high-order feature matrix, generating an authority approval threshold through a causal reinforcement learning framework, constructing a user behavior causal graph to generate strategy network parameters, and storing the strategy network parameters to distributed nodes of a regional data center; based on strategy network parameters stored by distributed nodes, security multi-party computing is adopted, cross-node collaborative optimization is carried out, and global defense strategy parameters are generated through a security aggregation algorithm. According to the method, security multi-party computing is adopted, cross-node collaborative optimization is performed, and the global defense strategy parameters are generated in combination with homomorphic encryption and a block chain fragmentation technology, so that the collaboration efficiency and strategy consistency among distributed nodes are improved on the premise of ensuring data privacy.

The invention discloses a network security analysis early warning system based on artificial intelligence, and the system comprises a data collection layer which captures full flow based on DPI, aggregates firewall logs, terminal behaviors and threat intelligence, and constructs a structured data pool; through TLS fingerprint identification of AI driving, the encrypted traffic is penetrated, and a sampling strategy is dynamically adjusted in combination with reinforcement learning. The intelligent analysis layer is used for carrying out cross validation on known threats and abnormal behaviors; the time sequence CNN extracts encrypted traffic features, and a novel threat detector is rapidly generated by using historical attack fragments in combination with a meta-learning framework; sHAP value driving dynamic feature selection and optimization feature vector input; the decision-making early warning layer is used for fusing multi-source features through a Bayesian network and generating 0-100 score risk scores; a self-adaptive threshold module is combined to adjust a score threshold in real time, and a high-risk event is pushed; the collaborative response layer is used for triggering a preset decision tree, deploying a GAN dynamic honeypot to trap an attacker and reversely tracing; the Neo4j visually restores the attack path, and blocking is executed after the threat is confirmed by a progressive response mechanism.

The invention provides a multi-mode-based man-machine interaction method and system, and relates to the technical field of man-machine interaction. Voice input of a user is received, environment information is collected, facial expressions of the user are captured, and voice features, environment features and facial features are obtained based on the input; utilizing a pre-trained emotion recognition model to generate emotion features of the user, and utilizing a pre-trained user behavior model to generate behavior features according to the current operation information of the user; after the features are obtained, the first weights and the second weights of the features are dynamically adjusted, feature weighted fusion is carried out, and comprehensive features are obtained; the multiple candidate responses are generated based on the comprehensive characteristics, the final response is screened out through a preset scoring mechanism and output, comprehensive understanding of the intention and the emotional state of the user is achieved through comprehensive utilization of multiple data sources, and meanwhile the weight of each characteristic can be dynamically adjusted so that more intelligent and humanized interaction experience can be generated.

The invention discloses a multi-modal information fusion and dynamic updating group behavior analysis method. The method comprises the steps of information integration and dynamic updating, multi-modal information obtaining and information preprocessing, feature extraction, feature representation conversion, alignment and fusion, and multi-modal information real-time processing and dynamic updating through a stream processing framework. A self-learning mechanism and a self-optimization mechanism are introduced; group portrait generation: extracting population statistical characteristics, geographic position distribution, behavior and emotional psychological characteristics, social relationships, social media and economic characteristics from the multi-modal data to generate a group portrait; analyzing group behaviors; and outputting visualization, and outputting group behavior prediction results in a visualized manner, including a behavior trend graph, a group clustering graph, an association rule graph and an abnormal behavior detection graph. According to the method, multi-modal data from various sources and forms are effectively integrated and analyzed, self-learning and self-optimization mechanisms are introduced, and finally group behavior analysis is performed.