1796 results about "Semantic alignment" patented technology

The present invention relates to the technical field of big data processing, and discloses a multi-source heterogeneous data fusion and processing method based on big data, including the following steps: data preprocessing, cleaning, complementing and feature extraction of multi-source heterogeneous data, providing standardized and high-quality input for subsequent processing; dynamic knowledge graph modeling, constructing a dynamic knowledge graph based on the semantic relationship of multi-source data, and generating a unified semantic representation; cross-modal alignment and deep fusion, semantically aligning multi-modal data, and generating a unified comprehensive representation through deep fusion of features; real-time stream processing, dynamically updating and processing real-time updated stream data to achieve real-time fusion of data; adaptive optimization, dynamically adjusting fusion strategy parameters, and improving the adaptability and robustness of the fusion strategy. The present invention realizes efficient fusion, real-time processing and semantic consistency expression of multi-source heterogeneous data through dynamic knowledge graph modeling, cross-modal alignment, real-time stream processing and adaptive optimization.

The invention relates to a knowledge graph construction method and system based on a large language model, and the method and system achieve the automatic construction and dynamic maintenance of a knowledge graph through multi-modal data fusion, reinforcement learning and comparative learning joint optimization, teacher-student model knowledge migration, time sequence dynamic analysis and an incremental updating mechanism. Constructing a reinforcement learning framework, and taking accuracy and integrity as reward indexes to train a large model to extract an entity relationship; a large-scale knowledge graph is used as a teacher model, and conflict resolution and semantic alignment of newly added knowledge and an existing graph are realized through a graph attention network; a verification rule is dynamically generated based on historical data and domain knowledge, and relation periodicity and mutation points are detected in combination with Fourier transform and a CUSUM algorithm; and finally, generating a traceable knowledge graph through incremental updating and version control. And the multi-modal data processing precision, the entity relationship extraction dynamic adaptability and the knowledge graph maintenance efficiency are improved.

The invention discloses an enterprise global data analysis method based on a knowledge graph and a large language model, and the method comprises the following steps: collecting internal and external multi-source data of an enterprise, cleaning, standardizing and mapping, and constructing a unified data graph; after a user inputs a natural language query, a reasoning target and task description are generated in combination with semantic understanding, two reasoning paths are formed through structural reasoning and semantic reasoning respectively, and the two reasoning paths are fused for interactive verification and optimization. Logic conflicts and data missing are detected in real time in the reasoning process, and supplementary knowledge is automatically called for local reasoning correction. Through the corrected reasoning result, the system carries out semantic alignment and correlation analysis on multi-source data, a unified enterprise data analysis view is dynamically generated, a reasoning path and evolution information are recorded in the reasoning and analysis process, and subsequent traceable query is supported. According to the method, the data integration, intelligent reasoning and decision support capabilities of an enterprise in a complex data environment are improved.

According to the research and development document processing method and device provided by the embodiment of the invention, unified processing of text, voice and image information is realized by constructing the multi-mode document analysis engine. Different types of data are converted into standardized feature representations through cross-modal preprocessing and a semantic alignment network. The system adopts a deep learning model to identify research and development elements, establishes a multi-modal relation graph, and realizes intelligent extraction and correlation analysis of various types of information in research and development documents. According to the method, the defects of the traditional technology in the aspects of multi-modal information fusion and knowledge system construction are effectively overcome, powerful support is provided for research and development process management and knowledge asset accumulation, and the standardized management level of research and development documents is remarkably improved.

The invention provides a method for intelligently generating a complete dispatch report for financial unfavorable assets, relates to the field of management systems, and aims at deeply fusing multi-source heterogeneous data such as legal instruments and financial statements and forming a comprehensive context sensing model for target assets through multi-modal feature extraction, semantic alignment and knowledge graph construction technologies; a potential and non-dominant risk factor combination deeply coupled with asset characteristics is automatically mined by applying a genetic algorithm and other evolutionary calculation methods, and a self-adaptive risk assessment network is constructed to dynamically and quantitatively assess and predict the comprehensive risk level; carrying out attribution analysis on a risk assessment result by adopting an interpretable artificial intelligence model, and clearly revealing a key influence path and core data evidence; and according to a report logic framework and a narrative template which can be dynamically adjusted by a user, automatically outputting a financial non-performing asset full-duty survey report including deep analysis, risk early warning, diversified disposal suggestions and compliance review key points.

The invention provides a scientific and technological literature intelligent retrieval method and related equipment based on generative artificial intelligence, and the method comprises the steps: carrying out the multi-layer semantic annotation of medical scientific and technological literatures, constructing a symptom-disease dynamic association map, and building a medical scientific and technological literature knowledge base; performing medical context analysis and multi-modal feature extraction on user query, and generating a unified retrieval vector in combination with Boolean operation nested analysis and semantic alignment processing; performing evidence grading retrieval and clinical scene matching based on the unified retrieval vector to obtain a preliminary candidate literature set, and optimizing the preliminary candidate literature set into a target candidate literature set through fine-grained semantic recalculation; and calculating a retrieval prior probability according to the evaluation dimension, performing knowledge weighted fusion on the candidate literature, and generating a medical science and technology literature recommendation report. According to the method, the medical term association relationship is deeply understood through the association map, the result is ensured to be matched with the patient characteristics through evidence grading retrieval and clinical scene matching and screening, and the accuracy of document retrieval is improved.

The invention discloses an online teaching interaction method based on a multi-modal knowledge graph, a medium and equipment, and the method comprises the steps: firstly collecting original teaching information, and carrying out the cross-modal semantic alignment processing, and obtaining structured teaching information; then constructing a multi-modal knowledge graph containing text concept entities, video key frame feature vectors and voice text transcription contents, and forming a concept-visual feature association edge, a concept-voice segment association edge and a cross-modal similarity association edge; converting the user interaction behavior data into a knowledge graph query vector, extracting a three-dimensional teaching situation sub-graph from the multi-modal knowledge graph, and generating progressive or comparative teaching content according to a user operation type; and finally, dynamically adjusting a teaching strategy by constructing a cognitive state tracking matrix, and generating a personalized learning navigation map. The problems of knowledge fragmentation and insufficient interaction intellectualization in virtual simulation teaching are solved, and the teaching effect of ideological and political education is improved.

The invention relates to a multi-modal public opinion risk early warning system and method based on a dynamic knowledge graph and federal reinforcement learning, and belongs to the technical field of public opinion analysis. The system comprises a data acquisition module, a modal fusion module, a knowledge graph construction module, a comparative learning module, a federal reinforcement learning modeling module and a response output module. The system is based on multi-source heterogeneous data, multi-modal semantic alignment of texts, images, videos and the like is achieved, entity relations and propagation paths are mined through a dynamically updated knowledge graph, collaborative modeling under privacy protection among terminals is achieved by fusing federal reinforcement learning, and then real-time sensing, level early warning and multi-level response strategy recommendation of public opinion risks are achieved. Based on the system, the method has the advantages of high fusion precision, high response speed and strong visual propagation path, and is widely applied to the fields of enterprise crisis management, government affair and public opinion monitoring and public safety.

The invention provides a multi-source heterogeneous corpus fusion method and system based on government affair service data, and the method comprises the steps: obtaining an original corpus set of a plurality of data sources in government affair service, carrying out the cross-modal semantic alignment processing of each corpus unit in the original corpus set, generating a normalized data block corresponding to each corpus unit, and carrying out the fusion of the data blocks; carrying out multi-modal semantic coding on the standardized data blocks to obtain semantic feature vectors of all corpus units, carrying out topological structure coding on association attribute sets among the standardized data blocks to generate a global structure relation graph, and carrying out dynamic weight distribution on the semantic feature vectors based on node connection weights in the global structure relation graph to obtain semantic feature vectors of all corpus units; and generating a fusion weight matrix, performing cross-modal feature fusion on the semantic feature vector to obtain a target semantic embedding representation, and generating a standardized corpus associated with the government affair service. According to the method, the semantic aggregation problem of the non-uniformly distributed corpus units is solved, and the government affair data governance efficiency and the cross-department cooperation capability are greatly improved.

The invention relates to a brilliance large-scale model culture knowledge generation and retrieval method combined with a knowledge graph. The method comprises the steps that entity extraction, relation modeling and dynamic completion processing are carried out according to multi-modal culture data, a culture knowledge graph is constructed in combination with a fuzzy time reasoning algorithm and a cross-modal alignment mechanism, and the multi-modal culture data comprises an unstructured culture text, image data and space-time metadata; the space-time metadata comprises text space-time metadata and image moment metadata; performing semantic alignment processing by combining a large model through a user query statement and a culture knowledge graph to obtain joint representation of cross-modal semantic alignment; and performing multi-modal retrieval-generation joint optimization based on the joint representation to obtain cultural knowledge description and associated multi-modal content. According to the method, through multi-modal data fusion, dynamic knowledge completion and fuzzy time reasoning, culture knowledge description can be accurately generated, associated multi-modal content can be retrieved, and the accuracy and the intelligent level of culture knowledge service are improved.

The invention discloses an adaptive question-answering system based on knowledge distillation and multi-modal dynamic fusion, and the system comprises a knowledge distillation module which is used for migrating knowledge of a teacher model pre-trained on corpora in the communication field to a lightweight student model, achieving model compression through optimizing a distillation loss function, and obtaining a multi-modal dynamic fusion model; the loss function comprises a soft label output by the teacher model and a KL divergence constraint output by the student model; the multi-modal knowledge fusion module comprises a feature extraction unit, a self-adaptive weighting unit and an attention fusion unit; the self-adaptive inference engine comprises a semantic analysis unit; according to the cross-modal reasoning method and system, semantic alignment of equipment parameters, protocol texts and topological graphs is achieved through the multi-modal dynamic fusion technology, and the cross-modal reasoning accuracy is improved; compared with an original model, the lightweight student model has the advantage that the reasoning speed is increased in a protocol analysis task.

The invention discloses a knowledge graph construction method for multi-modal data, and relates to the technical field of intelligent search optimization, and the method comprises the steps: matching similar knowledge entities based on a cross-modal consistency measurement method, combining redundant data through employing an information entropy deduplication strategy, carrying out semantic alignment through employing a graph convolution network, and dynamically adjusting a clustering structure, thereby achieving the optimization of the knowledge graph. The data integration quality of the knowledge graph is improved; aiming at the problem of lack of context sensing ability, search behavior data such as search keywords, click frequency and residence time are collected in real time, a search environment adaptation factor Gdf is calculated and evaluated, knowledge entity association weight is optimized by adopting reinforcement learning, and search suitability is improved; in order to solve the problem that multi-modal data is difficult to process, a cross-modal contrast learning method is adopted, a cross-modal matching degree index Mcf is calculated and evaluated, and the weight of each modal data is dynamically adjusted based on a logarithmic geometric mean, so that different modal data are fused in a shared vector space, and optimized modeling and retrieval of a cross-modal knowledge graph are realized.

The invention discloses a multi-source heterogeneous knowledge graph data fusion method and system, and the method specifically comprises the following steps: S1, collecting data from a plurality of heterogeneous data sources, cleaning the collected data, removing noise data and repeated data, and converting the data in different formats into a unified intermediate format; s2, a semantic model is established, and semantic heterogeneous concepts and relations in different data sources are recognized and aligned through a natural language processing technology and domain ontology knowledge. The invention relates to the technical field of knowledge graph data processing, and the multi-source heterogeneous knowledge graph data fusion method and system effectively reduce the problems of noise, repetition and semantic inconsistency in data through data cleaning, semantic alignment and conflict resolution, and ensure the accuracy and consistency of fused data, thereby improving the quality of a knowledge graph and improving the reliability of the knowledge graph. Reliable knowledge support is provided for application in the fields of medical treatment and the like.

The invention discloses a children MPP (Mycoplasma Pneumoniae) auxiliary diagnosis system based on multi-modal time series data modeling, and aims to improve the auxiliary diagnosis, clinical typing and severe prediction capabilities of mycoplasma pneumonia and reduce the workload of doctors. The MPP dynamic representation method based on multi-modal time sequence modeling is innovatively provided, multi-modal data such as images, electronic medical records and inspection indexes are fused, the time sequence feature embedding technology is adopted, and the evolution trend of the disease course can be effectively captured. Furthermore, deep fusion of different modal data is realized by using a progressive cross-modal semantic alignment strategy and a modal attention gating mechanism, and the problem of insufficient diagnosis accuracy caused by incompatibility of data modals in a traditional method is solved.

The invention relates to the technical field of computer vision and multi-modal learning, in particular to a weak supervision target detection method guided by cross-modal pseudo labels. According to the method, a labeled source domain data set is constructed to train an image classification teacher model, and a teacher-student network structure is constructed; clustering the regional features of the target domain image, allocating pseudo tags to each cluster by optimizing the allocation cost between the source domain category and the target domain cluster, and constructing a pseudo tag pool; and training a student model on the pseudo label pool for region feature detection of the target domain image. According to the method, a cross-modal attention mechanism is introduced, so that more accurate semantic alignment between a source category label and a target domain feature is realized; the stability of label distribution is improved by a structure keeping regular term; the generalization ability of the model is further enhanced by multiple rounds of pseudo-label confidence learning. The method can be widely applied to tasks such as target detection, cross-domain transfer learning and open world recognition, and efficient and accurate weak supervision target detection is realized.

The invention discloses a multi-modal semantic and physical law driven remote sensing image generation method, belongs to the technical field of computer vision and remote sensing image generation, and aims to solve the problems of insufficient cross-modal semantic alignment, low reliability of a generation result and insufficient physical mechanism fusion. The four-stage method comprises the following steps: firstly, rejecting low-quality samples from original data and unifying a spatial scale; then, extracting a multi-modal semantic vector by adopting a BLIP model and a CLIP model, and introducing a remote sensing physical rule to carry out vector optimization; then position coding and physical constraint conditions are embedded in the submerged space, and multi-source information joint modeling is achieved through a cross-modal encoder; and finally, by taking text description, physical priori knowledge and diffusion time steps as joint conditions, performing de-noising reasoning based on a Transform architecture, and completing back diffusion reconstruction by means of a trans-attention mechanism. According to the method, physical rationality and semantic consistency are improved, and a more reliable technical normal form is provided for remote sensing image generation in the fields of disaster monitoring, military simulation and the like.

The invention discloses a decision optimization method fusing enhanced multi-modal learning and a knowledge graph, and relates to the technical field of artificial intelligence and knowledge graphs. According to the decision optimization method for fusing enhanced multi-modal learning and the knowledge graph, dynamic fusion of multi-modal features is realized through a dynamic weight adjustment and semantic alignment constraint mode, semantic precision and interpretability are improved, the semantic deviation problem caused by traditional static feature fusion is effectively solved, and the method has the advantages of being high in robustness and high in reliability. And by recording an intelligent reasoning path selected by a hierarchical reinforcement learning agent, interactive graph structure display can be carried out, the advantage of transparency is achieved, meanwhile, the interpretability of the path is further improved in cooperation with a multi-target reward function, intelligent updating of the knowledge graph is carried out in cooperation with comprehensive confidence, intelligent growth of the knowledge graph is achieved, and the intellectual property of the knowledge graph is improved. And a fine-grained interpretable report is generated through an adversarial training mechanism and anti-factual reasoning, so that the false alarm rate of an output result is further reduced, and the decision transparency is improved.

The invention discloses an enterprise data asset intelligent analysis method and system based on AI recognition, and the method comprises the steps: receiving an enterprise multi-source heterogeneous data stream, carrying out the joint feature extraction and semantic alignment through a pre-trained multi-modal fusion recognition model, and generating a structured data asset recognition result; constructing a dynamic enterprise data asset atlas according to the structured data asset identification result in combination with the data access trajectory and authority metadata collected in real time; performing spatio-temporal evolution analysis on the dynamic enterprise data asset map, and extracting potential data value density features and risk exposure features; inputting the data value density features and the risk exposure features into a self-organizing mapping network to generate a data asset grading topological graph; and based on the data asset grading topological graph, through strategy constraint reinforcement learning, generating an executable data governance action sequence. According to the embodiment of the invention, the identification precision and real-time analysis capability of special assets of enterprises can be improved.

The invention relates to the technical field of shopping guide, in particular to a multi-mode user intention understanding and personalized shopping guide generation method and system, and the method integrates vision, voice and touch screen data to construct a commodity and user intention knowledge graph. Through a cross-modal contrast learning architecture and deep feature learning of user intentions, precise commodity matching and sorting are realized; and in combination with a knowledge-enhanced semantic alignment model, generating a personalized recommendation verbal skill, and making a shopping guide strategy by using a sequence-to-sequence neural network. Complementary information among different modal data can be captured, a unified semantic representation space can be learned, and the accuracy of user intention understanding is greatly improved; the system breaks through the technical barriers, significantly improves the intelligent shopping guide capability of the e-commerce platform, rebuilds the shopping experience of the user, shows huge commercial value, and injects new power for industry development.

The invention discloses a retrieval method and device based on a knowledge graph, computer equipment and a storage medium. The method comprises the steps that the knowledge graph is acquired, a node relation vector is output through a preset relation learning module, nodes related to user query content are retrieved from the knowledge graph, and an external knowledge representation result is generated; performing feature fusion on the user query content and the external knowledge representation result by using the correlation matrix to obtain a fused feature vector; performing image-text bidirectional semantic alignment on the fused feature vector based on an attention mechanism to generate a joint feature vector; and performing matching calculation based on the joint feature vector to obtain a matching score for representing the semantic correlation degree. According to the method, the user query content and the generated external knowledge representation result are fused through the correlation matrix, image-text semantics are aligned through an attention mechanism, the joint feature vector is generated, the matching score is calculated, and cross-modal retrieval is realized by using the matching score, so that the information retrieval accuracy of the community property management system is greatly improved.

The invention relates to the technical field of artificial intelligence, can be applied to business scenes of financial science and technology, medical health and the like, and discloses a decision-making method, device, equipment and medium based on multi-modal semantic alignment. Executing cross-modal alignment by taking the voice semantic map as a reference to generate associated information, fusing the voice features, the visual features, the action features and the associated information to generate a fusion feature vector, inputting a decision network to generate a decision feature vector and generate a task execution instruction, obtaining execution feedback information of the task execution instruction, and updating the decision network. According to the method, input is dominated by voice instructions, visual features, action features and semantic map depth alignment and fusion are combined, input naturalness and multi-modal data analysis and decision-making efficiency are improved, and interaction adaptability and decision-making accuracy of the model in a complex scene are improved.

The invention belongs to the field of natural language processing, particularly relates to a multi-modal document generation method and device based on multi-agent collaboration, and aims to solve the problems that an existing method is low in intention recognition accuracy, limited in retrieval range and not professional enough in content generation. The method comprises the following steps: generating a structured template; analyzing the text input by the user to identify a writing intention, and determining a target template; vectorizing each candidate resource feature to obtain a corresponding sparse vector, a dense vector and a knowledge vector, and performing semantic alignment; extracting context features of the input text, respectively performing multi-path retrieval recall, evaluating and sorting recall results, and screening out target features; and constructing a thinking chain in combination with the knowledge graph, and generating a multi-modal document according to the target template. According to the method, the outline structure can be extracted, the picture / table style can be recognized, the templates adaptive to different document types can be dynamically generated, and full-process automation from user input to document output is achieved.

The invention provides a semantic understanding method based on natural language processing and a science and technology operation platform system.The method comprises the steps that firstly, a target interaction data set containing a text instruction sequence, collaboration document content and resource scheduling request records in a science and technology management scene is obtained, and then cross-modal semantic alignment processing is conducted on the target interaction data set; and generating a structured semantic analysis result containing an intention decision tree and the like. Then, based on a dynamic process arrangement rule, the result is mapped to a predefined process node topological space, an initial process execution path set containing a node execution sequence chain and the like is obtained, conflict resolution is conducted on the set according to real-time system resource state data, and an optimized executable process path containing resource allocation priority and the like is generated; and finally, the optimized executable flow path is input into a science and technology management central system, automatic deployment and execution state tracking of a cross-department cooperation flow are realized, and the semantic understanding and flow processing capability of a science and technology management scene is effectively improved.

The invention relates to the technical field of machine vision, and discloses a machine vision-oriented cross-modal image-text analysis method, which comprises the following steps of: partitioning an input image to generate an image block sequence; inputting the image block sequence into a visual converter for multi-scale feature extraction, and generating target visual features; encoding the input text to generate a target text feature; inputting the target visual features and the target text features into a deep reconstruction bottleneck network for compression alignment, and generating a cross-modal compression vector; and inputting the cross-modal compression vector into a large language model to generate cross-modal decoding information, so that cross-modal redundant information can be effectively filtered, compact shared semantic representation can be learned, the information integrity of the compression process is ensured through bidirectional reconstruction verification, cross-modal semantic alignment is realized, and the method has the advantages of high efficiency and high reliability. Omnibearing cross-modal content generation from the whole to details is achieved, and the requirements of different application scenes are met.

The invention discloses a multi-label electrocardiogram classification method based on self-supervised pre-training and multi-modal semantic alignment, which belongs to the technical field of artificial intelligence, and comprises the following steps: realizing self-supervised pre-training of unlabeled data through a single-modal contrast enhancement network, generating global and local contrast views by adopting a multi-scale random cutting strategy, and classifying the global and local contrast views in a multi-scale random cutting mode; in combination with a teacher-student network architecture, the potential invariance features of the ECG signals are learned while negative sample dependence is avoided, the problem of annotation data scarcity is effectively relieved, and the feature robustness is improved. A multi-modal fusion mechanism based on label semantic guidance is provided, a time domain signal and a frequency domain time-frequency graph are mapped to a unified semantic space through fine-grained semantic alignment, local feature enhancement and cross-modal complementary information fusion are realized by using a cross attention mechanism, and the problem of semantic difference caused by modal heterogeneity in a traditional method is overcome. A multi-label comparison loss function based on a disease co-occurrence relation is proposed, a category discrimination boundary is dynamically optimized by modeling a label co-occurrence probability, the feature separability of a tail category is improved while the head category discrimination ability is enhanced, and the problem of sample category imbalance in a multi-label scene is remarkably relieved.

The invention discloses a personalized recommendation system based on semantic analysis, and relates to the technical field of personalized recommendation systems.The personalized recommendation system comprises a data acquisition module, a dynamic situational semantic modeling module and a recommendation engine module, the data acquisition module transmits acquired user behavior data and situational data to the dynamic situational semantic modeling module, and the dynamic situational semantic modeling module transmits the acquired user behavior data and situational data to the recommendation engine module; the dynamic situational semantic modeling module extracts sequence features of user behavior data through a behavior context analyzer, generates context embedding through a situational semantic embedding network, completes cross-modal semantic alignment of multi-modal data through a multi-modal fusion layer, and generates multi-level semantic embedding features. According to the method, the dynamic situational semantic modeling module is designed, the semantic alignment function of the situation and the behavior is achieved, the problem that a traditional recommendation system is insufficient in situation data utilization is solved, the adaptability of the recommendation result to the dynamic situation is enhanced, and the method is suitable for dynamically changing user demand scenes.

The invention provides a multi-modal heterogeneous model retrieval enhancement method and system, and the method comprises the steps: building a knowledge and application example double-corpus based on user multi-modal query, and designing a joint retrieval mechanism to obtain a result set; mapping and scheduling to obtain feature representation through special processing channels for texts, images and audios and a Spiking neural network with a segmented trapezoidal topological structure; constructing a three-stage cascade architecture of a basic model, an advanced model and human experts, and obtaining a decision path and answer candidate set in combination with a recursive and discarding decision mechanism; a Hamiltonian graph network is used for representing a multi-modal relation, and a gradient-free descent method is used for rapidly training and optimizing model parameters; an enhanced retrieval result is obtained through cross-modal semantic alignment and dynamic retrieval window adjustment; and high-quality response is obtained through context-aware sorting and retrieval enhanced reasoning. According to the method, the multi-modal information retrieval processing efficiency and the heterogeneous model reasoning response quality are improved.

The invention relates to a groundwater safety assessment method under an extreme climate event, which comprises the following steps: collecting multi-source heterogeneous data such as meteorological data, geological data, hydrological data and remote sensing data, and constructing a unified groundwater safety knowledge graph through standardized cleaning, semantic alignment and deletion completion; monitoring an extreme climate event in real time, and updating a node relation weight and sparsifying a transmission path based on knowledge graph dynamic evolution and a time sequence attention mechanism; performing risk propagation path reasoning on the dynamic knowledge graph in combination with an improved graph neural network, identifying key pollution nodes, and outputting a structured risk level and a coping suggestion; the system continuously optimizes atlas and model parameters based on evolution feedback, and high adaptability and reasoning precision of emergency response are achieved. According to the method, the intelligence, the real-time performance and the accuracy of underground water risk assessment are improved. The problems that the underground water pollution propagation path is difficult to dynamically identify and the decision adaptability is insufficient under extreme climate events are solved.

The invention discloses a multi-modal data-oriented multi-agent deep forgery attack detection system, and particularly relates to the technical field of electrical equipment image analysis and voice-assisted fault judgment, an image and voice semantic vector is constructed through a semantic extraction module, a semantic alignment module calculates an alignment score and a coverage ratio, and a multi-agent deep forgery attack detection result is obtained. The semantic judgment module recognizes semantic inconsistency, the agent fusion module outputs comprehensive fault judgment based on multi-dimensional scores, and the interpretation display module realizes risk area visualization and semantic deviation labeling and enhances the accuracy, interpretation and practicability of fault detection; structured semantic extraction and alignment judgment of the multi-modal power operation and maintenance data are realized, and the fault detection precision is improved based on dual-index consistency evaluation and multi-level conflict identification; a multi-agent confidence scoring and dynamic fusion mechanism is introduced, so that the judgment credibility is enhanced; and image-text linkage explanation output and report generation are supported, and operation and maintenance results of the power grid operation and maintenance scene can be rechecked and used conveniently.

The invention relates to the technical field of robot decision making, and provides a robot multi-modal fusion autonomous decision making method and system based on a large language model.The method comprises the steps that a robot obtains multi-modal environment information through a visual sensor, a touch sensor, an auditory sensor and a laser radar which are carried by the robot; performing preliminary filtering and noise reduction processing on the original sensor data, and synchronously recording all the sensor data by timestamps; performing space-time semantic alignment on the preprocessed multi-modal data, mapping pixel coordinates of a target in a visual target coordinate quantization original image to a robot coordinate system, performing uncertainty evaluation on a multi-modal signal through a dynamic Bayesian network, and taking entropy or variance as an uncertainty quantitative evaluation index. According to the method, the information quality is improved from a data fusion source, accurate and reliable basic support is provided for subsequent decision making, and decision making errors caused by data deviation are greatly reduced.