11794 results about "Multi modal data" patented technology

The invention belongs to the technical field of bridge diagnosis, and discloses a bridge crack intelligent diagnosis system based on multi-modal data fusion. By fusing multi-source data such as visual images, sound wave detection and vibration signals, comprehensive perception and characterization of crack features are realized; constructing a bridge crack characteristic spectrum diagram by adopting a cross-modal feature extraction and heterogeneous feature coding technology; generating a crack evolution situation map based on space-time correlation analysis and knowledge graph construction; the robustness of the system in a complex environment is improved through environmental adaptability feature enhancement and multi-scale characterization; constructing a bridge safety risk hypergraph in combination with multi-dimensional risk analysis and multi-agent collaborative diagnosis; analyzing and revealing a crack evolution mechanism by applying a causal relationship; and finally, through dynamic fusion and uncertainty quantification, a crack intelligent diagnosis comprehensive report is generated. According to the system, the limitation of traditional single-mode diagnosis is broken through, and dynamic prediction and accurate risk assessment of fracture evolution are realized.

The present invention provides a system and method for causality-augmented generative intelligence capable of autonomously discovering non-obvious actionable insights from heterogeneous and multimodal data sources. The system integrates a data ingestion unit for semantic and temporal harmonization of structured and unstructured datasets, a causal inference processor for constructing a dynamically evolving directed causal knowledge representation using perturbation-based validation, a latent representation processor that combines multimodal semantic embeddings with causal parameters to generate fused latent vectors, and a generative insight processor utilizing causally constrained generative reasoning to synthesize hypotheses anchored to verified cause-effect dependencies. A validation processor performs counterfactual assessment and observational verification to ensure retention of only those insights that remain consistent with causal ground truth.

The invention discloses an infrastructure risk operation optimization management system based on AI driving, and belongs to the field of computer data processing and commercial management, and the system comprises a multi-modal causal twinning construction module which integrates on-site multi-modal data streams to construct a dynamic space-time causal map; the risk evolution deduction module is used for performing anti-fact simulation based on a causal atlas to construct a prospective risk model; the collaborative configuration optimization module is used for solving an optimal collaborative defense strategy according to the risk model; the instruction analysis and digital prescription generation module is used for analyzing the defense strategy into a job digital prescription for a specific risk scene; and the intervention efficiency attribution and evolution correction module performs attribution analysis according to the execution effect of the digital prescription and adaptively updates the causal atlas. According to the method, a comprehensive method of constructing a dynamic causal map for risk deduction, coupling resource constraints for collaborative optimization and performing closed-loop feedback on a correction model is adopted, and active prediction, accurate intervention and continuous learning optimization of capital construction risks can be realized.

The invention relates to the technical field of artificial intelligence, can be applied to business scenes of financial science and technology, medical health and the like, and discloses a multi-modal causal reasoning and interpretation method, device, equipment and medium, and the method comprises the steps: obtaining original data streams of at least two different modals, and extracting modal features; a cross-modal attention mechanism is utilized to fuse modal features, and causal features are extracted through feature distillation; constructing a dynamic causal graph based on causal features, and updating an edge weight through a causal intensity function; identifying the causal relationship in the dynamic causal graph and performing anti-factual reasoning verification to evaluate the reliability of the causal relationship; and generating a causal interpretation result in combination with the dynamic causal graph and the causal relationship reliability. According to the method, the multi-modal data are fused, the causal features are extracted, and dynamic causal graph updating and anti-factual reasoning verification are combined, so that reliable modeling and explanation of the causal relationship in a complex scene are realized, the defects of single modal or simple fusion in the prior art are overcome, and the accuracy and interpretability of causal reasoning are improved.

The invention relates to the technical field of fault prediction, and provides a fault prediction method for a multi-modal cross-attention enhancement graph neural network, and the method comprises the steps: collecting the data of equipment; performing adaptive enhancement and normalization processing on the image data, performing sliding window segmentation, standardization and noise suppression on a time sequence numerical signal, and performing semantic vectorization coding on a maintenance log text; extracting low-dimensional spatial features of image data by using the pruned lightweight convolutional neural network, connecting time sequence features of modeling time sequence numerical signals in series, extracting context semantic expressions of maintenance log texts, integrating the features into multi-modal data, alternately taking each modal feature as Query and the other modal features as Key and Value, and obtaining multi-modal data; calculating attention weight and performing weighted fusion; constructing a modal node weighted graph, and performing inter-node feature propagation through a multi-layer graph attention network; and a residual service life regression prediction module and a degradation level classification module are deployed in parallel, and fault early warning is completed through multi-task joint optimization.

Disclosed in the present application are a multi-modal data processing method and apparatus, an electronic device, and a storage medium. The method comprises: acquiring a reference image and a reference text; extracting a reference visual feature of the reference image; by means of a multi-modal large language model, determining an embedding of the reference text, an embedding of a start mark of the reference visual feature, an embedding of the reference visual feature, and an embedding of an end mark of the reference visual feature; on the basis of the multi-modal large language model, splicing the embedding of the reference text, the embedding of the start mark, the embedding of the reference visual feature, and the embedding of the end mark into a target embedding sequence, performing attention processing on the basis of the embedding of the start mark, the embedding of the end mark, and an embedding selected by a sliding window in the target embedding sequence, and outputting a predicted sequence; and generating a predicted image and a predicted text on the basis of the predicted sequence.

The invention discloses a multi-modal interview automatic quality analysis and evaluation method and system based on a large model, and the method comprises the steps: collecting and storing multi-modal data, such as texts, audios, videos and behavior interaction, and carrying out the preprocessing of the multi-modal data to form a standardized data set; utilizing a preset interview structure and a large model to dynamically guide the process, adjusting the topic rhythm according to real-time feedback, and recording stage conversion information to form logic trajectory data for process coherence management; automatically coding text data through a large language model, extracting features such as keywords and performing topic clustering, performing cross validation and semantic fusion in combination with data analysis results of each modal, and generating deep analysis results such as psychological states; and generating a comprehensive assessment report containing qualitative description, quantitative score and psychological abnormality or cognitive disorder risk prompts based on a deep analysis result, thereby providing a basis for psychological health assessment and cognitive competence evaluation. According to the method, automatic analysis of multi-modal data is realized, and evaluation scientificity and efficiency are improved.

The invention relates to the technical field of signal devices, and discloses a traffic signal control method and system based on vehicle-road cloud multi-modal data fusion, and the method comprises the steps: collecting multi-modal traffic data synchronously in real time through a vehicle-end sensor, road-side sensing equipment and a cloud Internet platform; fusing the heterogeneous data by adopting a space-time alignment algorithm, and constructing a standardized space-time feature matrix; traffic flow prediction is carried out based on a multi-layer space-time diagram neural network trained by a federated learning mechanism, and a signal control instruction is generated through reinforcement learning and a multi-objective optimization model; and issuing the green wave parameter, the dynamic timing scheme and the cross-domain coordination strategy to a roadside signal machine through the cloud edge coordination architecture to execute control. The problems that in the prior art, low-delay private network communication cannot be achieved, the data fusion efficiency is low, unmanned driving is not supported, and the deployment cost is high are solved, and the purposes of low-delay communication, high reliability and low risk are achieved.

The invention relates to a multi-modal data and RAG technology fused power design knowledge base construction method, and belongs to the technical field of power software development. The method comprises the following steps: carrying out collection and information extraction on multi-source heterogeneous original data; the method comprises the following steps of: constructing a multi-dimensional knowledge element structure containing parameters, specifications and case relationships by carrying out classification, specialized and precise processing and cross-modal association on data; based on a vector, graph and relational database mixed storage architecture, semantic vector efficient retrieval, knowledge graph relation management and business data synchronization are achieved respectively; and a dynamic optimization result is subjected to hybrid retrieval, a dual-drive reasoning mechanism outputs compliance conclusions and bases, and a retrieval enhancement generation service ensures that output contents conform to specifications. And systematic management and intelligent application of the electric power design knowledge are realized.

The invention discloses a user behavior prediction system and method based on multi-modal data fusion, and particularly relates to the field of user behavior prediction, and the system comprises a multi-modal data collection module, a preprocessing and feature extraction module, a cross-modal fusion module, a user behavior prediction module, and a model optimization and feedback module. According to the system, multi-dimensional original data such as visual sense, auditory sense, text, physiological signals and environment context of a user are acquired in real time through a multi-modal data acquisition module; then, deep networks such as ResNet, VGGish and BERT are adopted to extract high-dimensional feature vectors of all modals, and contribution weights of features of different modals are dynamically learned through an attention mechanism; and finally, based on a time sequence model of Transform and LSTM, analyzing fusion features, and outputting probability distribution of future behavior intentions. And parameter joint optimization and continuous learning are realized through a multi-objective loss function and end-to-end back propagation.

The invention relates to the technical field of multi-modal information processing, and provides a deep semantic collaborative fusion method for heterogeneous multi-modal data. The invention provides a dynamic adaptive fusion framework aiming at the problems that a modal interaction mechanism is rigid and semantic modeling is shallow in the prior art. The method comprises the following steps: carrying out feature coding and alignment on text, audio and video modal data to generate unified-dimension single-modal representation; dynamic interaction is realized through an enhanced multi-head gating fusion module, and double-path features are generated; and carrying out cross-modal depth modeling on the basis of a stacked Transform encoder, and outputting final fusion semantics. Wherein the multi-head attention path calculates cross-modal mapping by taking a text as a query vector and taking an audio / video as a key value vector; the gating path generates a dynamic weight through cosine similarity and a learnable temperature parameter; and the dual-path adaptive fusion adopts a balance factor alpha weighted combination. According to the method, the multi-modal data fusion precision and the system robustness are improved, and the method is suitable for government affair service, man-machine interaction and other scenes.

A multimodal intelligent agent system for dynamic environmental monitoring and user-centered support, consisting of: a multimodal sensor module configured to continuously acquire environmental and behavioral data from multiple input modalities, including at least one visual sensor, at least one acoustic sensor, at least one environmental conditions sensor, and at least one proximity or motion detection sensor, each generating modality-specific data streams representing visual images, audio waveforms, physical environmental parameters, and motion signatures within a monitored environment; a data preprocessing and fusion subsystem that is operationally coupled with the multimodal sensor module and configured to normalize, temporally align, and transform the modality-specific data streams into high-dimensional feature embeddings using a variety of encoders, wherein the visual encoder uses convolutional or vision transformer architectures, the audio encoder uses a spectral-temporal feature extractor, and the sensor encoder transforms raw analog data into context vectors suitable for multimodal alignment; a multimodal processing unit consisting of a transformer-based large language model (LLM) trained on paired multimodal datasets and configured to perform semantic fusion, context abstraction, and inference across the aforementioned aligned multimodal feature embeddings to generate a contextual understanding of environmental and behavioral states; an adaptive agent controller coupled to the multimodal inference processing unit and configured to instantiate, manage, and terminate a variety of task-specific intelligent agents, each agent being a software unit configured to perform a specialized function selected from meeting summarization, behavioral analysis, misplaced object detection, or environmental anomaly identification, with the agents dynamically interacting with the inference engine to retrieve contextually relevant multimodal embeddings for task execution; a personalization and adaptive learning subsystem consisting of a user preference database and a neural memory structure configured to update and refine model parameters based on user-specific interaction history, thereby enabling personalized output generation, prioritization of recommendations, and long-term behavioral adaptation; and An output generation interface is operationally connected to the adaptive agent controller and configured to produce multimodal output in textual, visual, and auditory form. The interface is capable of displaying human-readable summaries, notifications, and visual reconstructions of identified entities or environmental states.

The invention discloses a judicial scene-oriented multi-modal data fusion method and system. The method comprises the following steps: 1) collecting multi-modal data in a judicial application scene and converting the multi-modal data into data in a uniform format; 2) mapping the data to a unified semantic space to realize cross-modal alignment; then associating the multi-modal data to obtain the text description of the same case and the corresponding image evidence as the multi-modal features of the corresponding case; 3) constructing a knowledge graph of a judicial application scene based on the multi-modal features; driving multi-modal data fusion based on the knowledge graph and constructing a multi-modal evidence chain of each entity; 4) quantifying the integrity of the knowledge graph and the reliability of the multi-modal evidence chain according to a preset index, marking abnormal nodes in the knowledge graph according to a quantification result, and adjusting the abnormal nodes; 5) generating a structured knowledge graph according to the knowledge graph and creating a dynamic desensitization report; edges in the structured knowledge graph represent relationships between legal entities, and each relationship is bound with a multi-modal evidence chain.

The invention relates to a cloud edge cooperative computing framework and processing method for multi-modal data stream fusion processing, and the method comprises the following steps: S1, carrying out the noise suppression based on an original data stream collected by an edge computing node through employing an improved Wiener filtering algorithm, achieving the signal denoising through the adaptive threshold wavelet transformation, and obtaining a cloud edge data stream; and a timestamp alignment technology is utilized to solve the problem of time delay difference of multi-modal data, and a space-time alignment purified data stream is generated. Through combination of the improved Wiener filtering algorithm and the adaptive threshold wavelet transform, the noise suppression efficiency of the original data stream is significantly improved, the timestamp alignment technology effectively solves the time delay difference of the multi-modal data, the generation of the space-time alignment purified data stream ensures that the subsequent processing has a unified time sequence benchmark, and the efficiency of noise suppression of the original data stream is improved. The space-time attention fusion network adopts a collaborative architecture effect of a bidirectional gating circulation unit and a lightweight 3D convolutional network.

The invention relates to the technical field of artificial intelligence, can be applied to business scenes of financial science and technology, medical health and the like, and discloses a decision-making method, device, equipment and medium based on multi-modal semantic alignment. Executing cross-modal alignment by taking the voice semantic map as a reference to generate associated information, fusing the voice features, the visual features, the action features and the associated information to generate a fusion feature vector, inputting a decision network to generate a decision feature vector and generate a task execution instruction, obtaining execution feedback information of the task execution instruction, and updating the decision network. According to the method, input is dominated by voice instructions, visual features, action features and semantic map depth alignment and fusion are combined, input naturalness and multi-modal data analysis and decision-making efficiency are improved, and interaction adaptability and decision-making accuracy of the model in a complex scene are improved.

The invention discloses a cross-border e-commerce compliance intelligent auditing platform and a multi-language contract analysis method, and relates to the field of cross-border contract compliance auditing. In the multi-modal data access step, customs codes, laws and regulations and other multi-source data are collected, and 18 kinds of language contract texts are analyzed; in the cross-language semantic alignment step, a knowledge graph is constructed, and multi-language legal concept mapping is achieved; in the compliance risk reasoning step, a rule engine and an agent cooperatively check a contract, and the compliance conclusion confidence is calculated; the dynamic risk assessment step adopts an LSTM network to analyze historical data and predict a risk trend; in the multi-language report generation step, a multi-format bilingual or multilingual report is generated based on a template engine, encrypted and archived. According to the invention, cross-border contracts are audited efficiently and intelligently, dynamic adaptation laws and regulations are analyzed in multiple languages, compliance risks are identified accurately, and a multi-language report is generated quickly; therefore, the checking efficiency is improved, the manual workload is reduced, the compliance risk is reduced, and the enterprise cross-border business competitiveness and the risk response capability are enhanced.

The invention discloses a multi-modal visual fusion complex scene small target detection tracking method and system, and relates to the technical field of unmanned aerial vehicle target tracking, and the method comprises the steps: employing a visible light camera, an infrared thermal imager and a laser radar sensor which are carried on an unmanned aerial vehicle platform, and synchronously collecting RGB images, thermal infrared images and point cloud data; the consistency of the multi-modal data is ensured through data preprocessing and space-time alignment; constructing a lightweight double-branch network to extract multi-scale features, generating a fusion feature map by adopting adaptive weighted fusion, and generating depth information by utilizing point cloud to assist in scale estimation; a small target detection head is designed based on the fusion feature map, and precise detection is realized in combination with a feature pyramid network, adaptive scale prediction and a context awareness suppression mechanism; furthermore, through multi-mode cooperative tracking, including target association, spatio-temporal context modeling, trajectory prediction and a re-detection mechanism, tracking continuity is ensured.

The invention discloses an accurate health risk early warning analysis system and method based on multi-modal medical data fusion. The system comprises a multi-source data acquisition module, a preprocessing module, a dynamic fusion module, a risk assessment module, an interpretability module and a dynamic early warning module. According to the method, multi-modal data are collected, feature vectors are generated through preprocessing and cross-modal fusion, a comprehensive health risk index is calculated through a double-flow model (time sequence LSTM + static GNN), abnormal association is analyzed in combination with causal reasoning, a threshold value is dynamically adjusted, grading early warning is triggered, and finally the model is optimized through reinforcement learning. According to the scheme, deep fusion and dynamic evaluation of multi-modal data are achieved, the accuracy, timeliness and interpretability of risk early warning are improved, the method is suitable for scenes such as chronic disease management and intensive care, and powerful support is provided for clinical decision making.

The invention discloses a method for constructing a cross-modal knowledge graph, and relates to the technical field of knowledge graphs, and the method comprises the steps: obtaining multi-source heterogeneous data, and carrying out the collection, structural analysis, modal recognition and classification, cleaning and standardization processing, so as to form structured multi-modal data; extracting entities and relationships of the identified modals from the structured multi-modal data, and summarizing the entities and relationships to form a multi-modal knowledge element set; mapping different modal entities in the multi-modal knowledge element set to a unified semantic space, and generating a unified entity relationship set through semantic matching, alignment and fusion; and normalizing the data into knowledge triples, and storing and organizing the knowledge triples through a graph database to form a cross-modal knowledge graph. According to the method, the problems of difficulty in multi-modal heterogeneous information alignment and difficulty in entity relationship extraction can be relieved, semantic association is enhanced, and knowledge graph integrity and multi-scene adaptability are improved.

The invention relates to the technical field of multi-modal data, provides a pilot competency dynamic evaluation method, system and device based on multi-modal data and a storage medium, and solves the problems of low accuracy of pilot competency evaluation and poor pertinence of training guidance. The method comprises the steps of collecting flight control data, physiological signal data, psychological assessment data, subjective scale data and international civil aviation organization core competency information, performing alignment fusion on multi-source data by adopting a time synchronization algorithm, and identifying an attention fixation mode based on a hidden Markov model. And constructing a workload index by fusing a subjective scale and a physiological entropy value, inputting the multi-modal features into a pre-trained competency assessment model, outputting three levels of psychological assessment indexes including a basic ability layer, a dynamic presentation layer and a risk early warning layer, and finally generating a personalized training report. According to the invention, the accuracy of pilot competency evaluation and the pertinence of training guidance are improved.

The invention provides a perception collaborative decision-making method and system based on multi-modal heterogeneous data fusion, and relates to the technical field of artificial intelligence, and the method comprises the steps: inputting a global environment situation perception graph into a pre-trained multi-target collaborative decision-making model; the multi-target collaborative decision-making model forms a multi-target decision-making feature set by analyzing the resource entities and the incidence relation in the graph; based on the multi-target decision feature set, decision optimization is carried out to obtain a comprehensive collaborative scheduling scheme; performing instruction analysis and packaging on the comprehensive collaborative scheduling scheme to obtain an executable instruction sequence; and issuing the executable instruction sequence to a corresponding decision node and a control terminal in parallel through a distributed communication architecture to complete real-time scheduling of resources and collaborative issuing of control instructions. According to the invention, by constructing a linkage mechanism of multi-modal data fusion, dynamic environment perception and collaborative decision execution, intelligent perception and quick response to a complex environment are realized.

The invention discloses a data center machine room AI energy-saving control method and system, a digital twin model of a machine room operation state is constructed through a holographic perception and heterogeneous data fusion technology, centimeter-level monitoring of an equipment state and environmental parameters is realized, and the system integrates a laser radar array, an acoustic sensor and a gas sensor network. The time-space alignment of multi-modal data is completed by combining edge computing nodes, holographic mapping including thermodynamic characteristics, vibration characteristics and gas leakage risks is formed, historical temperature control strategy characteristics are extracted by adopting a variational auto-encoder based on a dynamic strategy generation mechanism of generative artificial intelligence, and a load trend is predicted by combining a long-short-term memory network. Constructing a self-adaptive strategy pool; the multi-agent reinforcement learning framework enables temperature control, equipment scheduling and power grid response to form game optimization, the strategy robustness in a complex scene is improved, and the system innovatively fuses power grid real-time electricity price and carbon transaction data so as to establish a multi-target decision system.

The invention relates to the technical field of multi-modal data processing, and discloses a multi-modal threat perception method and system based on a space-time diagram neural network, and the method comprises the steps: obtaining a multi-modal original data set in a vehicle insurance claim settlement link, and carrying out the business relation mining and space-time dynamic analysis, and obtaining an entity space-time relation diagram; inputting the entity space-time relation graph into a space-time graph neural network for space-time fusion to obtain a node threat embedding vector; performing graph contrast learning and cross-modal feature discrimination on the node threat embedding vector to obtain a vehicle insurance threat feature vector; and carrying out fraud space-time propagation modeling based on the vehicle insurance threat feature vector, and generating a vehicle insurance threat blocking strategy, the method can accurately predict a propagation path and an influence boundary of gang fraud in a vehicle insurance ecological network, and identifies potential threats and starts prevention measures before a fraud behavior is completely displayed.

The invention discloses a cloud edge cooperation method for AI intelligent Internet of Things equipment data processing, and relates to the technical field of data processing, and the method comprises the steps: S1, intelligent data collection, S2, edge side AI preprocessing, S3, edge-cloud end cooperation reasoning, S4, intelligent data transmission, S5, cloud end AI big data analysis, S6, real-time feedback and self-optimization, S7, adaptive resource scheduling, and S8, full-link visualization. Through AI-driven dynamic sampling and multi-modal data fusion, the efficiency and precision of data acquisition are optimized, redundancy or omission caused by fixed sampling is avoided, meanwhile, the transmission load is reduced, layered task dynamic unloading and intelligent transmission protocol optimization are achieved, the flexibility and stability of cloud edge collaboration are improved, and the cloud edge collaboration efficiency is improved. Manual intervention is reduced through a real-time feedback and self-optimization mechanism, the autonomy of the system is enhanced, and the interpretability and fault diagnosis capability of the system are remarkably improved through a visual panel and a causal reasoning model.

The invention relates to the technical field of intelligent early warning of digital twinning, and particularly discloses a slope multi-physics field fusion early warning decision-making system based on digital twinning, which is characterized in that physical monitoring data representing the macroscopic state of a slope and microscopic physical response signals reflecting internal damage evolution are synchronously acquired through a multi-modal data sensing module; space-time alignment, standardization and cross-modal fusion analysis are carried out through a damage eigenstate extraction module, and a unique eigendamage variable for quantitatively representing the real-time degradation degree of the material strength is interpreted; the twinborn self-evolution module takes the variable as a core observed quantity, and drives parameters and states of a slope mechanical model to be cooperatively and dynamically updated by adopting a data assimilation method, so that high-fidelity tracking of a digital model on physical reality is realized; and the prospective early warning decision module deduces a future spatio-temporal evolution path of the material strength parameters based on the calibrated model, and realizes graded early warning and intelligent decision support by combining Monte Carlo simulation and quantification of the instability risk probability.

The invention belongs to the technical field of power equipment, and discloses a power equipment fault early warning method based on multi-source data fusion, and the method comprises the steps: constructing multi-dimensional feature association through multi-modal data time-space association collection and hierarchical fusion driven by a knowledge graph; a space-time weight matrix is used for correcting sampling deviation, fault mechanism knowledge is combined to strengthen key feature contribution degree, false alarm and missing alarm caused by data isolation are effectively avoided, early recognition of hidden defects of equipment is realized, and global perception capability of early warning is improved. A meta-learning enhanced cross-equipment early warning model and reinforcement learning dynamic threshold decision are adopted, cross-equipment rapid adaptation under a small number of samples is realized through a ''meta-micro'' double-circulation mechanism, and a nonlinear law of fault evolution can be accurately described by combining a three-dimensional dynamic threshold matrix to balance an equipment state, an environment and an operation and maintenance strategy. The model generalization problem of different types of equipment in a complex environment is solved, and the adaptability to scenes such as load fluctuation and environment sudden change is improved.

The invention discloses a digital intelligent switch cabinet state comprehensive sensing system based on AI, and the system comprises a multi-source data collection module, a data preprocessing and synchronization module, an edge calculation feature extraction module, an AI intelligent fusion recognition module, an expert rule diagnosis module, and a cloud comprehensive evaluation and decision module. Various types of sensors are deployed to respectively acquire environmental parameters, electrical parameters and partial discharge signals generated in the operation process of the switch cabinet to form an original multi-modal data stream. The method has the advantages that the recognition precision and response speed of the complex operation state of the switch cabinet are improved, hidden faults under multi-modal data mismatch can be effectively found, and the misjudgment and missed judgment risks are reduced. Meanwhile, a closed-loop diagnosis system is constructed, intelligent evaluation and interpretable feedback of fault types, positions and trends are achieved, scientificity and reliability of operation and maintenance decisions are enhanced, and the method is suitable for intelligent upgrading of an electric power system.

The invention discloses a traffic large model construction and decision-making method and device based on multi-modal two-way map reasoning, and the method comprises the steps: constructing a multi-modal data set of a text, an image and a track, generating fusion features through spatial-temporal clustering and cross-modal Transform coding, carrying out the two-way map reasoning in combination with a traffic knowledge map, and carrying out the decision-making of the traffic large model. The method comprises the following steps: generating an embedded representation through a forward graph neural network, reversely mapping a decision scheme generated by a language model to a graph to verify consistency, outputting knowledge to enhance embedding, fusing multi-modal features and knowledge embedding by adopting an LoRA multi-task joint fine tuning technology, adapting to traffic field tasks, deploying a real-time inference engine, and carrying out real-time inference on the traffic field. And processing the dynamic data flow through an aging perception attention mechanism, and outputting traffic event identification, path planning and scene question and answer results in parallel. Compared with the prior art, the method has the advantages that the problems of insufficient multi-source heterogeneous data fusion, low knowledge utilization efficiency and poor real-time decision consistency can be solved, and the semantic understanding and decision accuracy of the traffic large model is effectively improved.

The invention relates to the technical field of automatic driving, and discloses an automatic driving lane changing trajectory planning method based on deep learning, and the method comprises the steps: carrying out the data collection and preprocessing of a multi-modal sensor; performing spatial feature extraction and time sequence modeling on the preprocessed multi-modal data by adopting a CNN-LSTM hybrid architecture, performing feature fusion through an attention mechanism, and outputting a first feature extraction vector; taking the detected vehicles as graph nodes to construct a traffic graph, learning an interaction relationship between the vehicles through a graph attention network and a message passing mechanism, and calculating a scene urgency score and a safety score; generating a lane changing decision based on the deep Q network and the strategy gradient; and generating a trajectory based on the generative adversarial network. The technical problems that an existing lane changing track planning method cannot adapt to the dynamic traffic environment, lacks the ability of understanding complex multi-vehicle interaction and is difficult to balance safety and urgent conflict requirements are solved, and intelligent, safe and efficient automatic driving lane changing track planning is achieved.

The invention discloses a data annotation method and system of a collaborative computing architecture based on quantum computing, and belongs to the field of data annotation. The method comprises the steps that S1, multi-modal data are input and preprocessed; s2, extracting features of each mode after preprocessing; s3, coding the features of each mode into a quantum state, and carrying out mode fusion; s4, performing label reasoning on the quantum state after modal fusion, and performing label constraint optimization by using a quantum approximate optimization algorithm; s5, based on a quantum Bayesian network or an approximate causal graph generation method, generating explanation according to a modal contribution causal path, and deducing marginal contribution of each modal to final label prediction by using a joint probability measurement result; and S6, outputting a labeling result. According to the method, a quantum-classical cooperative computing architecture is designed, the efficiency and accuracy of multi-modal data labeling are remarkably improved, the interpretability, the distributed processing capacity and the high-dimensional feature modeling capacity of the system are enhanced, and a brand new solution thought is provided for development of the multi-modal labeling technology.