5110 results about "Multimodal data" patented technology

The invention provides an industrial environment monitoring and accident prediction method fusing multi-modal data, and relates to the technical field of data processing, and the method comprises the steps: carrying out the semantic collection and causal association preprocessing of multi-modal heterogeneous data collected in real time through constructing a dynamic industrial knowledge graph; a customized deep learning model is adopted to extract deep abstract features of each mode, and weak signals and potential risks are accurately represented and uncertainty is quantified; a high-fidelity digital twin model is utilized to drive a deep reinforcement learning algorithm, and dynamic optimization and verification are performed to generate a multi-level and multi-target preventive intervention strategy combination; an intervention strategy is executed through an edge-end-cloud three-layer collaborative intelligent architecture, and online learning and system sustainable evolution are realized by using a closed-loop data feedback mechanism. According to the method, the sensing and early warning capability of the early weak and complex abnormal state of the industrial environment can be remarkably improved, the accident evolution path is accurately predicted, and credible explanation is provided.

The invention relates to the technical field of equipment fault diagnosis, and discloses an equipment fault diagnosis and prediction method based on deep learning, and the method comprises the following steps: S1, collecting multi-modal data in real time through a plurality of sensors installed on equipment; s2, preprocessing the collected data; s3, constructing a hybrid deep learning model; s4, dynamic weighted fusion is performed on the features of different modal data by using an attention mechanism, and comprehensive feature representation is generated; s5, using the marked fault data and normal data to supervise and train the model; s6, inputting equipment operation data acquired in real time into the trained model, and judging the state of the equipment; and S7, generating a potential fault early warning signal based on a prediction result of the model. A piezoelectric vibration sensor and a thermal infrared imager are arranged on a motor bearing through vibration, temperature and sound sensors, vibration waveforms, thermal imaging slices and time-frequency diagrams are synchronously captured, and composite state characteristics such as mechanical wear and temperature anomaly of equipment are comprehensively reflected.

The invention belongs to the technical field of bridge diagnosis, and discloses a bridge crack intelligent diagnosis system based on multi-modal data fusion. By fusing multi-source data such as visual images, sound wave detection and vibration signals, comprehensive perception and characterization of crack features are realized; constructing a bridge crack characteristic spectrum diagram by adopting a cross-modal feature extraction and heterogeneous feature coding technology; generating a crack evolution situation map based on space-time correlation analysis and knowledge graph construction; the robustness of the system in a complex environment is improved through environmental adaptability feature enhancement and multi-scale characterization; constructing a bridge safety risk hypergraph in combination with multi-dimensional risk analysis and multi-agent collaborative diagnosis; analyzing and revealing a crack evolution mechanism by applying a causal relationship; and finally, through dynamic fusion and uncertainty quantification, a crack intelligent diagnosis comprehensive report is generated. According to the system, the limitation of traditional single-mode diagnosis is broken through, and dynamic prediction and accurate risk assessment of fracture evolution are realized.

The invention provides a multi-modal enterprise credit risk assessment method based on a knowledge graph, which integrates data such as enterprise relationships, industry policies and supply chain information by constructing an enterprise financial knowledge graph, processes entity static attributes and associated information by using a multi-modal embedding technology, captures the associated information in combination with a heterogeneous graph neural network, and evaluates the credit risk of an enterprise. And the dynamic space-time attention mechanism mines time and space features of the time series data, identifies a core risk conduction path based on an attention weight, and finally fuses graph-level features, dynamic space-time features and business rules to output a structured evaluation result. According to the method, multi-modal data is effectively integrated, the problem of incidence relation modeling deficiency is solved, deep fusion of enterprise multi-source data and accurate extraction of risk features are realized, and the accuracy and interpretability of enterprise credit risk assessment can be effectively improved.

The present invention provides a system and method for causality-augmented generative intelligence capable of autonomously discovering non-obvious actionable insights from heterogeneous and multimodal data sources. The system integrates a data ingestion unit for semantic and temporal harmonization of structured and unstructured datasets, a causal inference processor for constructing a dynamically evolving directed causal knowledge representation using perturbation-based validation, a latent representation processor that combines multimodal semantic embeddings with causal parameters to generate fused latent vectors, and a generative insight processor utilizing causally constrained generative reasoning to synthesize hypotheses anchored to verified cause-effect dependencies. A validation processor performs counterfactual assessment and observational verification to ensure retention of only those insights that remain consistent with causal ground truth.

The invention relates to a knowledge graph construction method and system based on a large language model, and the method and system achieve the automatic construction and dynamic maintenance of a knowledge graph through multi-modal data fusion, reinforcement learning and comparative learning joint optimization, teacher-student model knowledge migration, time sequence dynamic analysis and an incremental updating mechanism. Constructing a reinforcement learning framework, and taking accuracy and integrity as reward indexes to train a large model to extract an entity relationship; a large-scale knowledge graph is used as a teacher model, and conflict resolution and semantic alignment of newly added knowledge and an existing graph are realized through a graph attention network; a verification rule is dynamically generated based on historical data and domain knowledge, and relation periodicity and mutation points are detected in combination with Fourier transform and a CUSUM algorithm; and finally, generating a traceable knowledge graph through incremental updating and version control. And the multi-modal data processing precision, the entity relationship extraction dynamic adaptability and the knowledge graph maintenance efficiency are improved.

The invention discloses an agent autonomous decision control method based on multi-modal data fusion. The method comprises the following steps: S1, synchronously collecting multi-source heterogeneous data; s2, dynamic weight adaptive fusion is carried out; s3, generating a task-driven decision; and S4, performing autonomous decision closed-loop optimization. According to the method, through dynamic weight distribution and space-time correlation modeling, the problems of heterogeneity and environment adaptation in multi-modal data fusion are solved; furthermore, a risk-sensitive reinforcement learning framework and a closed-loop feedback mechanism are combined, so that full-link cooperative control from data fusion, strategy generation to optimization execution is realized. In the mechanism level, the method breaks through the limitations of static fusion, single-target optimization and offline training, can adapt to a dynamic environment, ensures that the intelligent agent is in a complex scene such as noise interference, illumination abrupt change and task emergency switching, and meets the requirements of decision-making efficiency, safety and environment robustness at the same time.

The embodiments of the present disclosure belong to the technical field of data processing. Provided is a multimodal scenario risk determination method based on a generative AI large language model. The method specifically comprises: step 1, acquiring multimodal data to form a target data set, wherein the multimodal data comprises visual data and text data; step 2, using an ALBEF algorithm to extract key features corresponding to the target data set, and fusing the key features into a comprehensive scenario representation; and step 3, on the basis of a preset safety index and a large language model, evaluating a risk degree corresponding to the comprehensive scenario representation, comparing the risk degree with a risk threshold, and determining whether the scenario corresponding to the comprehensive scenario representation is a high-risk scenario. By means of the solution in the present disclosure, a high-risk scenario can be rapidly recognized and identified, so as to provide a basis for taking emergency measures, thereby enhancing the real-time response capability.

The invention relates to the technical field of multi-modal data processing, and discloses a multi-modal sensor fusion inspection method and system, and the method comprises the steps: collecting the multi-modal original data of power equipment through a multi-modal sensor in an inspection robot, and constructing a feature vector set; performing adaptive weight calculation on the multi-modal sensor according to the feature vector set to obtain a sensor weight set; carrying out conflict identification and resolution on the multi-modal original data to obtain a fusion data set; performing abnormal feature extraction on the power equipment based on the fused data set to obtain an abnormal feature set; and carrying out routing inspection trajectory optimization based on the abnormal feature set to obtain a target routing inspection path sequence, and carrying out equipment state joint prediction in combination with historical equipment routing inspection data to obtain an equipment fault prediction result. And thus, more accurate equipment state joint prediction is realized.

The invention discloses a trajectory optimization method based on a multi-source heterogeneous positioning data fusion algorithm, and relates to the technical field of intelligent navigation and high-precision positioning, multi-modal data are acquired through a multi-modal sensor array, positioning redundancy of scenes such as tunnels and indoor scenes is enhanced, a weight distribution strategy is dynamically adjusted through an Actor-Critic network architecture, and the positioning accuracy is improved. The state space input comprises an environment semantic tag, a historical error sequence and a real-time noise variance, the output action space is continuous weight distribution of each data source, a multi-target reward function optimization strategy is combined, scene adaptability is realized, a local SLAM map, inertial navigation error parameters and a weight distribution strategy are shared in real time based on a V2X protocol, and the real-time performance of the system is improved. According to the method, a single device accumulative error is compensated by using adjacent vehicle data, a terminal locally trains an error compensation model, parameters are uploaded to a cloud end through differential privacy encryption, the cloud end adopts a FedAvg algorithm to aggregate a global model and issue the global model, the error difference between devices is inhibited, and dynamic road network updating and scene differentiation model distribution are supported at the same time.

The invention relates to the field of semiconductor packaging material defect detection, in particular to a semiconductor packaging material defect detection method based on deep learning, which comprises the following steps: acquiring a surface image, and extracting a two-dimensional contour and a feature point set; preprocessing the image, and separating a packaging material main body area; constructing a two-dimensional defect identification model based on Transform, and outputting a two-dimensional detection result; scanning suspected and unknown defect areas to obtain three-dimensional point cloud data, and extracting geometric and texture features; fusing two-dimensional and three-dimensional data through a space-time alignment model; utilizing the multi-modal fusion model to output defect positions and types; and evaluating the defect importance based on the material node connectivity and the stress distribution, and generating a visual detection report. According to the invention, high-precision detection of semiconductor packaging material defects is realized, the defect identification rate, the positioning precision and the detection efficiency are improved through multi-modal data fusion and a deep learning model, and a visual report can be generated based on material structure quantification defect importance.

The invention relates to a network public opinion intelligent classification and emergency decision-making system based on multi-modal fusion and dynamic evolution, and belongs to the field of network public opinion monitoring and big data analysis and artificial intelligence. The system comprises a multi-source data acquisition and preprocessing module used for crawling multi-modal data, constructing a propagation path map after preprocessing, and identifying key propagation nodes; the multi-dimensional classification engine module is used for carrying out conflict intensity quantification on public opinion events and dynamically updating a rule word bank to keep the adaptability of a conflict intensity quantification model; the event graph construction and anomaly detection module is used for constructing a public opinion propagation path and public opinion event generality logic chain mode, monitoring public opinion propagation speed and giving an alarm; the stakeholder dynamic risk assessment module is used for finely classifying network public opinion participants, providing a basis for differential propagation intervention and simulating public opinion evolution to carry out risk simulation; and the intelligent decision-making and emergency response module executes different levels of emergency measures based on the risk index according to the hierarchical response strategy.

The invention discloses an archive knowledge base construction and retrieval method and system based on multi-modal data fusion. The method comprises the steps that heterogeneous archive data are cleaned, image features are extracted through CNN, text features are extracted through Transform, audio is converted into text and then subjected to similarity, a unified feature vector is generated, and metadata is constructed according to archive code association; creating a graph database instance, defining nodes and relationship types, importing entities and relationships, and storing feature vectors and metadata; the features are mapped to a high-dimensional shared semantic space, positive and negative sample pairs are constructed to update embedded layer parameters, self-attention is used in modalities, a shared attention mechanism is used between modalities, weights are adjusted according to archive features, and unified knowledge representation is generated; segmenting the steering quantity of the multi-modal data, storing the steering quantity into a database, and adopting hierarchical indexing and optimizing as required; related document fragments are retrieved through RAG technology vectors, answers are generated with the help of a large language model, and session feedback is provided. The file retrieval efficiency and accuracy are improved.

The invention provides a power station equipment state real-time monitoring and diagnosing method and system based on cloud edge collaboration, and the method comprises the steps: adjusting a data collection period dynamically determined based on an adaptive sampling frequency adjustment algorithm, and collecting a vibration signal, a temperature signal and a current signal through a multi-source heterogeneous sensor array disposed in a power station equipment body; carrying out preprocessing by utilizing the edge computing node, generating a compressed feature vector, and uploading the compressed feature vector to a cloud end through an MQTT protocol; a multi-modal data fusion analysis module is started through a cloud, a three-dimensional evaluation matrix of the equipment health state is constructed in combination with historical operation data and environmental parameters of the equipment, and a calculation task distribution strategy between an edge calculation node and the cloud is adjusted in real time according to an evaluation result of the three-dimensional evaluation matrix. Abnormal mode recognition based on a deep residual network and fault source tracing double-channel analysis based on a physical model are executed, fault types and fault reasons are diagnosed, and the accuracy and timeliness of fault diagnosis are guaranteed.

The invention relates to the technical field of AIGC content generation, discloses an AIGC content generation method and system based on multi-modal fusion, and aims to solve the problems of decentralization, low efficiency and insufficient originality of a traditional content generation tool. Multi-modal data such as texts, images, videos and audios are integrated, user intentions are analyzed in combination with intelligent retrieval and a domain knowledge base, automatic generation from multi-modal input to high-quality creative content is achieved, a cross-modal collaborative generation technology is adopted, semantic features are dynamically aligned, and logically coherent content is generated. The content emotional value is enhanced through an emotional analysis and dynamic optimization strategy, the homogenization bottleneck is broken through, meanwhile, an automatic quality evaluation and format adaptation mechanism is integrated, deep application of scenes such as text travel, advertisement, e-commerce and interactive network television service is supported, marketing copywriting, short videos and cross-platform distribution schemes can be efficiently generated, and the market competitiveness is improved. And the content production efficiency and the creativity transmission are obviously improved.

The invention discloses a logistics robot path planning method based on multi-modal perception, and relates to the technical field of robot path planning. Laser radar, visual camera and IMU data are fused, and environment state feature vectors are generated through multi-modal data synchronization and space-time alignment; the method comprises the following steps: analyzing environmental semantics by using models such as PointPill and YOLOv8, extracting dynamic characteristics, and identifying obstacles; constructing a space-time risk field, searching a path by a space-time algorithm, converting path points into a continuous trajectory, and optimizing the continuous trajectory; deviation is evaluated in real time, dynamic re-planning is triggered, and multi-robot cooperation and environment semantic understanding are included. Through multi-mode perception fusion, hierarchical planning, multi-target optimization and a cooperation mechanism, the obstacle detection accuracy and the obstacle avoidance success rate are improved, the path planning time is shortened, the energy consumption is reduced, the multi-robot conflict is reduced, the task efficiency is improved, the environment semantic understanding and task adaptive ability is enhanced, and the method is suitable for scenes such as intelligent storage and the like and has wide application prospects. The automation level is improved.

The invention relates to the technical field of artificial intelligence, can be applied to business scenes of financial science and technology, medical health and the like, and discloses a multi-modal causal reasoning and interpretation method, device, equipment and medium, and the method comprises the steps: obtaining original data streams of at least two different modals, and extracting modal features; a cross-modal attention mechanism is utilized to fuse modal features, and causal features are extracted through feature distillation; constructing a dynamic causal graph based on causal features, and updating an edge weight through a causal intensity function; identifying the causal relationship in the dynamic causal graph and performing anti-factual reasoning verification to evaluate the reliability of the causal relationship; and generating a causal interpretation result in combination with the dynamic causal graph and the causal relationship reliability. According to the method, the multi-modal data are fused, the causal features are extracted, and dynamic causal graph updating and anti-factual reasoning verification are combined, so that reliable modeling and explanation of the causal relationship in a complex scene are realized, the defects of single modal or simple fusion in the prior art are overcome, and the accuracy and interpretability of causal reasoning are improved.

The invention relates to the technical field of tunnel power supply and distribution, in particular to a tunnel modular prefabricated cabin power supply and distribution intelligent substation self-adaptive regulation and control system based on edge calculation. Comprising an edge calculation and AI decision-making unit which is used for realizing rapid acquisition, processing and instant decision-making of tunnel power supply and distribution multi-dimensional data, generating a power supply and distribution adaptive regulation and control strategy by deploying calculation resources and a machine learning algorithm at edge nodes close to a data source, and converting the strategy into an executable regulation and control instruction; a cloud platform collaborative management unit; and an intelligent sensing and internet-of-things unit. According to the invention, hierarchical decision control of the tunnel power supply and distribution system is realized by constructing a hybrid architecture of edge computing and cloud platform collaboration and a priority judgment mechanism; according to the invention, multi-modal data are integrated through the multi-protocol communication link module and the full-scene data fusion analysis module, and data association analysis is realized through Kalman filtering, D-S evidence theory and other algorithms.

The invention discloses a mechanical equipment state monitoring method and system based on multiple sensors, and the method comprises the five core steps: multi-modal data collection and preprocessing, dynamic feature fusion, adaptive threshold diagnosis, digital twin fault tracing and predictive maintenance decision. All-domain coverage of equipment is realized through a three-layer sensor network architecture, the problems of data synchronization and interference resistance are solved by utilizing a temperature and vibration integrated sensor, deep fusion and anomaly detection of multi-source data are realized in combination with an attention mechanism, a Gaussian mixture model, a three-dimensional convolutional neural network and the like, and finally a precise maintenance strategy is generated through digital twinning and reinforcement learning. The multi-sensor-based mechanical equipment state monitoring system comprises a sensor network layer, an edge computing layer, a cloud platform layer and a man-machine interaction layer, supports federated learning to protect data privacy, improves real-time diagnosis capability through edge-cloud collaboration, and enhances a reality interface to realize intelligent operation and maintenance interaction.

The invention provides a micro-grid fault diagnosis and dynamic recovery method based on deep reinforcement learning, and the method focuses on the multi-modal features of key nodes through a graph attention mechanism, extracts fault features through a multi-layer graph attention layer, and captures the spatial dependence relation of micro-grid nodes to recognize a fault propagation path. Meanwhile, spatial features and historical multi-modal data are fused with the help of a gating circulation unit, space-time joint feature representation is constructed, pre-fault symptom time sequence evolution is captured, intermittency and early fault detection capacity are enhanced, the multi-modal feature data fusion problem is solved, and high-precision fault diagnosis is achieved. A knowledge distillation technology is adopted to deploy a lightweight student model at edge equipment, millisecond-level emergency response is realized, fault diffusion is prevented, and meanwhile, the accuracy of diagnosis and repair strategies is guaranteed. The optimal repair strategy is generated at the cloud through the teacher model by using the global data, the system can adapt to the topological change of the micro-grid and novel faults, and the fault processing capability is continuously improved.

The invention relates to the technical field of power line inspection, and provides a line holographic anomaly detection method and system based on cross-modal intelligent cooperation. The method comprises the following steps: acquiring multi-modal data; performing cross-modal fusion to generate an association tensor; the abnormal joint reasoning uses a time sequence diagram neural network and reinforcement learning to output abnormal confidence; the dynamic knowledge driven decision adaptively adjusts a detection threshold through Bayesian calculation and transfer learning; local real-time response is realized through layered edge calculation; and multi-target collaborative optimization feedback improves the detection precision. The system is composed of a multi-mode perception fusion layer, an intelligent analysis layer, an edge execution layer and an optimization control layer. According to the method, the problems of multi-modal information isolation, response delay and environmental adaptability are solved, and the defect detection rate and the system robustness are remarkably improved.

The invention discloses an industrial equipment fault prediction method based on multi-modal data, and belongs to the technical field of specific calculation models, and the method comprises the steps: carrying out the preprocessing according to the collected multi-modal data of the operation of industrial equipment, so as to unify the format of the multi-modal data, and obtaining the structural data; extracting features of the structured data one by one according to data categories, and obtaining a multi-modal fusion feature through a dynamic fusion mechanism; according to the multi-modal fusion features, a fault prediction classification score is obtained through a deep neural network model to perform fault prediction; and when the drift parameter of the multi-modal data is greater than a preset threshold value, performing incremental training on the deep neural network model through the multi-modal data collected in real time to update parameters of the deep neural network model. Through multi-modal data unified processing, dynamic feature fusion, deep neural network modeling and an online learning mechanism, the problems of insufficient multi-modal data fusion, prediction uncertainty quantization deficiency, poor model adaptability and the like are solved.

The present disclosure relates to a multimodal computing-based early intelligent graded screening system for a brain disease. The system comprises: a multimodal data acquisition unit, configured to acquire multimodal data of a target patient under a specified screening grade to form a screening data set; a multimodal feature generation, completion, fusion and calculation unit, configured to generate and complete feature data of modal features in the screening data set to obtain complete modal features, and extract pathological features for calculation to obtain a first screening result; a knowledge-based intelligent screening unit, configured to encode the feature data of the modal features in the screening data set into corresponding graph structure data features, use a multimodal association graph optimized by expert knowledge to match the graph structure data features to obtain knowledge-based association features, and obtain a second screening result on the basis of the knowledge-based association features; and an intelligent graded screening unit, configured to carry out weighted calculation on the first screening result and the second screening result to obtain a final screening result. The present disclosure achieves accurate early screening of brain diseases of patients.

The invention relates to the technical field of state monitoring of wind generating sets, and discloses a fault early warning and service life prediction method and system for a wind generating set, and the method comprises the steps: obtaining first state data, second state data and image data of a target wind generating set, and forming multi-dimensional data; fusing the multi-dimensional data by using a multi-modal fusion model to obtain multi-modal data fusion features of the target wind generating set; and performing fault early warning and / or life prediction on the target wind generating set based on the multi-modal data fusion features. By integrating the multi-modal data, the problem that fault features are difficult to comprehensively capture by a single data source is solved, fault early warning and service life prediction are performed by utilizing the multi-modal data fusion features, the false report and missing report rate of faults is reduced, accurate quantitative prediction of the remaining service life of the wind generating set is realized in combination with the data driving model, and the prediction efficiency is improved. By improving the accuracy of fault early warning and life prediction, the wind generating set is effectively operated and maintained in advance.

The invention discloses a collaborative personalized learning system and method based on a large model, and relates to the technical field of collaborative learning, and the system collects and fuses multi-modal data, generates a student state vector, plans a personalized learning path based on the state and a knowledge graph, and generates multi-modal explanation content according with the student style. The method comprises the following steps: extracting a student reasoning path, identifying thinking deviation, analyzing task performance, generating a path and content adjustment suggestion, identifying a motivation state, triggering an intervention strategy, receiving a teacher strategy, and realizing path and style co-construction. The multi-modal content of the matched style is generated, adaptive intervention is achieved in combination with inference analysis and emotion adjustment, and the learning efficiency and the teaching response intelligent level are improved.

The invention relates to a multi-modal data and RAG technology fused power design knowledge base construction method, and belongs to the technical field of power software development. The method comprises the following steps: carrying out collection and information extraction on multi-source heterogeneous original data; the method comprises the following steps of: constructing a multi-dimensional knowledge element structure containing parameters, specifications and case relationships by carrying out classification, specialized and precise processing and cross-modal association on data; based on a vector, graph and relational database mixed storage architecture, semantic vector efficient retrieval, knowledge graph relation management and business data synchronization are achieved respectively; and a dynamic optimization result is subjected to hybrid retrieval, a dual-drive reasoning mechanism outputs compliance conclusions and bases, and a retrieval enhancement generation service ensures that output contents conform to specifications. And systematic management and intelligent application of the electric power design knowledge are realized.

The invention discloses an integrated power distribution cabinet energy efficiency dynamic optimization method based on multi-modal data fusion, and relates to the technical field of intelligent power grids. The problems of failure of an energy efficiency optimization model and poor long-term operation stability caused by data time sequence misalignment and error accumulation of a multi-source sensor in the prior art are solved. According to the scheme, the time offset is dynamically corrected through the adaptive time sequence deviation prediction model; compensating missing data by adopting a non-uniform time step reconstruction algorithm and Kalman filtering; detecting an error drift trend through an exponentially weighted moving average model, updating a feature weight, and inhibiting long-term error accumulation; constructing a self-adaptive time sequence attention fusion network model, and fusing physical constraints and a data driving mechanism to generate an optimization decision; bayesian optimization is utilized to quantify parameter uncertainty, closed-loop feedback execution data is carried out, and model parameters are dynamically updated; according to the invention, the precision, long-term stability and equipment safety of energy efficiency optimization of the power distribution cabinet are remarkably improved, and efficient and reliable operation under multi-physics field coupling constraint is ensured.

The invention belongs to the field of artificial intelligence, and provides a dialogue interaction system based on multi-mode emotion perception and knowledge graph dynamic enhancement. A user edge terminal obtains multi-modal data, a lightweight Transform fusion network is adopted, the multi-modal data is converted into a fusion feature vector through cross-modal attention fusion and dynamic weight adjustment, and the fusion feature vector and historical conversations in a preset round are compressed in real time; the cloud service platform inputs the compressed data into DKGE, mining and fusing feature vectors and entity knowledge and emotional relations implied in historical dialogues in real time in the dialogue interaction process to update a dynamic knowledge graph, performing knowledge enhancement processing based on the dynamic knowledge graph, and constructing an initial reply prototype of the current round of interaction of the user; inputting the fusion feature vector and the updated dynamic knowledge graph into a dialogue strategy model, and determining a response strategy and a knowledge calling direction of the current round of dialogue; and the edge terminal generates real-time interaction reply information according to the initial reply prototype, the response strategy and the knowledge calling direction.

The invention provides an AI-based marketing video auditing method, and relates to the technical field of AI marketing video auditing, and the method comprises the steps: obtaining a multi-modal data original structure set, and extracting image semantic features, voice expression features, text semantic features and scene label information, and obtaining an image semantic feature set, a visual rhythm feature set, a voice expression feature set, a voice and picture synchronous association vector structure, a text semantic feature set and a subtitle semantic and image main body linkage relation graph. By constructing an image semantic feature set, a voice expression feature set, a text semantic feature set and a visual rhythm feature set and fusing the image semantic feature set, the voice expression feature set, the text semantic feature set and the visual rhythm feature set into a multi-modal content fusion feature tensor, unified modeling of an AI marketing video at visual, auditory and semantic levels can be realized; and subsequent microscopic consistency detection, compliance knowledge graph and emotion semantic conflict identification are effectively performed, so that full-link risk perception and accurate auditing of video contents are realized.

The invention provides a high-speed rail part crack real-time detection method, and belongs to the technical field of image detection based on computer vision. Obtaining a hyperspectral image, a visible light image, three-dimensional point cloud data and eddy current signal data of the surface of the high-speed rail part; performing spatial registration on the high-spectral image and the visible light image on the surface of the high-speed rail part, and performing time registration on the three-dimensional point cloud data based on the eddy current signal data; feature enhancement and standardization processing are carried out; performing feature extraction and fusion on the obtained standardized multi-modal data based on a multi-modal feature fusion network to obtain a fusion feature map representing crack details of the high-speed rail parts; based on a self-adaptive crack segmentation method, a crack contour is extracted from the point cloud, and then whether the part has a crack or not and the length and depth of the crack are calculated. According to the invention, three kinds of modal data are creatively integrated, the information dimension limitation of single-modal detection is broken through, and all-weather and non-contact intelligent efficient diagnosis of submillimeter cracks is realized under complex working conditions.