Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

20842 results about "Multi modal data" patented technology

Multi-modal data collection simply describes using more than one data-collection technology to accomplish a task. It can easily be argued that we have been doing multi-modal data collection for decades. Technically speaking, entering information on a keypad is a form of data collection,...

Medical decision-oriented multi-modal data dynamic fusion and labeling method and system

The invention provides a medical decision-oriented multi-modal data dynamic fusion and labeling method and system, and relates to the technical field of medical decision-making, and the method comprises the steps: collecting medical images, electronic medical records and physiological monitoring data of a patient, extracting multi-modal features such as a focus form, a disease type, a symptom, a treatment record and a physiological index, and carrying out the fusion and labeling of the multi-modal features; and constructing a dynamic fusion labeling model. According to the model, multi-modal features are coded, similarity is calculated, a feature association graph is constructed, feature aggregation is carried out, labeling is carried out in combination with a medical knowledge base, and a dynamic labeling vector is generated. And inputting the dynamic labeling vector into a medical decision support model, and based on similar case analysis and deep reinforcement learning, generating a diagnosis and treatment suggestion containing a treatment scheme and early warning information. By dynamically fusing multi-modal data and an online updating mechanism, the accuracy and efficiency of medical decision making can be improved, and personalized diagnosis and treatment are realized.
Owner:北京紫云智能科技有限公司

News scene three-dimensional reconstruction and visualization method based on multi-source remote sensing data

The invention provides a news scene three-dimensional reconstruction and visualization method based on multi-source remote sensing data, and relates to the technical field of three-dimensional reconstruction, and the method comprises the steps: extracting multi-modal data feature points through employing a spatial position weighted iterative nearest point algorithm and a tensor projection algorithm; after bidirectional attention deep neural network processing and local geometric consistency constraint screening, registration multi-source remote sensing data is generated, data enhancement is carried out through a variational auto-encoder network and a generative adversarial network, the enhanced data is used for depth information generation and texture feature migration, an initial three-dimensional grid model is generated based on a Poisson reconstruction algorithm, and the initial three-dimensional grid model is constructed; and carrying out topological optimization by combining multi-view projection consistency constraint and a grid optimization algorithm to obtain an optimized three-dimensional scene model, setting a space division function, carrying out space division, constructing a grade model and a depth region, generating a scene semantic segmentation map, and updating a visual scene.
Owner:XINHUA FUSION MEDIA TECH DEV (BEIJING) CO LTD

Intelligent operation and maintenance method fusing multi-modal data and active learning

The invention relates to the technical field of intelligent operation and maintenance, and discloses an intelligent operation and maintenance method fusing multi-modal data and active learning, and the method comprises the steps: deploying a hierarchical Internet of Things equipment network in a target operation and maintenance region, collecting a heterogeneous operation and maintenance data set, and generating an operation and maintenance feature data set through employing a unified building operation and maintenance service framework cooperating with a plurality of MCPs; inputting the operation and maintenance feature data set and the user feedback information into a deep learning intention recognition model for intention recognition and demand analysis to obtain structured user demand data and an operation and maintenance task priority sequence; according to the operation and maintenance feature data set and the structured user demand data, performing real-time evaluation on the equipment operation state to obtain fault risk early warning data; autonomous decision analysis is carried out through an agent type artificial intelligence engine, an equipment maintenance scheme and a resource scheduling scheme are generated, and then the problems that in traditional operation and maintenance, fault prediction is single, the model adaptability is poor, and prediction results are difficult to convert into effective decisions are solved.
Owner:SHENZHEN GEMDALE BUILDING ENG CO LTD

Intelligent access control management method and system based on multi-mode identification and Internet of Things technology

The invention discloses an intelligent access control management method and system based on multi-mode identification and an Internet of Things technology. According to the method, multi-modal data are collected through distributed Internet of Things equipment, synchronous alignment and self-adaptive preprocessing are carried out, and multi-scale features are extracted and dynamically fused. And carrying out deep feature fusion by adopting nonlinear dimension reduction, multi-order tensor decomposition, a graph convolutional network and a multi-head attention mechanism. And in combination with the dynamically updated identity template, realizing multi-factor identity verification, including biological feature matching, behavior pattern analysis and environment context verification. And determining a verification result through a self-adaptive decision threshold, and performing dynamic access control, self-adaptive people flow control and real-time abnormal event detection. The method improves the accuracy of identity recognition, system safety and adaptability, and is suitable for intelligent access control management in complex environments such as construction sites, parks and shopping malls.
Owner:NANJING SHENGFENG INFORMATION TECH CO LTD

Dynamic data pipeline construction method based on artificial intelligence and multi-modal data processing

The invention relates to a dynamic data pipeline construction method based on artificial intelligence and multi-modal data processing, and belongs to the technical field of data fusion treatment and artificial intelligence. According to the method, a multi-modal deep learning model is adopted, semantic analysis is performed on business documents, user requirements and function specification instructions, and key indexes, data requirements and logic constraints are extracted; constructing a multi-modal knowledge graph, performing unified representation and feature extraction on structured and unstructured data resources, and performing embedded learning by using a graph neural network; based on a deep semantic matching model and a hybrid recommendation system, realizing accurate matching of business requirements and data resources; modeling generation of a data processing flow into a Markov decision process by adopting a reinforcement learning strategy, and automatically generating an optimized data processing workflow; the adaptability of the model to new business requirements is improved by utilizing a meta-learning algorithm; and dynamic scheduling and execution of tasks are realized through a workflow engine. According to the invention, the data processing efficiency can be improved.
Owner:BEIJING INST OF COMP TECH & APPL

Industrial environment monitoring and accident prediction method fusing multi-modal data

The invention provides an industrial environment monitoring and accident prediction method fusing multi-modal data, and relates to the technical field of data processing, and the method comprises the steps: carrying out the semantic collection and causal association preprocessing of multi-modal heterogeneous data collected in real time through constructing a dynamic industrial knowledge graph; a customized deep learning model is adopted to extract deep abstract features of each mode, and weak signals and potential risks are accurately represented and uncertainty is quantified; a high-fidelity digital twin model is utilized to drive a deep reinforcement learning algorithm, and dynamic optimization and verification are performed to generate a multi-level and multi-target preventive intervention strategy combination; an intervention strategy is executed through an edge-end-cloud three-layer collaborative intelligent architecture, and online learning and system sustainable evolution are realized by using a closed-loop data feedback mechanism. According to the method, the sensing and early warning capability of the early weak and complex abnormal state of the industrial environment can be remarkably improved, the accident evolution path is accurately predicted, and credible explanation is provided.
Owner:SHANGHAI YUNLIN COMM TECH CO LTD

Small unmanned aerial vehicle inspection data management method

The invention discloses a small unmanned aerial vehicle inspection data management method, relates to the technical field of unmanned aerial vehicle inspection, and is used for solving the problem that unmanned aerial vehicle inspection data management is not clear. According to the invention, through a high-precision positioning technology and timestamp synchronization, multi-modal data collected by an unmanned aerial vehicle is associated with an equipment position, space-time consistency of inspection data is realized, multi-modal feature fusion and dimension reduction processing are carried out, unified feature representation is generated, the precision and efficiency of data analysis are improved, historical data and real-time collected data are combined, and the accuracy and efficiency of data analysis are improved. Equipment states are monitored in real time, fault modes are identified and classified, risks are evaluated, fault development trends are predicted through a time sequence prediction model, maintenance suggestions are generated, the timeliness of fault diagnosis is improved, based on fault detection results, routing inspection task priorities and path planning are dynamically optimized, an optimal routing inspection path is generated, and a flight path is corrected; the intelligence and the precision of the inspection task are improved, and the efficiency and the safety of the unmanned aerial vehicle inspection system are remarkably improved.
Owner:NANCHANG POWER SUPPLY BRANCH OF STATE GRID JIANGXI ELECTRIC POWER CO LTD

Multi-modal document retrieval enhancement generation method based on large model

The invention belongs to the technical field of multi-modal data processing, and particularly relates to a multi-modal document retrieval enhancement generation method based on a large model, which comprises the following steps: receiving query content input by a user for a multi-modal document; processing the query content by adopting an embedded model, generating a query vector representing user query semantic information, and storing the query vector in a vector database; analyzing the multi-modal document to obtain long text information, segmenting the long text information into data blocks by adopting a recursive partitioning strategy, and numbering and marking the data blocks; carrying out vectorization processing on the data blocks by adopting an embedded model to generate document vectors, storing the document vectors into a vector database, and constructing a hierarchical index structure; retrieving in a vector database based on the query vector, and returning a retrieval result; and processing a retrieval result by utilizing a large language model to generate response content conforming to the query intention of the user. According to the method, the multi-modal document can be effectively analyzed and processed, and the accuracy and comprehensiveness of analysis are improved.
Owner:杭州长望智创科技有限公司

Ancient building risk prediction management and control method and system based on large model

The invention discloses an ancient building risk prediction management and control method and system based on a large model, and relates to the technical field of ancient building protection. An ancient building risk prediction management and control system based on a large model comprises a multi-source data acquisition module, a multi-modal data fusion module, a large model risk prediction module, a risk assessment and early warning module, a digital twinborn dynamic management and control module and a repair decision module. According to the method, the structure state data, the visual image data and the environment data are fused, so that the state of the historic building is comprehensively perceived; the self-supervised learning pre-trained ViT-GPT model is combined with the LSTM time sequence model, so that the time sequence analysis capability and the multi-modal data processing capability of risk prediction are remarkably improved, the future damage probability and the risk trend of the historic building can be accurately predicted, and the defects of the prior art in long-time span risk prediction are overcome.
Owner:PUTIAN UNIV

Home abnormal state signal detection method and system based on multi-mode sensing

The invention provides a home abnormal state signal detection method and system based on multi-modal sensing, and relates to the technical field of detection, and the method comprises the steps: collecting human body motion, acoustics and environment parameters through a millimeter wave radar, an acoustic sensor and an environment sensor, inputting the parameters into a deep fusion network, a spatio-temporal attention mechanism and a multi-scale convolutional neural network are used to extract a spatio-temporal feature sequence, and joint probability features are formed in combination with an acoustic feature spectrum analysis result. And inputting the joint probability features into a time sequence knowledge graph, and obtaining scene adaptive features after processing by a graph attention network and a double-flow auto-encoder network. And finally, the multi-task learning network is combined with the risk propagation neural network and the recursive neural network to carry out anomaly prediction and risk level evaluation, and is combined with historical early warning information to output an early warning strategy. According to the invention, multi-modal data can be effectively fused, the accuracy and reliability of home abnormal state detection are improved, and a more accurate risk assessment and early warning strategy is provided.
Owner:DEXIAOBAO HEALTH TECHNOLOGY (CHANGZHOU) CO LTD

Automatic substrate glass surface defect detection method and system based on machine vision

The invention provides an automatic substrate glass surface defect detection method and system based on machine vision, and relates to the technical field of defect detection.The method comprises the steps that multi-modal data of the surface of substrate glass are obtained and preprocessed, cross-modal feature fusion is conducted on extracted image features, acoustic features and thermal imaging features, and a multi-modal feature fusion model is obtained; inputting the fused feature graph into a pre-constructed graph neural network to obtain a topological feature graph representing a defect structure; cascading the topological feature map and the fused feature map, and inputting the cascaded topological feature map and the fused feature map into a multi-task learning network to obtain a segmented suspected defect region and a corresponding defect type; performing defect positioning by using a pre-trained target detection model, and determining an accurate position of a defect through bounding box regression to obtain a target defect region image; and uploading the target defect area image to a cloud end, and accurately identifying the defect type of the target defect area by using a defect detection model deployed in the cloud end to obtain a final defect detection result.
Owner:OPTICAL MEASUREMENT IND INTELLIGENT EQUIP (NANJING) CO LTD

Transformer life prediction method and system based on machine learning and Internet of Things

The invention provides a transformer life prediction method and system based on machine learning and the Internet of Things, and relates to the technical field of transformer life prediction, and the method comprises the steps: collecting and preprocessing the multi-modal operation data of a transformer, constructing a multi-modal data fusion model based on a graph neural network, and carrying out the prediction of the life of the transformer; and mapping the dimensionality-reduced feature data into a graph structure, inputting transformer multi-modal operation data collected in real time into an optimized fuzzy inference system to obtain a health state evaluation result and a final residual life prediction result of the transformer, and obtaining a prediction result of the residual life of the transformer on the basis of transformer maintenance suggestions and historical maintenance records. Constructing a transformer digital twinborn model for maintenance strategy optimization, inputting the health state evaluation result, the residual life prediction result and the transformer maintenance suggestion into the transformer digital twinborn model, performing fault simulation and predictive maintenance analysis, and based on the fault simulation and predictive maintenance analysis result, obtaining a maintenance strategy of the transformer. And generating an intelligent maintenance scheme in combination with the expert knowledge base.
Owner:CHANGZHOU RUIWU TECH CO LTD

Multi-source heterogeneous data fusion and processing method based on big data

The present invention relates to the technical field of big data processing, and discloses a multi-source heterogeneous data fusion and processing method based on big data, including the following steps: data preprocessing, cleaning, complementing and feature extraction of multi-source heterogeneous data, providing standardized and high-quality input for subsequent processing; dynamic knowledge graph modeling, constructing a dynamic knowledge graph based on the semantic relationship of multi-source data, and generating a unified semantic representation; cross-modal alignment and deep fusion, semantically aligning multi-modal data, and generating a unified comprehensive representation through deep fusion of features; real-time stream processing, dynamically updating and processing real-time updated stream data to achieve real-time fusion of data; adaptive optimization, dynamically adjusting fusion strategy parameters, and improving the adaptability and robustness of the fusion strategy. The present invention realizes efficient fusion, real-time processing and semantic consistency expression of multi-source heterogeneous data through dynamic knowledge graph modeling, cross-modal alignment, real-time stream processing and adaptive optimization.
Owner:CHENGDU BIG DATA GRP CO LTD

Task planning system and method for intelligent robot with body based on multi-dimensional situation awareness

The invention discloses a system and a method for task planning of an intelligent robot with a body based on multi-dimensional situation awareness, and particularly relates to the technical field of task planning of the intelligent robot with the body, space-time alignment is carried out on asynchronous heterogeneous data generated by a multi-modal sensor channel, and cross-modal space-time features are extracted through a cross-modal feature fusion network; a fusion situation matrix is generated, dynamic causal modeling is used to update association strength among the multi-modal data, an anti-factual reasoning engine is used to identify and trace abnormities, and an abnormities traceability result is output; dynamically adjusting the reliability weight of each sensing channel through a multi-modal credibility evaluation model by using the fusion situation matrix and an abnormal traceability result; and on the basis of the reliability weight, inputting the fusion situation matrix into a real robot dynamic model and a digital twin virtual model, executing collaborative predictive control, and starting an adaptive rule evolution mechanism when a safety score is lower than a threshold value, thereby solving the problem of fusion matrix distortion in dynamic obstacle avoidance and precise grabbing tasks.
Owner:ZHIMOU (ZHEJIANG) TECHNOLOGY DEVELOPMENT CO LTD

Multimodal data-based method and system for recognizing cognitive engagement in classroom

PendingUS20250022314A1Speech recognitionEnergy efficient computingCognitive engagementData set
A method and system are introduced for recognizing cognitive engagement in classrooms by utilizing multimodal data. Associated modalities of cognitive engagement are learned from visual and audio data. This approach involves constructing a multidimensional representation model for cognitive engagement that includes behaviors, emotions, and speech. To identify engagement, three distinct deep learning models are employed: You Only Look Once version 8 (Yolov8) for analyzing body posture, Efficient Network (EfficientNet) for facial expressions, and Text Convolution Neural Network (TextCNN) for speech text. These models are trained and refined with the aid of student engagement surveys, leading to a decision-making process that integrates the results from the different modalities. Additionally, a dataset and a data annotation system are developed for engagement recognition. This innovative method aims to achieve detailed engagement recognition, addressing various perception needs in real-world applications.
Owner:HUAZHONG NORMAL UNIV

Medical diagnosis intelligent decision-making system based on multi-modal data fusion

The invention discloses a medical diagnosis intelligent decision-making system based on multi-modal data fusion, and the system comprises the steps: obtaining a medical image, biological features and electronic medical record data of a patient through a patient information collection unit, and extracting feature vectors from different modal data through a feature processing unit; the core of the system is a modal fusion engine, and the module combines a deep neural network and a Bayesian inference engine to realize deep learning and probability-dependent modeling of an association relationship between features; carrying out weight optimization on different modal features by adopting an attention mechanism through a self-adaptive feature optimizer, and generating optimized feature representation; the intelligent diagnosis analysis unit calculates the diagnosis probability based on the optimized features, performs auxiliary diagnosis in combination with a case knowledge base when the confidence coefficient is insufficient, and outputs a diagnosis report containing diagnosis categories, basis and suggestions; according to the method, multi-source heterogeneous data are fused, deep learning and probabilistic reasoning are combined, intelligent medical auxiliary diagnosis is achieved, and diagnosis accuracy and interpretability are improved.
Owner:BIG DATA & INFORMATION TECH RES INST OF WENZHOU UNIV +1

Controllable video generation method and system based on multi-modal fusion

The invention discloses a controllable video generation method and system based on multi-modal fusion. The method comprises the following steps: acquiring multi-modal data and preprocessing the multi-modal data; extracting feature vectors, normalizing the feature vectors, and combining the feature vectors to form a multi-modal feature matrix; converting the multi-modal feature matrix into a feature sequence; performing dynamic weight distribution on each modal feature in the feature sequence; performing multi-level deep fusion on the weighted feature sequence to obtain fusion feature representation; inputting the fused feature representation into a multi-scale space-time attention mechanism to obtain enhanced feature representation; processing by using a space attention module, a time attention module and a cross attention module in sequence to obtain global enhanced feature representation, and generating multi-frame hidden space representation; and decoding to generate a high-resolution video frame and carrying out post-processing to obtain a final high-quality video sequence. According to the method, the space-time coherence of the video sequence is kept, and the video content with higher quality and stronger controllability is generated.
Owner:NANJING ARTIFICIAL INTELLIGENCE CHIPS RES INST OF AUTOMATION CHINESE ACAD OF SCI

Systems and methods for identifying an event

A decentralized event detection and geolocation system is disclosed, utilizing a dynamically adaptable Spontaneously Emergent Geolocation Network (SEGNet) comprising mobile, stationary, and other devices equipped with multi-sensor capabilities. The system operates by obtaining sensor data from distributed devices, applying trained models to detect potential events of interest (EOIs), and dynamically forming device networks for collaborative geolocation and event validation. The system employs synchronization mechanisms and geolocation techniques, including Time Difference of Arrival (TDOA) and / or Angle of Arrival (AoA), to determine event locations through multimodal data fusion and iterative validation processes. The architecture supports multiple operational modes, including decentralized, centralized, and hybrid configurations, enabling operation in connectivity-limited environments and enhanced processing when server access is available. The system is designed to accommodate dynamic task delegation and real-time threshold adjustments based on environmental conditions, providing reliable event detection and geolocation across diverse scenarios.
Owner:ZEL TECHNOLOGIES LLC

Intelligent pulmonary nodule grading method and system based on multi-modality feature fusion

An intelligent pulmonary nodule grading method and system based on multi-modality feature fusion. The method comprises: using clinical multi-modality data of a physical examination crowd to acquire an ROI and a VOI of a pulmonary nodule on the basis of a chest CT examination image and an examination report, designing a multi-task feature extraction network based on an attention mechanism, and acquiring radiomics features and deep-layer image features from the ROI and the VOI; designing a cross-modality feature fusion method based on graph representation learning, designing a multi-modality information extraction method, acquiring a specific feature representation and graph structure of each modality, and then performing feature representation and graph structure fusion; and providing a pulmonary nodule grading GCN model optimization and clinical verification method based on self-supervised learning, so as to implement fine grading of pulmonary nodule malignancy having only slight differences. A new way is provided for the design of a fine-grained classification algorithm; and in the aspect of solving a clinical problem, provided is a classification algorithm optimization method and clinical verification scheme in which generative self-supervised learning and comparative self-supervised learning are organically combined.
Owner:ZHENGZHOU UNIV

Bridge crack intelligent diagnosis system based on multi-modal data fusion

PendingCN120873887AEngineeringMulti source data
The invention belongs to the technical field of bridge diagnosis, and discloses a bridge crack intelligent diagnosis system based on multi-modal data fusion. By fusing multi-source data such as visual images, sound wave detection and vibration signals, comprehensive perception and characterization of crack features are realized; constructing a bridge crack characteristic spectrum diagram by adopting a cross-modal feature extraction and heterogeneous feature coding technology; generating a crack evolution situation map based on space-time correlation analysis and knowledge graph construction; the robustness of the system in a complex environment is improved through environmental adaptability feature enhancement and multi-scale characterization; constructing a bridge safety risk hypergraph in combination with multi-dimensional risk analysis and multi-agent collaborative diagnosis; analyzing and revealing a crack evolution mechanism by applying a causal relationship; and finally, through dynamic fusion and uncertainty quantification, a crack intelligent diagnosis comprehensive report is generated. According to the system, the limitation of traditional single-mode diagnosis is broken through, and dynamic prediction and accurate risk assessment of fracture evolution are realized.
Owner:CHANGZHOU INST OF TECH

Land space planning environment influence monitoring method and device

The invention discloses a territorial space planning environment influence monitoring method and device. The method comprises the following steps: collecting and preprocessing multi-source environment monitoring data; performing spatio-temporal feature extraction on the preprocessed multi-source environment monitoring data by using a self-supervised spatio-temporal diagram mask transfer attention network to obtain initial spatio-temporal features; performing fusion processing on the initial spatial-temporal characteristics based on a multi-mode diffusion converter to obtain fusion characteristics; calculating an environmental impact evaluation index based on the fusion feature; and based on a preset visualization template and an early warning rule, generating a dynamic monitoring result including a space-time evolution diagram, a trend analysis diagram and early warning information according to the environmental influence evaluation index. Through deep learning and multi-modal data fusion technologies, the precision and efficiency of environmental impact monitoring are improved, and accurate evaluation and timely early warning of environmental impact in the land space planning implementation process are realized.
Owner:BEIJING ZHONGLIAN WORLD CONSTR PLANNING & DESIGN CO LTD

Scene space three-dimensional model dynamic modeling method based on multi-modal data

The embodiment of the invention provides a scene space three-dimensional model dynamic modeling method based on multi-modal data, and the method comprises the steps: fusing the feature information of a video sequence, point cloud data and IMU data, and building a space-time corresponding relation through employing a particle filtering algorithm. Scene semantic understanding is realized based on a Transform multi-modal scene analysis network, scene scores are calculated in combination with spatial density, depth occlusion and shape change features, and an optimal modeling algorithm is selected through a strategy network of deep reinforcement learning training. Meanwhile, scene dynamic features are analyzed by using a deep neural network and a graph neural network, adaptive grid subdivision and geometric constraint reconstruction of a dynamic region are realized, and a reconstruction threshold is dynamically adjusted according to a scene semantic type. According to the method, the three-dimensional reconstruction quality and efficiency of a large-scale complex scene are effectively improved.
Owner:UNIVERSAL UBIQUITOUS TECH CO LTD

Intelligent water conservancy inspection method, device and equipment and storage medium

The invention relates to the technical field of data processing, and discloses an intelligent water conservancy inspection method, device and equipment and a storage medium. The method comprises the following steps: performing dynamic fusion processing on acquired multi-dimensional water conservancy environment data of a preset target area to obtain a target digital twinborn body; performing agent collaborative analysis on the digital twinborn body to obtain a self-adaptive inspection strategy; performing multi-modal sensing data acquisition on the target area according to the self-adaptive inspection strategy to obtain multi-modal data; performing federated learning processing on the multi-modal data to obtain holographic situation awareness data; performing causal reasoning analysis on the holographic situation awareness data to obtain a dynamic risk map; and inputting the dynamic risk map into a preset deep Q network for prevention and control strategy analysis to obtain an active prevention and control strategy and an inspection report. According to the invention, the efficiency and accuracy of intelligent water conservancy inspection are improved.
Owner:HEBEI UNIV OF ENG

Live broadcast room content identification and intelligent distribution method and system based on multi-modal fusion

The invention provides a live broadcast room content identification and intelligent distribution method and system based on multi-modal fusion, and relates to the technical field of machine learning, and the method comprises the steps: collecting video, audio and bullet screen data of a live broadcast room, and carrying out the feature extraction; performing cross-modal interaction, feature alignment and feature reconstruction on the multi-modal features by using the multi-modal feature fusion model to generate cross-modal fusion features; and carrying out content identification based on a pre-trained content classification model to obtain a category label and a time sequence feature. Furthermore, a live broadcast room content portrait and an association network are constructed, and a live broadcast room real-time score is generated and a live broadcast room display weight and a push strategy are dynamically adjusted in combination with user historical watching behavior data and a deep reinforcement learning model. Through multi-modal data fusion and deep reinforcement learning, accurate identification and personalized recommendation of the live broadcast room content are realized, and the user experience and the platform operation efficiency are effectively improved.
Owner:BEIJING YIHUI ZMENG NETWORK TECH

Commercial credit evaluation and supervision method based on multi-modal coevolution algorithm

The invention relates to a business credit evaluation and supervision method based on a multi-modal coevolution algorithm, and the method comprises the steps: collecting and cleaning multi-modal data, guaranteeing the data quality, uniformly storing the multi-modal data as a structural database, extracting features through cross-modal comparative learning, and enabling the features to be embedded into a dynamic multi-modal knowledge graph, a modal sharing network is used to extract and fuse modal sharing and specific features, consistency of modal feature distribution is optimized through adversarial learning, a global co-evolution credit scoring model is initialized, a dynamic multi-objective algorithm is used to optimize accuracy, robustness and interpretability of the model, and a credit scoring result is obtained. A causal reasoning model is introduced to analyze driving factors of enterprise credit changes, a causal relationship is established, a closed-loop feedback system is designed, a credit evaluation model is updated, adaptability is enhanced, an interpretable credit evaluation report is generated based on a generative adversarial network, and transparency and credibility are enhanced. According to the method, the accuracy and adaptability of credit assessment are improved, and the robustness and interpretability of an assessment result are ensured.
Owner:SOUTHEAST UNIV

Personalized precise irrigation method based on Internet of Things and deep learning

The invention discloses a personalized precise irrigation method based on Internet of Things and deep learning, relates to the technical field of agricultural intellectualization, and solves the problem that for farmland with various crop varieties, specificity of different crops may not be fully considered in the prior art, and the irrigation efficiency is high. In order to solve the problem of lack of personalized irrigation strategies for specific crop types and growth stages in the prior art, multi-modal data and deep reinforcement learning are fused: soil humidity, weather, crop images and other multi-source data are integrated, a reinforcement learning algorithm is adopted to dynamically adjust the irrigation strategy, and the irrigation strategy is dynamically adjusted on the basis of actual crop growth feedback. Immediate optimization of irrigation decisions is realized, and environmental adaptability and decision accuracy are improved; through a crop-environment dynamic model, a dynamic coupling model of crop growth and environment factors is constructed, an irrigation plan is generated in a customized mode according to different crops, growth stages and environment conditions, and individuation and precision of irrigation are achieved.
Owner:HEILONGJIANG AGRI INVESTMENT BIG DATA TECH CO LTD

System and method for causality-augmented generative intelligence to discover non-obvious insights from heterogeneous data sources

The present invention provides a system and method for causality-augmented generative intelligence capable of autonomously discovering non-obvious actionable insights from heterogeneous and multimodal data sources. The system integrates a data ingestion unit for semantic and temporal harmonization of structured and unstructured datasets, a causal inference processor for constructing a dynamically evolving directed causal knowledge representation using perturbation-based validation, a latent representation processor that combines multimodal semantic embeddings with causal parameters to generate fused latent vectors, and a generative insight processor utilizing causally constrained generative reasoning to synthesize hypotheses anchored to verified cause-effect dependencies. A validation processor performs counterfactual assessment and observational verification to ensure retention of only those insights that remain consistent with causal ground truth.
Owner:MIA MD TOFAYEL GONEE MANIK

Multi-modal sensing humanoid robot action self-adaptive control method and multi-modal sensing humanoid robot action self-adaptive control system

The invention discloses a multi-modal perception humanoid robot action adaptive control method and system, and belongs to the technical field of computers, and the method comprises the steps: obtaining first multi-modal data; the first multi-modal data comprises first image data, first point cloud data and first force sense data; performing data fusion on the first image data and the first point cloud data, and constructing a three-dimensional model of the scene in real time; generating an optimal path according to the real-time environment data of the three-dimensional model, converting the optimal path into a joint instruction of the humanoid robot, and generating an action control strategy; according to the action control strategy, arm movement of the robot is adjusted through visual servo control, and first operation data are obtained; the path and action of the robot are adjusted in a closed-loop control mode, and second operation data are obtained; and reinforcement learning and regular evaluation are carried out through a DDPG algorithm, and an action control strategy is continuously optimized. According to the invention, multi-modal sensing robot motion adaptive control with strong adaptive capability, high operation precision and good robustness can be realized.
Owner:广州里工实业有限公司

Knowledge fusion multi-modal interaction method and apparatus based on improved alignment method

A knowledge fusion multi-modal interaction method and apparatus based on an improved alignment method. The method comprises: acquiring multi-modal data (S101); constructing a query transformation model (S102); on the basis of a multi-stream feature extraction method, performing improved processing on a vision transformer to obtain a multi-scale vision transformer (S103); on the basis of a joint text method, training the multi-scale vision transformer to obtain a target vision transformer (S104); freezing the target vision transformer, and on the basis of the target vision transformer, performing vision-text alignment training processing on the query transformation model to obtain a target query transformation model (S105); connecting the target query transformation model to a language model to obtain a knowledge fusion multi-modal interaction model (S106); and inputting the multi-modal data into the knowledge fusion multi-modal interaction model to obtain an interaction result (S107). In the embodiments of the present invention, the knowledge fusion multi-modal interaction model aligns visual representation with text representation, so that the mutual information between the visual representation and the text representation can be maximized, and the present invention can be widely applied to the technical field of artificial intelligence.
Owner:SUN YAT SEN UNIV +1

Intelligent fault diagnosis method and system for explosion-proof motor in natural gas industry

The invention discloses an intelligent fault diagnosis method and system for an explosion-proof motor in the natural gas industry. The method comprises the following steps of S10, obtaining multi-source heterogeneous data of the explosion-proof motor; s20, performing enhancement processing on the multi-source heterogeneous data of the explosion-proof motor to obtain an enhanced multi-mode data set of the explosion-proof motor; s30, performing feature extraction on the enhanced multi-modal data set of the explosion-proof motor by using a deep learning model to obtain a high-dimensional feature vector; s40, performing fault classification and fault severity evaluation by adopting a multi-task learning network based on the high-dimensional feature vector to obtain fault type probability distribution and a fault severity index; and S50, according to the fault type probability distribution and the fault severity index, generating an intelligent diagnosis report containing maintenance suggestions of the explosion-proof motor. According to the explosion-proof motor fault diagnosis method, multi-source heterogeneous data can be fully utilized, intelligent diagnosis is achieved, the diagnosis efficiency and accuracy are improved, and preventive maintenance is achieved.
Owner:PRELITE (TIANJIN) GAS EQUIP CO LTD +1