6569 results about "Semantic information" patented technology

The invention belongs to the technical field of multi-modal data processing, and particularly relates to a multi-modal document retrieval enhancement generation method based on a large model, which comprises the following steps: receiving query content input by a user for a multi-modal document; processing the query content by adopting an embedded model, generating a query vector representing user query semantic information, and storing the query vector in a vector database; analyzing the multi-modal document to obtain long text information, segmenting the long text information into data blocks by adopting a recursive partitioning strategy, and numbering and marking the data blocks; carrying out vectorization processing on the data blocks by adopting an embedded model to generate document vectors, storing the document vectors into a vector database, and constructing a hierarchical index structure; retrieving in a vector database based on the query vector, and returning a retrieval result; and processing a retrieval result by utilizing a large language model to generate response content conforming to the query intention of the user. According to the method, the multi-modal document can be effectively analyzed and processed, and the accuracy and comprehensiveness of analysis are improved.

The invention discloses an intelligent operation and maintenance management and alarm system based on a large model agent, and the system comprises a data collection and fusion module which is used for processing original data collected from an IT system to obtain an operation and maintenance data graph, and the original data at least comprise logs, indexes and link tracking data; the anomaly detection module is used for performing analysis based on the preprocessed data and the time sequence characteristics to determine abnormal data; the alarm optimization module is used for analyzing the semantic information corresponding to the abnormal data so as to determine the priority of the corresponding alarm; the root cause analysis module is used for performing fault reasoning on the abnormal data based on a pre-constructed knowledge graph and generating fault interpretation according to data corresponding to the abnormal data; and the automatic operation and maintenance decision module is used for generating an automatic repair strategy according to the abnormal data and the fault interpretation and executing the automatic repair strategy to repair the abnormal data.

The invention provides a construction scene prediction method and device fusing an image, a text and a BIM modal, and relates to the technical field of intelligent construction prediction management. According to the method, the BIM semantic graph is constructed by extracting the BIM semantic information of the BIM model, and the target detection recognition of the construction site video is carried out in combination with the YOLO model to obtain the target detection result; performing cross-modal alignment with the CLIP to realize deep fusion of the multi-modal data of the image, the text and the BIM to obtain a multi-modal heterogeneous graph; and inputting the multi-modal heterogeneous graph into a space-time sequence model for prediction, outputting prediction results of construction scenes at a plurality of moments in the future, and dynamically mapping the prediction results to a digital twinborn platform to realize risk early warning and visual display. The construction dynamic change can be captured in real time, the construction progress and risk can be accurately predicted, and the intelligent level of construction management is improved.

The invention discloses a hierarchical semantic-driven retrieval enhancement generation method and system, a natural hierarchical relationship and a semantic boundary of a document are effectively reserved by constructing a tree hierarchical structure based on a document chapter title, and a recursive semantic boundary splitting strategy is adopted to refine overlong text nodes, so that the semantic integrity is ensured, and the retrieval enhancement generation efficiency is improved. And the model input length limitation is met, and the semantic information is prevented from being lost. Meanwhile, node knowledge point extraction and abstract generation are achieved through a large language model, top-down multi-level title path transmission and bottom-up content aggregation are combined, the structural perception and semantic expression ability of nodes is enhanced, and in the retrieval stage, based on similarity distribution of query and node semantic expression, an adaptive retrieval threshold value is dynamically calculated, and the retrieval efficiency is improved. A fixed top-k retrieval strategy is replaced, intelligent screening of different query and hierarchical nodes is achieved, information coverage and redundancy suppression are balanced, and retrieval efficiency and accuracy are remarkably improved.

The invention discloses a lightweight malicious network traffic detection method based on heterogeneous modal feature fusion, and mainly solves the problems of low feature extraction efficiency and insufficient single modal feature representation of the existing method. Comprising the following steps: acquiring and optimizing a network flow data set, preprocessing the network flow data set, and extracting and generating spatial feature, time sequence feature and behavior pattern feature vectors; an improved self-attention mechanism network is constructed, a lightweight heterogeneous modal feature fusion model LMF is designed, multi-modal feature deep fusion is performed through dynamic weight distribution, a lightweight classification model is trained, and the model is utilized to detect network malicious traffic. Through the lightweight heterogeneous modal feature fusion model LMF and in combination with a dynamic weight distribution mechanism, spatial distribution, time sequence dependence and behavior semantic information of network traffic are deeply mined, data processing efficiency, malicious traffic detection accuracy and system robustness are improved, and the method is suitable for efficient malicious traffic identification and defense in the field of network security.

The invention relates to the technical field of image processing, can be applied to business scenes of medical health, financial science and technology and the like, and discloses an image sensitive character desensitization method which comprises the following steps: extracting a plurality of feature maps from a to-be-processed image, and inputting a feature pyramid network to generate pyramid features, fusing a spatial attention mechanism to enhance the response of a sensitive region in each level of feature map, generating candidate region coordinates through a region candidate network, mapping the candidate region coordinates to a to-be-processed image to obtain a local image, extracting text semantic information in the local image by using a codec based on a self-attention mechanism, and obtaining a local image; and performing sensitive word detection in combination with the semantic classification model, and performing local texture repair operation on the sensitive area according to a detection result to generate a desensitized image. According to the method, a space attention mechanism and a semantic understanding module are introduced on the basis of multi-scale feature fusion, so that the capability of detecting small-size sensitive characters in an image is improved.

The present invention relates to the technical field of knowledge graph completion tasks, and particularly relates to a knowledge graph link prediction method. The method comprises: using a precoding model to obtain an embedding layer vector, and constructing a corresponding masked triple; adding a corresponding position code to each element in the masked triple, so as to obtain a corresponding input sequence, inputting the input sequence into a trained main masking model, and outputting an entity classification probability; and on the basis of the entity classification probability, predicting potential candidate entities. The method further comprises: concatenating semantic information corresponding to the embedding layer vector and structural information obtained by an embedding model, so as to obtain fused head entity and relation representations, and constructing a corresponding fused masked triple; and adding a corresponding position code to each element in the fused masked triple, so as to obtain a corresponding fused input sequence. The present invention uses a precoding method, thereby effectively reducing the training burden on a model, and improving the inference speed of a model; and a fusion module is used before inputs are fed into a main masked model, thereby ensuring the integrity of textual description information and improving prediction accuracy.

The invention discloses a low-altitude airspace management method and system based on data analysis, and relates to the technical field of air traffic management and low-altitude airspace planning, and the method comprises the steps: 1, multi-scale dynamic grid division: adjusting the grid granularity in real time according to the aircraft density and environment complexity, generating a multi-scale grid by adopting an optimized quadtree data structure, and carrying out the multi-scale dynamic grid division; boundary smoothing processing is carried out on boundary areas of the grids with different scales; 2, intelligent airspace layering: dynamically adjusting the thickness of a height layer according to a risk score, and setting a buffer layer to enhance the flight safety; and 3, intelligent semantic annotation: giving dynamic semantic information such as risk levels and traffic priorities to the grid units, embedding airspace control regulations and flight rules, and realizing multi-source data fusion and real-time updating of semantic information. According to the method, the problems of static division limitation, boundary problem, insufficient real-time performance and the like in the prior art are solved, and the airspace resource utilization rate and the flight safety are improved.

The invention discloses a community intelligent monitoring and emergency linkage method and system fusing BIM spatial semantics, and the method comprises the steps: constructing a BIM scene map, and obtaining the attributes and mutual relationships of components and spatial regions in a BIM model; mapping a dynamic target detected in video monitoring into the BIM model, and obtaining spatial semantic information of the dynamic target; based on BIM spatial semantic information of a dynamic target, a target-environment interaction graph is constructed, a graph neural network model is used for training and reasoning, and specific complex events related to spatial contexts are recognized; taking the BIM model as a space-time reference, fusing multi-source heterogeneous data, and reconstructing by adopting a graph-based event association algorithm to form a complete event chain containing an atomic event sequence and an association relationship; and when an emergency event or an event chain is detected to indicate an emergency state, combining BIM preset information and real-time sensor data, dynamically generating an optimal emergency plan, and performing visual commanding and dispatching through a BIM three-dimensional scene and augmented reality.

The invention discloses an end-to-end monocular visual odometer method fusing space-time semantic information. According to the method, continuous image sequence frames are collected through a color monocular camera, and a multi-information fusion end-to-end deep learning framework is constructed; a heterogeneous training domain is adopted to set various data set course sharing parameter fusion training, continuous image sequences are input, and the end-to-end deep learning framework is dynamically coupled with hidden state feature vectors output historically, so that a feature mapping relation of time sequence perception is formed; and interpretable feature decoupling of the static background elements and the dynamic entity objects in the scene is realized. After iterative feature fusion, the system outputs sparse depth and camera motion poses which conform to scene geometric constraints, so that a camera trajectory estimation model with high robustness and strong generalization ability in a complex environment is constructed. According to the method, the positioning precision and stability of the monocular vision odometer are remarkably improved.

The invention discloses a multi-target scene visual SLAM (Simultaneous Localization and Mapping) method fusing target semantics and Gaussian splashing. The method comprises the steps that input data are preprocessed, two-dimensional space masks and semantic information corresponding to the two-dimensional space masks are extracted, global semantic identifiers are distributed to the two-dimensional space masks corresponding to a background and each target, and a target semantic segmentation map is obtained; reading first frame RGBD data as a current frame, acquiring a target semantic segmentation map corresponding to the current frame, and initializing camera pose parameters; respectively establishing an initial target Gaussian splashing model for the background and each target; adopting a Gaussian splash algorithm to render all the target Gaussian splash models to a next frame, and generating an RGB rendering image, a depth rendering image and a target semantic rendering image of the next frame; and designing a loss function between the next frame of rendered image and the corresponding real image, optimizing the pose of the camera, and updating the parameters of the target Gaussian splash model. According to the invention, simultaneous positioning and map construction of multiple target scenes are realized.

The invention discloses an unmanned aerial vehicle small target aerial photography detection method and system based on RT-DETR, a medium and equipment, and belongs to the technical field of unmanned aerial vehicle visual detection, aerial photography images are collected based on an unmanned aerial vehicle, the images are input into a trained target detection model, and target position and category information is obtained. The method comprises the following steps: firstly, extracting low-layer features of an image, and simultaneously capturing information of a channel dimension and a space dimension based on an efficient multi-scale attention mechanism; and outputting features through the convolution residual block. Key features are processed based on a single-scale feature interaction module; for the coded feature map, fusing information of a shallow layer and information of a deep layer step by step in an up-sampling and transverse connection mode; in a down-sampling stage, context semantic information of a target is reserved, and global and local information interaction is enhanced. According to the method, the accuracy and robustness of small target detection are remarkably improved, and the practicability and deployment value in actual application scenes such as unmanned aerial vehicle aerial photography and remote sensing monitoring are expanded.

The invention relates to the technical field of data processing, and discloses a cross-border e-commerce commodity recommendation system and method based on multi-source data fusion. The system comprises an acquisition module for performing data acquisition to obtain a cross-border unified data warehouse and federal learning cooperation data; the classification module performs text classification processing to obtain a user preference analysis result and an interpretable attention mark; a quantization module carries out quantization processing to obtain a cross-border selection feature matrix; the fusion module carries out weighted fusion processing to obtain a basic comprehensive score and a weight convergence detection result; the evaluation module performs risk evaluation processing to obtain risk probability distribution and cross-border compliance evaluation results; and the sorting module carries out real-time processing through lightweight preprocessing and a flow-type calculation pipeline to obtain a comprehensive score sorting list of cross-border selected products. The problem that a traditional cross-border e-commerce product selection method cannot effectively integrate multi-source heterogeneous data and cannot analyze user feedback text semantic information is solved.

The invention relates to the technical field of building model design, in particular to a building three-dimensional model lightweight design method and system based on artificial intelligence, and the method comprises the steps: extracting geometric-semantic features of a building model through a multi-scale curvature filtering and semantic segmentation network, constructing a fusion feature vector matrix, and obtaining a fusion feature vector matrix; by utilizing an integrated graph convolutional network and a self-adaptive neural simplification network of a double-branch attention mechanism, differential resampling is executed based on vertex importance weight, a simplified intermediate model is generated, a surface microstructure is recovered by means of a generative adversarial network, grid holes are corrected by combining Delou inner triangulation, and a surface microstructure is obtained. A non-uniform rational B-spline curved surface is adopted to reconstruct a key decoration component and output a lightweight model, so that the problems of insufficient geometric feature retention and semantic information splitting in the traditional technology are solved, intelligent, efficient and lightweight of a building three-dimensional model is realized, visual fidelity is ensured while data compression is performed, and the construction quality is improved. And the digital management requirement of the whole life cycle of the building is met.

The invention discloses a Transform-CNN medical image segmentation method and system based on multi-scale fusion semantic enhancement, and relates to the technical field of image segmentation, and the method comprises the steps: collecting medical image data, and constructing a medical image data set; constructing a medical image segmentation network model; training the medical image segmentation network model through the medical image data set; and inputting real-time acquired data into the trained medical image segmentation network model to obtain a medical image segmentation result. According to the invention, double encoders, LGDA modules and the like are adopted to capture multi-scale global features of kidney tumors and enhance representation, and the LGDA module is used for adapting to size and form changes of the kidney tumors; the MLCF module is used for supplementing information of the main encoder; the PSE module captures multi-scale global semantic information and integrates local context information, local features are fused through the SAM module, and kidney tumor positioning in an endoscope image is achieved.

The invention discloses a cockpit active recommendation system and method based on a knowledge graph and semantic reasoning, and relates to the technical field of intelligent cockpits. The system receives natural language voice input of a user, executes voice recognition and semantic analysis, extracts user intention, keywords and slot entities, generates structured semantic information, constructs or calls a knowledge graph structure with semantic relation edges in combination with environment context information, and obtains the knowledge graph structure with the semantic relation edges. Semantic path reasoning is carried out based on the path dependence weight and the semantic similarity, a semantic edge label guided graph attention mechanism is introduced to calculate a path consistency score, a candidate recommendation set is generated, the semantic fitting degree and the path score are fused to sort and output recommendation content, and the graph edge weight and the user portrait are updated based on user feedback. According to the method, semantic understanding precision, recommendation path interpretability and system adaptive capacity are improved, and the method is suitable for personalized voice recommendation, man-machine interaction and scene linkage control tasks in an intelligent cockpit.

The invention discloses a large-format image segmentation method and system fusing frequency domain perception and state space modeling. According to the method, three core parts including an encoder, a state space modeling module and a frequency domain perception feature fusion module are included, a feature extraction network based on a ResNet structure is adopted by the encoder, multi-scale feature extraction of an input image is achieved, and effective capture of deep semantic information is ensured. The state space module realizes adaptive optimization of image data through visual state space modeling, and constructs a hybrid architecture in combination with a wavelet multi-scale convolutional network, thereby effectively solving the limitation of the traditional CNN in the aspect of remote dependence capture. And the frequency domain perception feature fusion module effectively optimizes the feature reconstruction process through interactive fusion of multi-frequency domain information, so that efficient and fine pixel-level semantic segmentation is realized. The decoder can integrate a low-frequency global structure and high-frequency edge detail information, and the recovery effect of edges and textures in a high-resolution image is remarkably enhanced.

The invention discloses a multi-agent unmanned aerial vehicle cluster collaborative search method based on a semantic communication technology. The method comprises the following steps: defining a partially observable Markov decision process of an unmanned aerial vehicle cluster multi-agent system according to a search task demand; initializing a semantic communication protocol of interaction between the unmanned aerial vehicle cluster and the environment; summarizing and abstracting semantic information of the unmanned aerial vehicle cluster; unmanned aerial vehicle cluster collaborative search task allocation and path planning are carried out by using semantic information; dynamic adjustment and real-time task optimization of the multi-agent system are realized in combination with semantic information; and evaluating a task completion effect. According to the method, the problem of incomplete search coverage of the unmanned aerial vehicle cluster in the complex environment is effectively solved, the search efficiency and accuracy of the unmanned aerial vehicle cluster in the complex environment can be effectively improved, the search efficiency and task completion quality of the unmanned aerial vehicle cluster are remarkably improved, and the intelligent level and robustness of the system are enhanced.

The invention belongs to the technical field of target detection and recognition, and particularly relates to a thinking chain-based large and small model collaborative target detection and recognition method. According to the method, the small model is responsible for most of easy-to-detect targets, the calculation pressure of the large model is reduced, the large model is responsible for suspected samples, vision and language multi-mode reasoning is combined, the overall false detection rate and the omission ratio are both reduced, confidence evaluation is conducted through the joint probability, automatic screening and manual rechecking of uncertain results are achieved, the reliability of key results is guaranteed, and the method is suitable for large-scale popularization and application. According to the'pseudo thinking chain + pseudo label 'method, by means of reasoning and labels generated by the model, data dependence on manual labeling is reduced, only low-confidence samples are manually confirmed, the human intervention range is narrowed, the human cost is remarkably saved, and semantic information with finer granularity is provided for the model by introducing phrase-level feature descriptors. And the identification capability of complex target attributes and states is improved.

The invention discloses an optical remote sensing image salient target detection method based on progressive attention enhancement, and belongs to the technical field of computer vision. The method comprises the following steps: preprocessing an original data set; inputting the preprocessed image into a hierarchical progressive fusion encoder, capturing a global irregular topological structure and local fine-grained image details, and realizing cross-hierarchical feature fusion; inputting the output characteristics of the encoder into a global context enhancement module, and capturing multi-level context information by adopting a parallel multi-branch structure; and inputting the output features of the hierarchical progressive fusion encoder and the global context enhancement module into a multi-scale progressive attention enhancement decoder, carrying out hierarchical decoding on the input features by adopting a saliency-guided attention mechanism, and gradually aggregating deep semantic information and shallow detail features to realize coarse-to-fine progressive optimization, so as to improve the robustness of the multi-scale progressive attention enhancement decoder. And finally generating a saliency map. The method can effectively improve the processing performance of an irregular topological structure and a complex context relationship in the optical remote sensing image.

The invention relates to the technical field of building information modeling, in particular to a building drawing geometric feature extraction and visual modeling method and system, and the method comprises the steps: carrying out the self-adaptive noise reduction, contrast enhancement and line refinement processing of an original building drawing image, and carrying out the automatic layer separation based on colors and line types; linear geometric features and specific symbol geometric features in the drawing image are extracted, and a geometric feature set is constructed; component instantiation, attribute assignment and topological relation reasoning are carried out by using a predefined building component semantic rule base, and a building component semantic network is generated; and mapping the semantic network to a parameterized three-dimensional modeling engine, calling an IFC standard three-dimensional template, performing parameter driving and automatic assembly, generating a three-dimensional building model with semantic information and a spatial structure, and performing visual output. According to the method, efficient and standardized conversion from a two-dimensional building drawing to a three-dimensional building model can be realized, and the method has the remarkable advantages of processing complex drawings and high-precision modeling.

The invention discloses an antagonism prompt detection method and device based on a large language model and a medium, and belongs to the technical field of artificial intelligence safety. The technical problem to be solved by the invention is how to overcome the defects of static rule lagging, high manual maintenance cost and insufficient context understanding in the security protection process of a large language model in the prior art, and dynamic, real-time and high-precision antagonism prompt detection is realized. According to the technical scheme, the method comprises the following steps: feature extraction: comprehensively analyzing semantic information, structural information and context information in a user text, extracting semantic features, structural features and context features, performing Min-Max normalization on the semantic features, the structural features and the context features, and splicing the semantic features, the structural features and the context features into a 128-dimensional joint vector, performing feature selection on the joint vector through an L1 regularization logistic regression model, compressing to 10-dimensional core features, and removing redundant information; carrying out resistance scoring; and dynamically defending.

The invention discloses an inquiry model training method and device based on a long thinking chain, and relates to the field of large models, semantic information is extracted through strategy network analysis of a model, and an initial step decision is generated in combination with context information in a historical memory library; sending the initial step decision into a reasoning path generator, and reasoning to generate a primary diagnosis disease source and an intermediate diagnosis step; sending the primary diagnosis source and the intermediate diagnosis step into a verification module, performing pathological logic verification according to a case diagnosis report and a medical knowledge base, and feeding back a verification result; the reasoning path generator updates the historical memory bank based on the feedback result, the preliminary diagnosis disease source and the intermediate diagnosis steps; the strategy network continues reasoning based on user feedback input and the updated context information in the historical memory bank, and finally an inquiry result is output. According to the scheme, technical means such as reinforcement learning, self-adaptive backtracking and memory enhancement are introduced into a long thinking chain reasoning framework, so that a large language model realizes multi-aspect comprehensive improvement in medical question and answer and auxiliary diagnosis scenes.

The invention discloses a voice interaction method and device based on lip language enhancement, equipment and a storage medium, and the method comprises the steps: extracting lip language features based on an image sequence of a lip region, and carrying out the feature extraction of a voice signal, and obtaining an audio feature; performing cross-modal fusion coding on the lip language features and the audio features to generate mixed features containing audio-visual information; inputting the mixed features into a large language model, understanding the intention of the interaction object and generating a corresponding semantic reply; and finally, synthesizing into voice and / or converting into characters. According to the invention, by introducing the lip features, additional visual clues are provided for speech recognition, and the robustness and accuracy of speech recognition can be significantly improved; effective fusion coding is carried out on the lip language features and the sound features, and semantic information splitting caused by simple and independent recognition is avoided; and the capability of the large model is fully utilized, so that more natural and more intelligent interaction experience is realized.

The invention discloses a target behavior prediction method and related equipment, and relates to the technical field of target prediction, and the method comprises the steps: obtaining multi-modal historical behavior data and real-time environment semantic information of a target object; determining key causal event nodes based on the multi-modal historical behavior data and a space-time causal model; based on the key causal event nodes and the real-time environment semantic information, determining a dynamic causal graph; determining a future potential event type and a corresponding event occurrence probability based on the dynamic cause and effect graph; determining an event triggering constraint condition based on the event occurrence probability; determining node weight updating parameters of the dynamic causal graph based on an incremental learning framework and multi-modal historical behavior data input in real time; and determining a predicted behavior track of the target object based on the event triggering constraint condition and the updated dynamic causal graph. According to the invention, by fusing the multi-modal data and space-time causal reasoning, the accuracy and environmental adaptability of target behavior prediction are improved.

The invention discloses a multi-modal fusion rumor detection method and system based on a dynamic graph convolutional neural network. According to the method, a dynamic feature graph of a language propagation path is constructed, and potential features in the language propagation process are extracted and analyzed by utilizing time sequence changes and key node relations between nodes in a propagation graph. A neural network is adopted to extract and enhance image data, text semantic features are extracted in combination with a text feature modeling network, text feature vectorization expression is achieved based on a BERT model, and rich semantic information is obtained. And a gating mechanism is introduced to dynamically adjust fusion weights of different modal features, and an information fusion strategy is optimized. A collaborative attention mechanism is further adopted for deep fusion, interactive learning of text, image and propagation path features is enhanced, and the relevance of cross-modal and time series data is improved. And finally, inputting the fused feature vectors into a classifier for accurate classification, thereby realizing accurate detection of the social media rumors. According to the method, the multi-modal features are effectively integrated, and the false information identification efficiency is remarkably improved.

The invention relates to the technical field of artificial intelligence, and discloses a rapid labeling method based on an artificial intelligence large model, which comprises the following steps of: 1, extracting an entity set and a relationship set of a specific field through a document set of a target field, identifying nouns, terms and keywords related to the field in a document by using an entity identification technology to form an entity set, and marking the entity set and the relationship set; the method comprises the following steps of: establishing an entity set, analyzing logic and semantic relevance among entities by utilizing a relation extraction technology, generating a relation set, combining the entity set with the relation set to construct a knowledge triad, and the knowledge triad comprises an association relation among the entities. By introducing the domain knowledge graph, the entity and relation information extracted from the target domain document is structured into the knowledge triad, and the knowledge triad is matched with the to-be-annotated data to optimize the annotated semantic information, so that the domain adaptability and semantic consistency of the annotation result are realized, and higher accuracy and consistency effects of the annotation data in the professional field are obtained.

The invention discloses a virtual-real combination-based power generation side industrial control system network security target construction method. The method comprises the following steps of: constructing a virtual-real combination target environment consisting of a physical equipment layer and a virtual model layer; the heterogeneous industrial control protocol between the physical equipment layer and the virtual model layer is analyzed, protocol semantic information is extracted, and a bidirectional dynamic mapping rule of a physical equipment state and a virtual model state is generated based on the protocol semantic information; based on a state change event of the physical equipment layer, according to a bidirectional dynamic mapping rule, synchronizing changed equipment state data to the virtual model layer in real time, and simulating protocol behavior logic corresponding to the equipment state data in the virtual model layer according to a security test requirement; and based on the attack instruction or the abnormal state signal generated by the virtual model layer, according to the protocol specification format of the target physical equipment, converting the instruction or the signal into an executable control command, and driving the physical equipment layer to execute an operation corresponding to the control command.

The invention discloses a heterogeneous perceptual attention fusion network-based multi-modal image target detection method, which aims at the target detection problem in a complex scene, utilizes a double-backbone network to respectively perform multi-scale feature extraction on visible light and infrared images, generates a feature pyramid containing different levels of details and semantic information, and performs feature extraction on the visible light and infrared images. Through a heterogeneous perception enhancement module, a channel-space bidirectional attention mechanism is constructed, cross-modal heterogeneous enhancement is carried out on features of different levels, key region information is highlighted, by means of a refined channel attention fusion module, discriminative features are effectively amplified, modal specific noise is suppressed at the same time, and in a target detection link, an elliptical dynamic CIoU loss function is adopted, so that the target detection accuracy is improved. According to the method, the external connection ellipse or the inconnection ellipse is dynamically selected according to the proportion of the target to calculate the intersection-to-union ratio, geometric alignment of the anchor frame and the target is optimized, the detection precision and robustness of the multi-modal image target are effectively improved, and the method has wide application prospects in the fields of automatic driving, anomaly detection and the like.

The invention discloses a visual inertial positioning method based on dynamic target detection and semantic information constraint, and belongs to the field of motion estimation and dynamic environment processing. According to the method, a dynamic target detection mechanism is introduced, the dynamic target is effectively detected based on the target detection network, inertial navigation information and geometric constraints, the dynamic target is effectively recognized in the image processing process, the corresponding dynamic feature points are screened out, the mismatching rate of the dynamic features is remarkably reduced, and high-quality observation input is provided for back-end optimization. In the back-end sliding window optimization stage, a semantic information consistency constraint method is constructed, and the estimation stability of the system in a weak texture area or a repeated texture area is enhanced by utilizing the consistency of feature points in the same semantic area on a geometric structure. Visual inertia pose estimation is realized based on dynamic target detection and semantic information constraint, and high-robustness and high-precision pose estimation can still be realized in a complex environment with dynamic interference of pedestrians, vehicles and the like and severe scene change.