Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

2381 results about "Global information" patented technology

Infrared and visible light fusion method based on multi-scale feature interaction enhancement

The invention provides an infrared and visible light image fusion method based on multi-scale feature interaction enhancement, belongs to the field of image processing, and aims to improve the image fusion quality in a complex environment. The method comprises the following steps: firstly, performing feature extraction on infrared and visible light images through a double-flow multi-scale encoder to generate multi-level features of different scales; and then, cross-modal information interaction is carried out by adopting a feature interaction enhancement module, fusion features are optimized through a space and channel attention mechanism, and clear expression of key areas such as target heat radiation and image edges is ensured. In the decoding stage, multi-scale jump connection and layer-by-layer up-sampling are utilized to reconstruct a high-resolution fusion image. And finally, training the fusion network through a content-gradient joint loss function to realize balance between detail reservation and global information. The method is suitable for tasks such as monitoring and target identification in a complex scene, and can effectively improve the image fusion effect especially under the conditions of low illumination, smoke shielding and the like.
Owner:TIANJIN UNIV

Abdomen multi-organ image segmentation method fusing multi-scale features

The invention provides an abdomen multi-organ image segmentation method fusing multi-scale features, and the method comprises the steps: S1, constructing a space detail fusion module extraction network, which comprises a space attention module, a channel attention module, an up-sampling operation, a down-sampling operation and a feature smoothing operation; s2, designing an encoder, and connecting the encoder with the space detail fusion module; s3, transmitting a plurality of features with different resolutions obtained by the encoder to a decoder through jump connection for information recovery; and S4, arranging the abdominal multi-organ medical image data set to perform model training, and performing multi-organ segmentation prediction on the abdominal multi-organ medical image. According to the method, local and global information is fully utilized, multi-scale feature information is combined, the image feature extraction and expression capability of the model is improved, and the segmentation precision of multiple organs of the abdomen is improved.
Owner:XIHUA UNIV

Model training and scene recognition method and apparatus, device, and medium

Provided is a method for training a scene recognition model. The scene recognition model includes a core feature extraction layer, a global information feature extraction layer, an LCS module of at least one level with an attention mechanism, and a fully-connected decision layer. The method includes: acquiring parameters of the core feature extraction layer and the global information feature extraction layer by training based on a first scene label of a sample image and a standard cross-entropy loss; training a weight parameter of the LCS module of each level, based on a loss value acquired by performing a pixel-by-pixel calculation on a feature map output from the LCS module of each level and the first scene label of the sample image; and acquiring a parameter of the fully-connected decision layer by training based on the first scene label of the sample image and the standard cross-entropy loss.
Owner:BIGO TECH PTE LTD

Remote sensing image super-resolution method and product based on diffusion model and multi-modal large language model

The invention provides a remote sensing image super-resolution method based on a diffusion model and a multi-modal large language model, and relates to the technical field of image processing. According to the method, a pre-trained stable diffusion model is adopted, the model is trained on a large-scale data set, the stable and detailed super-resolution effect can be achieved, and the convergence speed is higher. According to the method, a multi-mode large language model is used for enhancing the semantic comprehension ability of the model, and detailed content description and texture description are provided. The invention also introduces an additional classifier which provides global information and a simple signal about the LR. The prior information is adaptively integrated into a diffusion model through a cross attention module. And finally, through combination of the strong generation capability of the stable diffusion model and the semantic understanding capability of the MLLM, the method provided by the invention is excellent in the aspect of remote sensing image reconstruction, and excellent performance is realized in the aspect of quality of a plurality of images on different data sets.
Owner:PEKING UNIV

Medical image segmentation method and device based on spatial perception and frequency domain information

According to the medical image segmentation method and device based on spatial perception and frequency domain information, the precision and robustness of medical image segmentation are effectively improved by combining frequency domain information guidance and a multi-head state spatial perception technology. Frequency domain transformation is performed on a medical image, high-frequency and low-frequency components of the image are separated by using a multi-scale decomposition technology, and low-frequency features are extracted to obtain global information. By introducing a learnable noise filtering mechanism, noise and irrelevant background information in a frequency domain are suppressed, so that the model can be focused on a lesion area more accurately. A multi-head perception visual state space module is designed on a bottleneck layer, lesion features of different scales are captured through a multi-scale adaptive feature fusion mechanism, and the capability of segmenting small-size lesions and complex structures is enhanced. A context focusing attention mechanism is introduced into jump connection, fusion of global information and local details is further enhanced, and the accuracy of a segmentation result is ensured; and finally, recovering a high-resolution segmented image through a decoder.
Owner:XIAMEN UNIV OF TECH

Edge perception multi-prototype learning-based few-sample medical image segmentation method

The invention relates to the technical field of medical image segmentation, in particular to a few-sample medical image segmentation method based on edge perception multi-prototype learning, and the method comprises the steps: inputting support and query images into a feature encoder, and extracting support and query feature maps of different sizes; inputting into a local attention fusion prototype generator to generate a support foreground prototype; processing the support mask through dynamic corrosion operation to generate an inner boundary prototype; generating a multi-foreground local prototype through a multi-layer perceptron; local and global information is optimized through multi-scale feature extraction, and a multi-scale prototype is obtained; fusing to obtain a multi-prototype foreground prototype; dynamic calculation weighting is carried out on the multi-prototype foreground prototype by using a double-stage prototype optimization network, and automatic calibration is carried out; then prediction is carried out through a prototype prediction module, and finally collaborative optimization is carried out through a loss calculation module; the method can effectively solve the problem of edge detail loss involved in the background technology.
Owner:CHANGSHU FIRST PEOPLES HOSPITAL (CHANGSHU OCCUPATIONAL DISEASE HOSPITAL) +1

Efficient skin disease segmentation method based on multi-scale and mixed attention mechanism

The invention discloses an efficient skin disease segmentation method based on multiple scales and a mixed attention mechanism, and the method comprises the steps: firstly carrying out the preprocessing of a skin mirror image through data enhancement, and then inputting the skin mirror image into a convolutional neural network for feature extraction, thereby obtaining a skin lesion segmentation region prediction map. According to the method, the convolutional neural network is applied, multi-scale features of local information and global information can be extracted from a skin mirror image, and the feature representation capability is enhanced through feature fusion. The objective of the invention is to solve the problems of changeable lesion areas, irregular shapes and the like of skin disease images, and improve the accuracy and robustness of skin disease image segmentation.
Owner:CHONGQING NORMAL UNIVERSITY

Denoising diffusion model driven texture enhanced infrared and visible light image fusion method and system

The invention discloses a de-noising diffusion model driven texture enhanced infrared and visible light image fusion method and system, and relates to the technical field of image processing and computer vision. The method comprises the following steps: performing multi-scale space-time modeling on infrared and visible light images; enhancing high-frequency texture features of the infrared image through an edge detection convolutional network; based on a bidirectional multi-scale convolution module and a bidirectional attention fusion module, capturing a remote dependency relationship and encoding multi-scale information to ensure full utilization of global information and accurate capture of local details; and optimizing the structural fidelity and color intensity distribution of the fusion result based on an adaptive structural similarity loss function and multi-channel intensity and texture loss so as to improve the quality of the fused image. According to the method, the process of diffusion feature extraction is enhanced by fusing the de-noising diffusion model and the texture enhancement strategy. Meanwhile, a bidirectional multi-scale convolution technology and an attention mechanism are combined, so that comprehensive utilization of global information and accurate mastering of local details are ensured.
Owner:ZHEJIANG SCI-TECH UNIV

Aggregated neighborhood embedded image text matching method based on clustering prototype guidance

PendingCN119091170AImprove robustnessAlleviating the "soft positive sample" problemCharacter and pattern recognitionData setEngineering
The invention discloses an aggregation neighborhood embedding image text matching method based on clustering prototype guidance, relates to the technical field of computer vision and natural language processing, and provides a full-granularity prototype guidance neighborhood embedding framework. The framework dynamically learns a unified representation space through clustering prototype momentum contrast learning, and relieves a soft positive sample problem; global and local feature joint extraction and an adaptive aggregation network are introduced, so that the global information understanding capability is improved; an intra-modal correlation degree matrix and an inter-modal correlation degree matrix are constructed, and the neighborhood relation of sample instances is deeply mined; and by adopting a momentum contrast learning technology, the negative sample capacity is expanded, and the feature discrimination capability is improved. Experimental results show that the performance of the FPNE on Flickr30K and MS-COCO data sets is superior to that of an existing most advanced method, the image-text matching precision and efficiency are remarkably improved, and high robustness and wide applicability are shown.
Owner:CHONGQING NORMAL UNIVERSITY

Multi-mode brain anomaly detection method and system based on machine learning

The invention relates to the technical field of biomedical engineering, in particular to a multi-mode brain anomaly detection method and system based on machine learning. The method comprises the following steps: acquiring brain medical image data of different modalities, and realizing spatial registration and alignment through a multi-modal registration algorithm based on mutual information; a multi-branch feature extraction model including a convolutional neural network, a converter and a state space model is utilized to perform feature embedding on the original image of each modal; performing frequency decoupling on the features of each mode through adaptive approximate wavelet transform, and decomposing the features into high-frequency detail information and low-frequency global information; a frequency band fusion strategy based on an attention mechanism is implemented on high and low frequency features of different modal images, and fused frequency sub-band features are input into a space-frequency Mama module. Through the adaptive frequency domain decomposition and cross-modal fusion mechanism, the multi-modal brain image information is effectively integrated, and the accuracy and robustness of brain anomaly detection are remarkably improved.
Owner:NANCHANG HANGKONG UNIVERSITY

Variable working condition migration bearing fault diagnosis method based on ViTCapsNet network in noise environment

The invention discloses a variable working condition migration bearing fault diagnosis method based on a ViTCapsNet network in a noise environment, and the method comprises the steps: firstly obtaining vibration signal data of a rolling bearing in different fault states under different loads, and adding Gaussian white noises of different degrees; performing continuous wavelet transform on the data added with the noise to obtain a two-dimensional wavelet image, and dividing the two-dimensional wavelet image into a training set and a test set; a ViT Encoder model is established, position information capable of being trained is added by using an Embedded module, position features are embedded into the model, and global information in the features is extracted through a multi-head self-attention mechanism in the Encoder module; constructing a capsule network which comprises a primary capsule layer and a digital capsule layer, further extracting spatial information in the features, and performing accurate fault diagnosis; and finally, performing visual interpretation on the features processed by the Encoder module by using Grad-CAM, and completing efficient migration fault diagnosis on a test data set.
Owner:NANJING TECH UNIV

AGV obstacle tracking method based on multi-modal data and interactive Kalman filtering

The invention belongs to the field of outdoor AGV obstacle tracking, and particularly relates to an AGV obstacle tracking method based on multi-modal data and interactive Kalman filtering. Comprising the following steps: calibrating a monocular camera and a laser radar, and establishing point cloud data and image data with synchronous timestamps; performing preprocessing and target detection operation on the point cloud data and the target detection frame to obtain preprocessed point cloud data and a generated target detection frame so as to obtain obstacle category information; acquiring an obstacle observation position according to the preprocessed point cloud data and the target detection frame; meanwhile, an improved SORT multi-target tracking algorithm is adopted, and a historical track of the obstacle is established; through a plurality of Kalman filtering fusion methods, a trajectory prediction result is obtained through prediction; finally, the AGV communicates with a dispatching center, global information is obtained, and the track trend is corrected in a targeted mode in combination with obstacle category information; and the historical track of the obstacle and the corrected prediction track are sent to the AGV for obstacle avoidance path planning.
Owner:CHENYANG ROBOT IND DEVELOPMENT GROUP CO LTD

Improved YOLOv11 power transmission line defect detection method

The invention discloses an improved YOLOv11 power transmission line defect detection method, which introduces an attention mechanism to optimize the feature extraction capability, improves the performance of a model in a complex scene, remarkably enhances the understanding of the model for local and global information, and improves the target detection precision. Spatial offset is carried out on a feature map in a detection head by adopting feature extraction, feature association of different scales is enhanced, the detection precision of a small target is improved, redundant calculation is reduced, the calculation efficiency and performance of a model are further optimized, diversified requirements in a complex environment are met, and the method is suitable for popularization and application. The feature learning ability of a self-integrated attention head optimization target detection model is introduced, and the detection precision and robustness are improved under the scenes of small object detection and complex backgrounds. The method effectively improves the detection precision and training efficiency, can achieve excellent performance in a high-precision requirement and a real-time task, and meets the requirements of safety risk management and control in the power industry.
Owner:SICHUAN POWER TRANSMISSION & TRANSFORMATION CONSTR +2

Multi-modal mixed expert psychological health evaluation system, method, medium and equipment

The invention belongs to the field of multi-modal data processing, and provides a multi-modal mixed expert psychological health assessment system, method, medium and equipment in order to solve the problem of low feature fusion efficiency caused by insufficient dynamic interaction between modals of an existing mixed expert model in a multi-modal scene. The multi-modal mixed expert psychological health assessment system comprises a multi-modal data acquisition module, and an adaptive grouping module. The cross attention fusion module is used for extracting features of corresponding modal data in the basic group and the auxiliary group, performing intra-group cross attention fusion on the features of each group, and generating intra-group features; performing global information fusion on all the intra-group features through a cross-group cross attention mechanism to obtain cross-group features; and after the cross-group features, the intra-group features and the original features of the multi-modal data are spliced, a dynamic gating weight is obtained through a gating network. The method can provide more accurate guidance for psychologists.
Owner:SHANDONG JIANZHU UNIV

Student classroom behavior detection method based on deep learning

The invention belongs to the technical field of target detection, and particularly relates to a student classroom behavior detection method based on deep learning. The method comprises the following steps: training an improved YOLOv11 model, and pruning the trained improved YOLOv11 model to obtain a student classroom detection model; acquiring a to-be-detected student classroom video and inputting the to-be-detected student classroom video into the student classroom detection model for processing to obtain a student classroom behavior detection result; the improved DeepSORT algorithm is used for processing the student classroom behavior detection result, tracking of the student classroom behaviors is achieved, the global information capturing capacity of the network is enhanced, the performance of the network under the shielding and multi-scale change scene is improved, and therefore the robustness and detection precision of the network are improved, and the method has good application prospects.
Owner:CHONGQING UNIV OF POSTS & TELECOMM

Infrared weak and small target detection method based on double-branch attention mechanism

The invention belongs to the field of target detection, and particularly relates to an infrared weak and small target detection method based on a double-branch attention mechanism. The method comprises the following specific steps: step 1, preparing data sets: performing data enhancement on a first infrared data set and a second infrared data set for pre-training, and performing model fine adjustment on a third infrared data set; 2, constructing a network model, wherein the whole model is composed of a target analysis path, a splicing path and a target reconstruction path; according to the method, a global semantic enhancement large-kernel attention module is introduced into a coding module of a target analysis path, global information is obtained and utilized from four angles of a channel, a transverse direction, a longitudinal direction and a receptive field, and a double-branch single-head attention module is introduced into a splicing path to obtain association degrees among different scales; the purpose of eliminating the semantic difference between the target analysis path and the target reconstruction path is achieved, the difference between the weak and small target and the background is expanded, and the detection precision of the infrared weak and small target is improved.
Owner:CHANGCHUN UNIV OF SCI & TECH

Pyramid vision Transform-based polyp image segmentation method

The invention relates to a polyp image segmentation method based on pyramid vision Transform, and belongs to the technical field of digital medical treatment. The method comprises the following steps: capturing global context information through a pyramid vision Transform module, and obtaining feature maps with different resolutions from a plurality of extraction stages; detail information is extracted from the shallow feature map of the first stage through a cyclic cross attention module, and shallow features are obtained; a multi-scale feature fusion sensing module is used for carrying out fusion processing on the feature maps extracted in multiple stages, deep features are extracted, and the deep features and shallow features are fused and then used for image segmentation; and designing weighted binary cross entropy loss and weighted IoU loss to train the above process, and segmenting the polyp image after training is completed. According to the invention, through the pyramid vision Transform module, the cyclic cross attention module and the multi-scale feature fusion perception module, the global information learning ability, the detail information capturing ability and the segmentation performance are significantly improved.
Owner:CHONGQING UNIV OF POSTS & TELECOMM

Self-adaptive bilateral distillation personalized federal learning method based on diffusion model

The invention belongs to the technical field of federated learning, and particularly relates to a self-adaptive bilateral distillation personalized federated learning method based on a diffusion model. According to the method, a guidance mechanism is designed, so that mutual distillation is carried out between a global model and a local model, efficient transmission of knowledge is realized, and the adaptability of a personalized model to specific data distribution of a client is enhanced. Meanwhile, a conditional diffusion model is introduced to generate high-quality pseudo data, and the aggregated global model is finely adjusted by using the pseudo data. The process not only effectively makes up global information possibly lost in the local-global mutual distillation process, but also further optimizes the performance of the global model. By combining the mutual distillation and conditional diffusion fine tuning technology, the data privacy is protected, meanwhile, the balance between the personalized performance and the global generalization ability is achieved, and the method is suitable for a multi-client cooperation scene in a non-independent identically-distributed non-IID data environment.
Owner:QILU UNIVERSITY OF TECHNOLOGY (SHANDONG ACADEMY OF SCIENCES) +1

Power load prediction method and system based on multi-modal and multi-scale feature fusion

The invention provides a multi-modal multi-scale feature fusion-based power load prediction method and system, and relates to the technical field of power load prediction. According to the method, a cross-modal attention mechanism is adopted to fuse multi-scale load time sequence features and satellite image features; decoding the fusion features through a multi-step prediction decoder to obtain a multi-step power load prediction result; the acquisition mode of the satellite image features comprises the following steps: performing feature extraction on a satellite image based on a GRVT model to obtain the satellite image features; the GRVT model includes a number of stacked gated residual encoder network elements. According to the method, satellite image feature extraction is carried out based on the GRVT model, the global information processing advantage of Transform is utilized, the single-layer information retaining capability of a residual connection mechanism is also utilized, the two are combined to efficiently extract satellite image features, the satellite image features are fused during power load prediction, and the situation that the data type is single, and the prediction efficiency is high is avoided. And the precision of power load prediction is improved.
Owner:HEFEI UNIV OF TECH

Breast cancer histological image segmentation and classification method based on multi-scale deep learning

The invention discloses a breast cancer histological image segmentation and classification method based on multi-scale deep learning. The method comprises the following steps: firstly, carrying out data acquisition and preprocessing; then, a breast cancer deep full-slice image-interstitial segmentation and classification model is constructed, and the model adopts a multi-scale attention fusion network architecture and comprises a feature extraction module, an attention mechanism module, a feature fusion module and a segmentation and classification module; and carrying out model training through the preprocessed data, finally completing breast cancer histological image segmentation and classification through the trained model, and carrying out Grad-CAM visualization. Through an innovative multi-scale deep learning architecture and technical measures, the automation degree, precision and robustness of breast cancer histological image segmentation and classification are remarkably improved, and powerful technical support is provided for clinical application. Compared with the prior art, the method has the advantages that higher automation level, higher segmentation and classification precision and higher global information utilization capability are realized functionally.
Owner:HANGZHOU DIANZI UNIV

Target tracking method and system based on grouping attention feature extraction network

The invention provides a target tracking method and system based on a grouped attention feature extraction network, and the method comprises the steps: obtaining a template and search features based on a template and a search image through employing the grouped attention feature extraction network; learning local areas of the template target image features and the search image features, and aggregating to obtain target information; splicing the template with the search features to obtain the input of the self-attention encoder in the asynchronous interaction stage; the fused features are combined with a prediction head for calculation, a classification regression result is obtained, and a target is predicted according to the classification regression result; training the target tracking model in combination with the large-scale data set to obtain a trained target tracking model, and performing accurate target tracking on the target by using the trained target tracking model; according to the multi-scale feature extraction network based on Transform, sequential local and global information interaction in the windows and between the windows is promoted, and the optimal balance between performance and speed is achieved.
Owner:NANCHANG INST OF TECH

Micro-expression recognition method based on cross-source double-branch dynamic space-time diagram convolutional network model

The invention relates to a micro-expression recognition method based on a cross-source double-branch dynamic space-time diagram convolutional network model, and belongs to the technical field of deep learning and pattern recognition. According to the method, a cross-source double-branch twin space-time diagram structure network is provided to mine subtle motion features of a facial structure when expressions change, and domain invariant features are learned through a twin structure. According to the design, facial global information is modeled through a global information and dynamic spatio-temporal feature extraction network, fine motion information of a facial key structure is extracted through an attention enhancement twin spatio-temporal diagram fusion network, facial structure association when expressions occur is fully modeled, and domain adaptation is performed by constructing cross-domain joint loss.
Owner:SHANDONG UNIV

Overflow surface defect detection and identification method and system based on multi-modal feature and prompt mechanism

The invention relates to the technical field of industrial defect detection, and discloses an overflowing surface defect detection and identification method and system of a multi-modal feature and prompt mechanism, and the method comprises the steps: collecting image data and point cloud data of the surface of a water turbine runner overflowing surface, and converting the point cloud data into a surface defect two-dimensional image; obtaining a high-resolution defect image based on an image super-resolution reconstruction method of a deep recursive network; inputting a defect image into the generative adversarial network, judging a data source, generating a high-quality defect image through an objective function optimization generative model, and expanding a data set for a defect identification detection model to use; and inputting the point cloud data into the PDE-Net to construct a dynamic adjacency graph, and mapping the point cloud data to a specific defect category after graph convolution and multi-layer feature extraction. The method has the beneficial effects that a mixed attention mechanism is introduced in a multi-modal feature extraction stage, global information and local details in a defect image are fully extracted, and the detection capability of complex defects is improved.
Owner:SICHUAN HUANENG TAIPING YI HYDROPOWER CO LTD +1

Semi-supervised cardiac image segmentation method based on Mamba-Transform double structure and contrast learning

The invention belongs to the technical field of medical image processing, and particularly relates to a semi-supervised cardiac image segmentation method based on a Mamb-Transform double structure and contrast learning. A segmentation model used in the method comprises an MTSeg encoder and a VNet decoder, an input image enters the MTSeg encoder after being subjected to linear layer and position embedding operation, the MTSeg encoder comprises a plurality of MTSeg encoding modules, the VNet decoder comprises a plurality of decoding modules, and output features of the MTSeg encoding modules are in jump connection with the corresponding decoding modules after being subjected to projection operation; performing normalization operation on the output feature vector of the VNet decoder to obtain a segmentation result; the MTSeg coding module comprises an improved Mama branch and a Transform branch, and output feature vectors of the two branches are fused through a cross attention layer to obtain an output feature vector of the MTSeg coding module; the improved Mama branch comprises an MS module and a DRFB module which are connected in series, the MS module is used for enhancing the feature extraction and calculation efficiency of the model, and the DRFB module is used for enhancing the expression ability and training stability of the model. The method achieves the effective integration of a long-range dependency relationship, global information, multi-scale information and local information, improves the feature extraction capability of the model, and obtains a finer and more accurate segmentation result.
Owner:HEBEI UNIV OF TECH

Multi-modal data fusion method and device, computer equipment and storage medium

The invention relates to a multi-modal data fusion method and device, computer equipment and a storage medium. The method comprises the following steps: carrying out high-order feature extraction on multi-modal data to be fused by utilizing a special modal encoder to obtain high-order embedding features of the multi-modal data; the high-order embedded features are decoupled in combination with expert knowledge constraints and a multi-head attention mechanism, and high-frequency features representing local information and low-frequency features representing global information are obtained respectively; and aligning the low-frequency features by using a cosine alignment algorithm, performing orthogonal projection on the high-frequency features by using an orthogonal projection technology, and performing multi-level fusion on the aligned low-frequency features and the orthogonally projected high-frequency features through an adaptive deep fusion network to obtain fused multi-modal joint features. According to the method, the problems of heterogeneity and isomerism of the data are solved, effective fusion of different modal data is ensured, and the accuracy and individuation capability of disease risk early warning can be remarkably improved.
Owner:SHENZHEN INST OF ADVANCED TECH CHINESE ACAD OF SCI

Multi-mode unmanned aerial vehicle image segmentation method

The invention discloses a multi-mode unmanned aerial vehicle image segmentation method. The method comprises the steps of 1, constructing an unmanned aerial vehicle image semantic segmentation base network; 2, capturing high-resolution local features in a shallow layer through a convolutional neural network (CNN) module, extracting from deep features of the image through a multi-head self-attention mechanism in a transformer module, and fusing the local features with deep global features through jump connection; 3, constructing a parallel convolution module, and optimizing the features fused in the step 2; 4, combining feature extraction results in the step 2 and the step 3, fusing feature information through a cross attention mechanism, and improving a segmentation effect in a complex scene; and 5, constructing a double attention module, and optimizing local and global information processing of the image. The method has the characteristics that the global and local features are efficiently fused, the multi-scale target is accurately segmented, and the robustness in a complex scene is remarkably improved.
Owner:XIDIAN UNIV +1

Robot visual touch perception method and system based on transformer and comparative learning

The invention discloses a robot visual touch sensing method and system based on transformer and comparative learning, and belongs to the technical field of robots. According to the method, an enhanced Swin Transform model is introduced to serve as a sensing model, the enhanced model takes Swin Transform as a backbone network, a coordinate attention mechanism is introduced into the backbone network, global information is extracted through a sliding window attention mechanism of the backbone network, local feature extraction is enhanced through the coordinate attention mechanism, and the global information is extracted through the sliding window attention mechanism. The global and local features in the visual and tactile data can be effectively extracted and fused. According to the enhanced Swin Transformer model, in-modal comparative learning and inter-modal comparative learning are specifically performed through multi-modal comparative learning training in a pre-training stage of an earlier stage, and richer and more accurate discrimination features are generated, so that the intelligent sensing and operation capability of the robot to the environment is effectively improved. According to the method, the features can be effectively extracted in multi-modal data fusion, and deeper multi-modal data fusion is realized by utilizing the correlation and synergistic effect of the features among different modals.
Owner:SHENZHEN INST OF ADVANCED TECH

Multi-modal named entity recognition method based on semantic alignment and cross-modal graph fusion

The invention belongs to the technical field of natural language processing and multi-modal information extraction, and particularly relates to a multi-modal named entity recognition method based on semantic alignment and cross-modal graph fusion, which comprises the following steps: S1, acquiring a data sample containing a text sequence and image content; s2, encoding the text and the image into vectors respectively; s3, similarity is calculated through a trainable bilinear function, and optimization is carried out through loss comparison; s4, cross-modal attention is used to enhance association information between modals; s5, determining the proportion of reserved image information through a modal matching module; s6, introducing a gating mechanism to dynamically fuse visual and text features; s7, realizing local and global information complementation by a cross-modal graph fusion model; and S8, inputting the fused representation into the CRF layer to predict the entity type. According to the method, fine semantic alignment can be realized in a weak image-text correlation context, and balance between local entity recognition and global semantic understanding can be achieved.
Owner:ANHUI UNIVERSITY OF TECHNOLOGY

Space-air network layered cooperation route optimization method and system

The invention discloses an aerospace network hierarchical cooperation routing optimization method and system, and relates to the field of aerospace information network communication, and the method comprises the steps: constructing a system model comprising an unmanned plane, satellite nodes and a communication link, defining link attributes, building a routing optimization objective function, introducing a reputation value-based multi-agent cooperation mechanism, and carrying out the routing optimization of the unmanned plane, the satellite nodes and the communication link. A hierarchical reinforcement learning framework is designed, inter-cluster decision and intra-cluster decision are divided, an upper layer selects an inter-cluster path according to global information, a lower layer plans an unmanned aerial vehicle path and defines a state, an action space and a reward function according to local information, an intelligent agent interacts with an environment to collect empirical data, and optimal routing communication is selected in combination with a real-time reputation value after training. Adjustment can be carried out according to network changes; according to the invention, the cooperative reliability is improved through a reputation value mechanism, the decision complexity is reduced through hierarchical learning, the routing performance is optimized, the network robustness, expandability and adaptability are enhanced, and the cooperative communication efficiency of the unmanned aerial vehicle and the satellite and the system stability are improved.
Owner:BEIJING UNIV OF POSTS & TELECOMM +2

Aircraft 4D trajectory prediction method and system based on double-attention space-time diagram convolution

The invention discloses an aircraft 4D trajectory prediction method and system based on double-attention space-time diagram convolution, and belongs to the technical field of intelligent time sequence prediction. According to the method, the self-attention mechanism is utilized to reconstruct the predefined adjacent matrix, and optimization is continuously carried out along with network training so as to improve generalization of the network. The problem that the distance-based predefined adjacency matrix cannot fully reflect the potential relationship between the aircrafts in the dynamic scene is effectively solved. On the basis of a learnable adjacency matrix based on a self-attention mechanism, the correlation between nodes is further aggregated by using the graph attention network, and additional feature extraction is performed on the space-time diagram based on the powerful global information aggregation capability of the graph attention network, so that the model can capture more potential relationships between the nodes, and the accuracy of feature extraction is improved. And the problem that the correlation between the nodes cannot be effectively aggregated by the traditional graph convolution operation is improved. And the prediction accuracy of the future trajectory of the aircraft is further improved.
Owner:UNIV OF ELECTRONICS SCI & TECH OF CHINA