62 results about "Multimodality image registration" patented technology

The invention discloses a surgical robot resection navigation method based on multi-modal image registration. The method comprises the following steps: acquiring a preoperative multi-modal medical image of a patient and reconstructing a three-dimensional model containing a focus and a dangerous structure; acquiring an intraoperative real-time image; extracting cross-modal invariant features from the pre-operative and intra-operative images by using a first deep learning network, and establishing an initial corresponding relation; predicting an initial non-rigid deformation field by using a second deep learning network, based on the pre-operative / intra-operative image and the initial corresponding relation and in combination with physical model constraints; tracking a surgical tool in real time, and updating a non-rigid deformation field in real time by adopting an incremental calculation strategy; applying the updated deformation field to the preoperative three-dimensional model to generate a deformed model; and mapping the tool position to the deformed model, and generating a navigation instruction. According to the method, soft tissue deformation can be accurately compensated, multi-modal image information is effectively fused, navigation information is dynamically updated in real time, and the precision and safety of robot surgery are remarkably improved.

The invention discloses a multi-modal image registration method based on self-modal correlation and cross-modal estimation. The method comprises the following steps: firstly, constructing an initial frame model composed of image frequency decomposition, self-modal similarity coding, cross-modal matching estimation, optical flow generation and refinement and an image transformation module, and then performing spatial transformation on registered visible light and thermal infrared images to construct a training set; and training the model to obtain a visible light and thermal infrared registration model, and performing multi-modal image registration on visible light and thermal infrared images through the visible light and thermal infrared registration model. According to the method, the self-modal similarity of the visible light and thermal infrared images can be fully constructed, the accuracy of cross-modal feature matching estimation is improved, and high-precision registration of the visible light and thermal infrared images with remarkable view angle difference and modal difference is realized; and in addition, the visible light and thermal infrared images can still be stably registered under severe environment conditions such as night, rainy days and foggy days, and the robustness is relatively high.

The invention discloses a multi-modal image registration method and system based on deformation adaptation and computer equipment, and the method comprises the steps: collecting a plurality of groups of multi-modal images, carrying out the gray standardization, and constructing a diversified registration data set; building a registration network model comprising a pyramid coding module, a deformation adaptive module, a cross-modal interaction module and a registration parameter estimation module; inputting an image pair into the modules in sequence, respectively extracting basic feature mapping, deformation feature mapping and interaction enhancement feature mapping, and finally outputting an estimation conversion parameter matrix; a training process is supervised through a preset loss function, optimal network parameters are selected, and a trained registration model is obtained; in practical application, an image pair to be registered is input into the trained model, a conversion parameter matrix is obtained, and image registration is completed. The multi-modal image registration performance can be effectively improved, and the method still has good robustness and adaptability especially under the condition that serious geometric distortion and significant modal difference exist.

Disclosed in the present invention are a multi-modal image registration method and system based on iterative optimization. The method comprises: acquiring several low-light images and processing same, in order to obtain a training set and a validation set; constructing a multi-modal image registration neural network model; training the multi-modal image registration neural network model by means of the training set, and using a loss function to calculate loss, in order to obtain a trained multi-modal image registration neural network model; using the validation set to perform iterative optimization on the trained multi-modal image registration neural network model, and determining whether an iteration termination condition is met, in order to obtain an iteratively optimized multi-modal image registration neural network model; and acquiring multi-modal images in a real scene, forming image pairs to be registered, and inputting said image pairs into the iteratively optimized multi-modal image registration neural network model for processing, in order to obtain registered and fused images. The method can improve the robustness of image pairs, which each consist of a reference image and an image to be matched, during a registration process.

The invention provides a large-scale multi-modal image registration method and system based on structured enhancement driving, and belongs to the field of image registration, and the method comprises the steps: collecting images of different modals, and obtaining a to-be-registered image data set which comprises a to-be-registered image and a reference image; randomly selecting two images with different modalities from the to-be-registered image data set, and extracting two feature map pairs with different downsampling resolutions; obtaining a rough feature map and a reference anchor point for the feature map pair with one resolution; performing structure enhancement on the rough feature map by using a reference anchor point to obtain an enhanced feature map; calculating a matching probability between the enhanced feature maps, and obtaining a rough matching result; obtaining a fine matching result by combining the rough matching result with the feature map pair of the other resolution; and calculating a transformation matrix of the image by using the fine matching result, transforming the to-be-registered image, and carrying out sub-pixel level alignment on the to-be-registered image and the reference image to obtain a final registration result.

The invention relates to the technical field of image processing, in particular to a cerebrovascular image recognition and analysis system based on deep learning. The system comprises a cerebrovascular image registration module, a CT pixel communication module, a lesion area identification and segmentation module and a lesion risk assessment module, and can obtain a cerebrovascular CT angiography image, perform image enhancement and multi-modal image registration, and generate a cerebrovascular CT multi-modal registration image sequence; acquiring cerebrovascular CT pixel point clouds in the corresponding modals and performing blood vessel cluster connectivity analysis to generate corresponding cerebrovascular CT pixel cluster connectivity graphs in the respective modals; and based on a deep learning multi-branch network architecture, introducing an attention mechanism to carry out lesion region identification and segmentation, obtaining a corresponding cerebrovascular stenosis degree and a lesion thrombus area, and carrying out lesion risk number evaluation on the corresponding cerebrovascular CT image lesion region to obtain a cerebrovascular CT lesion risk number. According to the invention, accurate recognition of the cerebral vascular image lesion condition can be realized.

The invention discloses a multi-modal image registration method and system based on multi-scale feature fusion and Bayesian regression, and the method comprises the steps: a double-flow feature coding module processes a fixed image and a to-be-corrected image, and generates a multi-scale feature map; the Bayesian deformation prediction module generates a deformation field sample by using a Monte Carlo random inactivation mechanism based on decoder features, and directly outputs prediction during training. The multi-scale deformation fusion module integrates deformation fields of different levels, generates a global dense deformation field through resolution alignment, cross-scale attention enhancement and adaptive weight fusion, and balances a global trend and local details. The progressive feature refinement module fuses distortion features, decoder up-sampling features and fixed image features, optimizes feature consistency through dynamic weight distribution and a cross-dimension attention mechanism, and drives iterative optimization. And gradually improving the precision through multi-scale iteration, and finally outputting an optimal registration field. The method has the advantages that the cross-modal feature consistency is enhanced, and the robustness to noise and deformation is improved.

The invention discloses a multi-modal image registration and fusion method and system based on deep learning, and the method comprises the steps: generating a pseudo-infrared image through a cross-modal image generation network, so as to reduce the modal difference between source images, and precisely estimating a deformation field between the images through a multi-scale registration network; collaborative optimization of a registration network and a fusion network is realized by adopting a joint training framework, and semantic information of a fusion image is enhanced by applying a semantic-guided fusion framework. The technology is mainly used in the fields of automatic driving, video monitoring, medical imaging and the like, provides richer and more accurate scene description than a single sensor, and particularly has important application value in the aspects of day and night monitoring and target recognition.

The invention discloses an unsupervised multi-modal image registration method and system based on dual-path training, and the method comprises the following steps: obtaining a plurality of multi-modal image data, so as to construct a multi-modal image pair data set; constructing a multi-modal registration model; based on a registration network and a translation network, performing dual-path collaborative registration on the floating image to obtain a dual-path registration image; inputting the reference image and the dual-path registration image into a discriminator network for discrimination; a composite loss function is constructed, and a target function is established by using a minimum-maximum game method; training a multi-modal registration model based on the target function and the multi-modal image pair data set; according to the method, the dual-path collaborative registration framework is constructed through the registration network and the translation network, and the image alignment problem of different imaging modes is converted into the registration problem in the same mode, so that the problems of semantic gaps and measurement misalignment in cross-modal feature matching are solved, and the problem of cross-modal similarity measurement is avoided.

The invention belongs to the technical field of image processing, and relates to a different-source image registration method, system and device for large and small scenes and a medium, and the method comprises the steps: carrying out the n * n cutting of a large-scene image, and determining a target image block through the correlation index calculated through an NCC algorithm and the number of matching points calculated through a feature matching method, finally, the OS-SIFT algorithm is used for further completing registration of different-source images, the test result of visible light-thermal infrared data sets with different scene sizes shows that both the success rate and the registration precision of the method can meet the current practical application requirements, and the problem of multi-modal image registration under the condition that the scene sizes are greatly different can be solved.

The invention provides a multi-modal image registration method and system based on a deformation field, a storage medium and equipment, and the method comprises the steps: obtaining multi-modal images, including a 2D ultrasonic image and a 3D reconstruction image, and carrying out the coarse registration; obtaining a 2D ultrasonic image sequence of continuous frames, and obtaining rigid change parameters of the organ in a respiratory cycle; acquiring organ contour feature data points of each frame of 2D ultrasonic image, and performing elastic registration to obtain a 2D elastic deformation field of the organ; constructing a 2D-3D deformation function of the organ, and obtaining a 3D deformation field in a three-dimensional space; and updating vertex coordinates of the target organ in the 3D reconstruction image according to the 3D deformation field to obtain a mapped three-dimensional image to represent rigid and elastic changes of the organ. According to the method, by fusing the multi-modal image information and combining rigid and elastic deformation analysis, accurate registration of the ultrasonic image and the CT / MRI image is achieved, and the precision of multi-modal medical image registration is effectively improved.

The application discloses a kind of guided path planning method and system.Method includes following steps:S1.MRI image and ultrasound image are obtained, multi-modal image registration and segmentation are carried out, and MRI image and ultrasound image are fused.S2.real-time modeling of electric field distribution, based on dynamic conductivity model and finite element method, the electric field intensity distribution of cartilage and surrounding tissue after electrode applied pulse is calculated in real time, to provide input for subsequent pulse parameter optimization and path planning.S3.pulse parameter adaptive optimization, through model predictive control algorithm, real-time adjustment pulse voltage and pulse width, so that the electric field intensity of ablation area accurately covers target cartilage, while protecting blood vessel / nerve sensitive area from damage.S4.restrained path search is carried out, and guided path planning is realized.

Systems / techniques that facilitate multi-modal image registration via modality-neutral machine learning transformation are provided. In various embodiments, a system can access a first image and a second image, where the first image can depict an anatomical structure according to a first imaging modality, and where the second image can depict the anatomical structure according to a second imaging modality that is different from the first imaging modality. In various aspects, the system can generate, via execution of a machine learning model on the first image and the second image, a modality-neutral version of the first image and a modality-neutral version of the second image. In various instances, the system can register the first image with the second image, based on the modality-neutral version of the first image and the modality-neutral version of the second image.

A method for aligning multiple depth cameras in an environment based on image data can include accessing, by one or more processors, a first plurality of point cloud data points corresponding to a first pose for a subject and a second plurality of point cloud data points corresponding to a second pose for the subject. The method can include determining, by the one or more processors, a frame of reference for the image data based on at least one of the first pose or the second pose. The method can include transforming and aligning, by the one or more processors, at least one of the first plurality of point cloud data points or the second plurality of point cloud data points to the frame of reference.

The disclosure generally relates to methods and systems for image co-registration of multi-modal temporal sensing. Conventional techniques for image co-registration of multi-modality images focus on either the spatial or temporal domain and thus are not of high accuracy and do not preserve both global and local characteristics for the matching. The present disclosure solves the technical problems in the art for image co-registration of multi-modal temporal sensing using a deep-learning based multi-input-output encoder-decoder network with a Gabor Jet Model. The deep-learning based multi-input-output encoder-decoder network is utilized for the feature extraction. A distinctive Gabor-jet layer of the Gabor Jet Model is utilized for the similarity matching. The Gabor-jet layer generates a Gabor jet graph which provides sparse feature points for matching between matching images and reference images.

The present invention discloses a three-dimensional CT / PET image registration method based on mutual information and L-BFGS optimization, which belongs to the technical field of calculation, inference or counting. The registration method of the present invention is a two-stage cascade process of rigid coarse registration and B-spline elastic fine registration. It utilizes the precision advantage of mutual information measurement and noise reduction preprocessing to solve the problem of lack of precision of traditional multimodal image registration methods due to data modality and grayscale differences. It also adopts L-BFGS to improve the optimization algorithm, which greatly reduces the number of iterative calculations in traditional registration, gives full play to the speed advantage of the second-order convergence optimization algorithm, and greatly reduces the registration time. In summary, the registration method of the present invention shows high precision and speed advantages in the three-dimensional CT / PET multimodal image registration problem.

The invention relates to the technical field of industrial machine vision detection, in particular to an intelligent grading and feedback system based on image recognition. Comprising a multi-modal image acquisition unit used for synchronously acquiring a visible light image, an infrared thermal imaging image and a hyperspectral image of a target object; the dynamic self-adaptive preprocessing unit is in communication connection with the multi-modal image acquisition unit, performs combined preprocessing on the acquired multi-modal image, and comprises a dynamic noise reduction algorithm based on target edge features, a self-adaptive contrast enhancement algorithm and a multi-modal image registration algorithm; and the closed-loop feedback unit is respectively in communication connection with the multi-modal image acquisition unit and the multi-dimensional dynamic threshold grading unit. Through multi-modal image acquisition and attention-multi-scale feature fusion design, the limitation of traditional single-modal detection is effectively broken through, the recognition capability of micro bubbles and hidden cracks of the quartz crucible is remarkably improved, surface stains and internal impurities are accurately distinguished, and the problem of subjective difference of manual visual detection is solved.

The invention discloses an equipment fault source positioning method and device based on multi-modal information fusion, and the method comprises the steps: carrying out the data collection of a power distribution room inspection robot which carries an infrared thermal imager, a visible light camera and a self-adaptive lifting mechanism, carrying out the preprocessing of a collected infrared image, constructing a power distribution room equipment data set, and carrying out the positioning of a power distribution room equipment fault source. The power distribution room equipment data set comprises a plurality of image pairs composed of infrared images and visible light images which are in one-to-one correspondence; performing multi-modal image registration on each image pair based on an improved whale optimization algorithm to obtain a registered image pair; fusing the registered image pair by using a deep learning fusion network containing a coordinate attention module to obtain a fused image; and carrying out fault source positioning based on the relative temperature difference and a Monte Carlo method according to the fused image. According to the invention, accurate identification and grade determination of a fault source are realized through four stages of processes of multi-modal data acquisition, accurate image registration, high-quality image fusion and automatic fault positioning.

The present disclosure provides a method and system for processing multi-modality images. The method may include obtaining multi-modality images; registering the multi-modality images; fusing the multi-modality images; generating a reconstructed image based on a fusion result of the multi-modality images; and determining a removal range with respect to a focus based on the reconstructed image. The multi-modality images may include at least three modalities. The multi-modality images may include a focus.

The present invention provides a multimodal image registration method, apparatus, electronic device, and storage medium. The method includes: performing multi-scale edge window filtering on a first image to be registered and a second image to be registered, respectively, to obtain multi-scale edge window filtered result images of the first image to be registered and the second image to be registered; based on the edge window filtered result images, obtaining a first feature point and a feature vector corresponding to the first feature point, as well as a feature vector corresponding to the second feature point; based on the feature vectors, performing feature matching twice on the first feature point and the second feature point to obtain a first pair of feature points with the same name and a second pair of feature points with the same name; and registering the first image to be registered and the second image to be registered based on the first pair of feature points with the same name and the second pair of feature points with the same name. The present invention provides a multimodal image registration method, apparatus, electronic device, and storage medium, which improve the accuracy and efficiency of multimodal image registration and enhance the robustness of multimodal image registration.

The present invention relates to the technical field of image processing, and particularly relates to a multi-modal retinal fundus image registration method and system. The reference image structure is passed through a first feature encoder to extract the reference rotation-equivariant features in four stages and the total reference rotation-equivariant feature map; the floating image structure is passed through a second feature encoder to extract the floating rotation-equivariant features in four stages and the total floating rotation-equivariant feature map; the reference rotation-equivariant features and the floating rotation-equivariant features in the four stages are respectively input into a key point detection decoder for feature fusion, and a reference key point prediction map and a floating key point prediction map are output; the total reference rotation-equivariant feature map and the total floating rotation-equivariant feature map are respectively input into a key point descriptor decoder to output the descriptor of each reference key point and the descriptor of each floating key point; through a registration module, the floating image is aligned to the reference image to obtain the registered image. The present invention improves the registration accuracy of multi-modal images.

The present invention discloses a multimodal image registration method, system and computer equipment based on feature decoupling, comprising the following steps: 1. Acquire and process a plurality of multimodal images to obtain a multimodal image registration data set with diversity; 2. Build a multimodal image registration neural network model based on feature decoupling; 3. Design a loss function to supervise the training of the multimodal image registration neural network model and optimize the registration process; 4. Use the trained multimodal image registration neural network model to process the multimodal image to be registered to obtain an image conversion parameter matrix, and convert the image to be registered in the image pair according to the image conversion parameter matrix to obtain the registered image. The multimodal image registration method based on feature decoupling proposed by the present invention can effectively improve the performance of multimodal image registration, especially in the case of large differences between different modalities, and exhibits good robustness.

The invention discloses an industrial product visual inspection system and method based on image processing, and relates to the technical field of industrial product visual inspection. According to the invention, through improving the SIFT-PSO registration algorithm and combining the hyperspectral, thermal imaging and 3D point cloud data acquired by multiple sensors cooperatively, multi-modal image registration with pixel-level precision is realized; an adaptive illumination compensation and noise suppression technology is adopted, so that the image quality is improved; by improving an LBP-TOP algorithm and combining geometric curvature features and heat distribution features, multi-dimensional information feature expression is constructed, and calculation load is reduced through a feature compression technology; a ResNet model is constructed, a network weight and a transfer learning mechanism are dynamically optimized through a genetic algorithm, and the accuracy and generalization ability of defect detection are improved; the method is suitable for product defect detection in a complex industrial environment, has the characteristics of high robustness, high precision and efficient processing, and realizes double breakthrough of detection precision and real-time performance.

The invention discloses a multi-modal image registration method and device based on a generative model and self-supervised learning, and a medium, and the method comprises the steps: carrying out the modal conversion of a source image through employing a Pix2Pix-Transform generative model, extracting global and local features from the source image and a target image through a DINO model in an image registration process, carrying out the region division of the image based on the features, and carrying out the recognition of a target image through a DINO model; for each region, weighted distribution is carried out according to the feature intensity and importance of the region, so that the key region can obtain higher weight and precision in the registration process, and the weight of the background region can be properly reduced. A region self-adaptive registration technology is adopted, high-precision non-linear registration is carried out on a key region, low-precision linear registration is carried out on a background region, and the overall registration efficiency is ensured. According to the self-adaptive weighted registration method, the overall precision and efficiency of image registration are effectively improved, and particularly, when a multi-modal image is processed, the registration difficulty caused by modal difference can be remarkably reduced.

The invention discloses an automatic multi-modal image registration method and system. The method comprises the following steps: initializing and preprocessing a registration image; extracting feature points of the registered image, and calculating descriptors; matching descriptors of the registered image pairs; calculating an affine matrix; and performing affine matrix processing on the to-be-registered image to obtain a registered image. According to the method, the registration problem of an endogenous imaging technology, a light field imaging technology, a tissue slicing technology and the like which cannot be solved in the prior art can be solved at the same time, fine adjustment is not needed for different modes, zero sample learning can be achieved, and the method can be suitable for imaging conditions of different scales from mesoscopic imaging to microscopic imaging and the like.

The application discloses a kind of multi-modal image registration methods based on disparity estimation.The specific steps are as follows:(1) build array imaging system;(2) construct data set;(3) input image into neural network, adopt double branch strategy, extract common features and unique features;(4) use channel attention enhancement module for feature enhancement, construct matching cost based on disparity, and get disparity map through disparity regression;(5) use the obtained disparity map, align images through homography distortion, realize the registration and fusion of multi-modal images;(6) construct loss function, including mean absolute loss function and least square generative adversarial loss function;(7) input the image of test set into neural network, get style generation graph, predicted disparity map and aligned and fused multi-modal image.The method of the application can obtain disparity map from multi-modal images of different angles, and then obtain registered and fused multi-modal images.

The invention discloses an image auxiliary registration method and device, and the method comprises the steps: obtaining at least two groups of auxiliary point pairs which are selected manually, determining transformation information based on all auxiliary point pairs, determining a corresponding selected region in a reference image and a to-be-registered image based on the transformation information, and determining a matching point pair in the selected region, and determining a transformation matrix based on the matching point pairs so as to complete image registration. According to the method and the device, the problem of relatively low accuracy of automatic registration of a complex multi-modal image can be solved, the matching points for registration can be automatically diffused to a full-image range, the operation amount of a user is extremely small, and the consumed time is relatively short. By using the method and the device, multi-modal image registration with large difference can be processed, the accuracy is obviously improved compared with automatic registration, and the requirement of analyzing and processing the image subsequently can be met.

The present invention provides a robust recognition method for small-sample remote sensing targets with cross-scenario multi-domain fusion, belonging to the technical field of target recognition. This method fuses cross-scenario multi-domain multi-modal remote sensing images, and uses meta-learning strategies and masked autoencoders for representation learning, significantly improving the efficient learning and accurate generalization ability of the recognition model under limited labeled data and complex and changeable environments; through improved multi-modal image registration technology and improved cross-modal multi-head self-attention mechanism, combined with improved multi-level feature fusion method, the registration accuracy and feature expression ability are improved; based on a lightweight real-time target recognition model of an improved deep neural network, combined with the anchor box mechanism and non-maximum suppression algorithm, the real-time and accuracy of target detection are realized. The present invention innovates cross-scenario generalization small-sample learning and representation learning methods, innovates multi-domain heterogeneous image registration, fusion and lightweight real-time target recognition technologies, and significantly improves cross-scenario target recognition accuracy and robustness.

The application discloses a kind of based on common feature and imaging parameter joint guiding image registration method and system, the method of the present application includes extracting deep feature map to target modality and reference modality, after projection to unified structure representation space, using bidirectional interactive attention mechanism carries out geometric alignment to obtain coarse registration transformation matrix, the coarse registration image of target modality and the input image of reference modality are respectively using depth estimation network to predict dense depth map, dense depth map is used to construct three-dimensional point cloud graph using imaging parameter, and fine registration transformation matrix is calculated to the three-dimensional point cloud graph of target modality and reference modality;Coarse registration transformation matrix and fine registration transformation matrix are fused to obtain the final registration transformation matrix, and the input image of target modality is used to generate the final coarse registration image of target modality using the final registration transformation matrix.The present application aims to solve the problem that the existing multi-modal image registration method is not good in complex scene registration effect.