50 results about "Multimodality image registration" patented technology

The invention discloses a multi-modal image registration method and system based on deformation adaptation and computer equipment, and the method comprises the steps: collecting a plurality of groups of multi-modal images, carrying out the gray standardization, and constructing a diversified registration data set; building a registration network model comprising a pyramid coding module, a deformation adaptive module, a cross-modal interaction module and a registration parameter estimation module; inputting an image pair into the modules in sequence, respectively extracting basic feature mapping, deformation feature mapping and interaction enhancement feature mapping, and finally outputting an estimation conversion parameter matrix; a training process is supervised through a preset loss function, optimal network parameters are selected, and a trained registration model is obtained; in practical application, an image pair to be registered is input into the trained model, a conversion parameter matrix is obtained, and image registration is completed. The multi-modal image registration performance can be effectively improved, and the method still has good robustness and adaptability especially under the condition that serious geometric distortion and significant modal difference exist.

Disclosed in the present invention are a multi-modal image registration method and system based on iterative optimization. The method comprises: acquiring several low-light images and processing same, in order to obtain a training set and a validation set; constructing a multi-modal image registration neural network model; training the multi-modal image registration neural network model by means of the training set, and using a loss function to calculate loss, in order to obtain a trained multi-modal image registration neural network model; using the validation set to perform iterative optimization on the trained multi-modal image registration neural network model, and determining whether an iteration termination condition is met, in order to obtain an iteratively optimized multi-modal image registration neural network model; and acquiring multi-modal images in a real scene, forming image pairs to be registered, and inputting said image pairs into the iteratively optimized multi-modal image registration neural network model for processing, in order to obtain registered and fused images. The method can improve the robustness of image pairs, which each consist of a reference image and an image to be matched, during a registration process.

The invention discloses a multi-modal image registration method and system based on multi-scale feature fusion and Bayesian regression, and the method comprises the steps: a double-flow feature coding module processes a fixed image and a to-be-corrected image, and generates a multi-scale feature map; the Bayesian deformation prediction module generates a deformation field sample by using a Monte Carlo random inactivation mechanism based on decoder features, and directly outputs prediction during training. The multi-scale deformation fusion module integrates deformation fields of different levels, generates a global dense deformation field through resolution alignment, cross-scale attention enhancement and adaptive weight fusion, and balances a global trend and local details. The progressive feature refinement module fuses distortion features, decoder up-sampling features and fixed image features, optimizes feature consistency through dynamic weight distribution and a cross-dimension attention mechanism, and drives iterative optimization. And gradually improving the precision through multi-scale iteration, and finally outputting an optimal registration field. The method has the advantages that the cross-modal feature consistency is enhanced, and the robustness to noise and deformation is improved.

The invention discloses a multi-modal image registration and fusion method and system based on deep learning, and the method comprises the steps: generating a pseudo-infrared image through a cross-modal image generation network, so as to reduce the modal difference between source images, and precisely estimating a deformation field between the images through a multi-scale registration network; collaborative optimization of a registration network and a fusion network is realized by adopting a joint training framework, and semantic information of a fusion image is enhanced by applying a semantic-guided fusion framework. The technology is mainly used in the fields of automatic driving, video monitoring, medical imaging and the like, provides richer and more accurate scene description than a single sensor, and particularly has important application value in the aspects of day and night monitoring and target recognition.

The invention discloses an unsupervised multi-modal image registration method and system based on dual-path training, and the method comprises the following steps: obtaining a plurality of multi-modal image data, so as to construct a multi-modal image pair data set; constructing a multi-modal registration model; based on a registration network and a translation network, performing dual-path collaborative registration on the floating image to obtain a dual-path registration image; inputting the reference image and the dual-path registration image into a discriminator network for discrimination; a composite loss function is constructed, and a target function is established by using a minimum-maximum game method; training a multi-modal registration model based on the target function and the multi-modal image pair data set; according to the method, the dual-path collaborative registration framework is constructed through the registration network and the translation network, and the image alignment problem of different imaging modes is converted into the registration problem in the same mode, so that the problems of semantic gaps and measurement misalignment in cross-modal feature matching are solved, and the problem of cross-modal similarity measurement is avoided.

The invention provides a multi-modal image registration method and system based on a deformation field, a storage medium and equipment, and the method comprises the steps: obtaining multi-modal images, including a 2D ultrasonic image and a 3D reconstruction image, and carrying out the coarse registration; obtaining a 2D ultrasonic image sequence of continuous frames, and obtaining rigid change parameters of the organ in a respiratory cycle; acquiring organ contour feature data points of each frame of 2D ultrasonic image, and performing elastic registration to obtain a 2D elastic deformation field of the organ; constructing a 2D-3D deformation function of the organ, and obtaining a 3D deformation field in a three-dimensional space; and updating vertex coordinates of the target organ in the 3D reconstruction image according to the 3D deformation field to obtain a mapped three-dimensional image to represent rigid and elastic changes of the organ. According to the method, by fusing the multi-modal image information and combining rigid and elastic deformation analysis, accurate registration of the ultrasonic image and the CT / MRI image is achieved, and the precision of multi-modal medical image registration is effectively improved.

The application discloses a kind of guided path planning method and system.Method includes following steps:S1.MRI image and ultrasound image are obtained, multi-modal image registration and segmentation are carried out, and MRI image and ultrasound image are fused.S2.real-time modeling of electric field distribution, based on dynamic conductivity model and finite element method, the electric field intensity distribution of cartilage and surrounding tissue after electrode applied pulse is calculated in real time, to provide input for subsequent pulse parameter optimization and path planning.S3.pulse parameter adaptive optimization, through model predictive control algorithm, real-time adjustment pulse voltage and pulse width, so that the electric field intensity of ablation area accurately covers target cartilage, while protecting blood vessel / nerve sensitive area from damage.S4.restrained path search is carried out, and guided path planning is realized.

Systems / techniques that facilitate multi-modal image registration via modality-neutral machine learning transformation are provided. In various embodiments, a system can access a first image and a second image, where the first image can depict an anatomical structure according to a first imaging modality, and where the second image can depict the anatomical structure according to a second imaging modality that is different from the first imaging modality. In various aspects, the system can generate, via execution of a machine learning model on the first image and the second image, a modality-neutral version of the first image and a modality-neutral version of the second image. In various instances, the system can register the first image with the second image, based on the modality-neutral version of the first image and the modality-neutral version of the second image.

A method for aligning multiple depth cameras in an environment based on image data can include accessing, by one or more processors, a first plurality of point cloud data points corresponding to a first pose for a subject and a second plurality of point cloud data points corresponding to a second pose for the subject. The method can include determining, by the one or more processors, a frame of reference for the image data based on at least one of the first pose or the second pose. The method can include transforming and aligning, by the one or more processors, at least one of the first plurality of point cloud data points or the second plurality of point cloud data points to the frame of reference.

The disclosure generally relates to methods and systems for image co-registration of multi-modal temporal sensing. Conventional techniques for image co-registration of multi-modality images focus on either the spatial or temporal domain and thus are not of high accuracy and do not preserve both global and local characteristics for the matching. The present disclosure solves the technical problems in the art for image co-registration of multi-modal temporal sensing using a deep-learning based multi-input-output encoder-decoder network with a Gabor Jet Model. The deep-learning based multi-input-output encoder-decoder network is utilized for the feature extraction. A distinctive Gabor-jet layer of the Gabor Jet Model is utilized for the similarity matching. The Gabor-jet layer generates a Gabor jet graph which provides sparse feature points for matching between matching images and reference images.

The present invention discloses a three-dimensional CT / PET image registration method based on mutual information and L-BFGS optimization, which belongs to the technical field of calculation, inference or counting. The registration method of the present invention is a two-stage cascade process of rigid coarse registration and B-spline elastic fine registration. It utilizes the precision advantage of mutual information measurement and noise reduction preprocessing to solve the problem of lack of precision of traditional multimodal image registration methods due to data modality and grayscale differences. It also adopts L-BFGS to improve the optimization algorithm, which greatly reduces the number of iterative calculations in traditional registration, gives full play to the speed advantage of the second-order convergence optimization algorithm, and greatly reduces the registration time. In summary, the registration method of the present invention shows high precision and speed advantages in the three-dimensional CT / PET multimodal image registration problem.

The invention relates to the technical field of industrial machine vision detection, in particular to an intelligent grading and feedback system based on image recognition. Comprising a multi-modal image acquisition unit used for synchronously acquiring a visible light image, an infrared thermal imaging image and a hyperspectral image of a target object; the dynamic self-adaptive preprocessing unit is in communication connection with the multi-modal image acquisition unit, performs combined preprocessing on the acquired multi-modal image, and comprises a dynamic noise reduction algorithm based on target edge features, a self-adaptive contrast enhancement algorithm and a multi-modal image registration algorithm; and the closed-loop feedback unit is respectively in communication connection with the multi-modal image acquisition unit and the multi-dimensional dynamic threshold grading unit. Through multi-modal image acquisition and attention-multi-scale feature fusion design, the limitation of traditional single-modal detection is effectively broken through, the recognition capability of micro bubbles and hidden cracks of the quartz crucible is remarkably improved, surface stains and internal impurities are accurately distinguished, and the problem of subjective difference of manual visual detection is solved.

The invention discloses an equipment fault source positioning method and device based on multi-modal information fusion, and the method comprises the steps: carrying out the data collection of a power distribution room inspection robot which carries an infrared thermal imager, a visible light camera and a self-adaptive lifting mechanism, carrying out the preprocessing of a collected infrared image, constructing a power distribution room equipment data set, and carrying out the positioning of a power distribution room equipment fault source. The power distribution room equipment data set comprises a plurality of image pairs composed of infrared images and visible light images which are in one-to-one correspondence; performing multi-modal image registration on each image pair based on an improved whale optimization algorithm to obtain a registered image pair; fusing the registered image pair by using a deep learning fusion network containing a coordinate attention module to obtain a fused image; and carrying out fault source positioning based on the relative temperature difference and a Monte Carlo method according to the fused image. According to the invention, accurate identification and grade determination of a fault source are realized through four stages of processes of multi-modal data acquisition, accurate image registration, high-quality image fusion and automatic fault positioning.

The present disclosure provides a method and system for processing multi-modality images. The method may include obtaining multi-modality images; registering the multi-modality images; fusing the multi-modality images; generating a reconstructed image based on a fusion result of the multi-modality images; and determining a removal range with respect to a focus based on the reconstructed image. The multi-modality images may include at least three modalities. The multi-modality images may include a focus.

The present invention provides a multimodal image registration method, apparatus, electronic device, and storage medium. The method includes: performing multi-scale edge window filtering on a first image to be registered and a second image to be registered, respectively, to obtain multi-scale edge window filtered result images of the first image to be registered and the second image to be registered; based on the edge window filtered result images, obtaining a first feature point and a feature vector corresponding to the first feature point, as well as a feature vector corresponding to the second feature point; based on the feature vectors, performing feature matching twice on the first feature point and the second feature point to obtain a first pair of feature points with the same name and a second pair of feature points with the same name; and registering the first image to be registered and the second image to be registered based on the first pair of feature points with the same name and the second pair of feature points with the same name. The present invention provides a multimodal image registration method, apparatus, electronic device, and storage medium, which improve the accuracy and efficiency of multimodal image registration and enhance the robustness of multimodal image registration.

The invention discloses an industrial product visual inspection system and method based on image processing, and relates to the technical field of industrial product visual inspection. According to the invention, through improving the SIFT-PSO registration algorithm and combining the hyperspectral, thermal imaging and 3D point cloud data acquired by multiple sensors cooperatively, multi-modal image registration with pixel-level precision is realized; an adaptive illumination compensation and noise suppression technology is adopted, so that the image quality is improved; by improving an LBP-TOP algorithm and combining geometric curvature features and heat distribution features, multi-dimensional information feature expression is constructed, and calculation load is reduced through a feature compression technology; a ResNet model is constructed, a network weight and a transfer learning mechanism are dynamically optimized through a genetic algorithm, and the accuracy and generalization ability of defect detection are improved; the method is suitable for product defect detection in a complex industrial environment, has the characteristics of high robustness, high precision and efficient processing, and realizes double breakthrough of detection precision and real-time performance.

The invention discloses a multi-modal image registration method and device based on a generative model and self-supervised learning, and a medium, and the method comprises the steps: carrying out the modal conversion of a source image through employing a Pix2Pix-Transform generative model, extracting global and local features from the source image and a target image through a DINO model in an image registration process, carrying out the region division of the image based on the features, and carrying out the recognition of a target image through a DINO model; for each region, weighted distribution is carried out according to the feature intensity and importance of the region, so that the key region can obtain higher weight and precision in the registration process, and the weight of the background region can be properly reduced. A region self-adaptive registration technology is adopted, high-precision non-linear registration is carried out on a key region, low-precision linear registration is carried out on a background region, and the overall registration efficiency is ensured. According to the self-adaptive weighted registration method, the overall precision and efficiency of image registration are effectively improved, and particularly, when a multi-modal image is processed, the registration difficulty caused by modal difference can be remarkably reduced.

The invention discloses an automatic multi-modal image registration method and system. The method comprises the following steps: initializing and preprocessing a registration image; extracting feature points of the registered image, and calculating descriptors; matching descriptors of the registered image pairs; calculating an affine matrix; and performing affine matrix processing on the to-be-registered image to obtain a registered image. According to the method, the registration problem of an endogenous imaging technology, a light field imaging technology, a tissue slicing technology and the like which cannot be solved in the prior art can be solved at the same time, fine adjustment is not needed for different modes, zero sample learning can be achieved, and the method can be suitable for imaging conditions of different scales from mesoscopic imaging to microscopic imaging and the like.

The application discloses a kind of multi-modal image registration methods based on disparity estimation.The specific steps are as follows:(1) build array imaging system;(2) construct data set;(3) input image into neural network, adopt double branch strategy, extract common features and unique features;(4) use channel attention enhancement module for feature enhancement, construct matching cost based on disparity, and get disparity map through disparity regression;(5) use the obtained disparity map, align images through homography distortion, realize the registration and fusion of multi-modal images;(6) construct loss function, including mean absolute loss function and least square generative adversarial loss function;(7) input the image of test set into neural network, get style generation graph, predicted disparity map and aligned and fused multi-modal image.The method of the application can obtain disparity map from multi-modal images of different angles, and then obtain registered and fused multi-modal images.

The invention discloses an image auxiliary registration method and device, and the method comprises the steps: obtaining at least two groups of auxiliary point pairs which are selected manually, determining transformation information based on all auxiliary point pairs, determining a corresponding selected region in a reference image and a to-be-registered image based on the transformation information, and determining a matching point pair in the selected region, and determining a transformation matrix based on the matching point pairs so as to complete image registration. According to the method and the device, the problem of relatively low accuracy of automatic registration of a complex multi-modal image can be solved, the matching points for registration can be automatically diffused to a full-image range, the operation amount of a user is extremely small, and the consumed time is relatively short. By using the method and the device, multi-modal image registration with large difference can be processed, the accuracy is obviously improved compared with automatic registration, and the requirement of analyzing and processing the image subsequently can be met.

The application discloses a kind of based on common feature and imaging parameter joint guiding image registration method and system, the method of the present application includes extracting deep feature map to target modality and reference modality, after projection to unified structure representation space, using bidirectional interactive attention mechanism carries out geometric alignment to obtain coarse registration transformation matrix, the coarse registration image of target modality and the input image of reference modality are respectively using depth estimation network to predict dense depth map, dense depth map is used to construct three-dimensional point cloud graph using imaging parameter, and fine registration transformation matrix is calculated to the three-dimensional point cloud graph of target modality and reference modality;Coarse registration transformation matrix and fine registration transformation matrix are fused to obtain the final registration transformation matrix, and the input image of target modality is used to generate the final coarse registration image of target modality using the final registration transformation matrix.The present application aims to solve the problem that the existing multi-modal image registration method is not good in complex scene registration effect.

The invention discloses a guiding path planning method and system. The method comprises the following steps: S1, acquiring an MRI image and an ultrasonic image, performing multi-modal image registration and segmentation, and fusing the MRI image and the ultrasonic image; s2, electric field distribution real-time modeling: calculating electric field intensity distribution of cartilage and surrounding tissues in real time based on a dynamic conductivity model and a finite element method after pulse is applied to the electrode, and providing input for subsequent pulse parameter optimization and path planning. S3, adaptively optimizing pulse parameters, and adjusting pulse voltage and pulse width in real time through a model predictive control algorithm, so that the electric field intensity of the ablation area accurately covers the target cartilage, and meanwhile, protecting a blood vessel / nerve sensitive area from being damaged. And S4, carrying out constraint path search to realize guiding path planning.

The invention discloses a multi-modal image unsupervised registration method based on alternative optimization training. The method comprises the following steps: firstly, simulating to generate a same-mode image pair with a random deformation truth value; during training, the mode conversion module converts a source image mode into a target image mode, and the homography estimation registration module carries out deformation prediction on a to-be-registered image pair. The homography estimation registration module uses the source image pair and the target image pair which are subjected to mode conversion, and self-supervised training is carried out by utilizing a simulation deformation truth value. The mode conversion module enables the source image to be approximately aligned with the target image through the homography estimation registration module, and supervises training through perception loss. And carrying out verification enhancement training on the two modules in combination with additional simulation deformation, and optimizing the training effect. In the training process of the homography estimation registration module and the modal conversion module, gradient truncation and weight updating are independent and alternately optimized. According to the invention, unsupervised registration of multi-modal images can be realized, dependence on manual annotation data is reduced, and the precision of multi-modal image registration is improved.

The present invention relates to the field of image registration technology, and discloses a multimodal image registration method based on attention mechanism and contrastive learning, comprising inputting an initial modal image A and an initial modal image B into a registration network, generating a deformation field, and optimizing the generation of the deformation field; inputting the initial modal image A into a translation network and obtaining a translated modal image B. t ; Translate modal image B t The image deformed by the deformation field is subjected to pixel block level contrast loss with the initial modal image A; the translated modal image B t The image deformed by the deformation field is subjected to a pixel-by-pixel L1 loss compared to the initial modal image B, prompting the registration network to generate a smooth deformation field. The translation network and registration network are trained and optimized to minimize the sum of the pixel-level contrast losses. This method achieves high registration accuracy and efficiency.

The present application relates to a kind of multi-modal medical image flexible registration algorithm based on multi-frame intelligent agent, based on reinforcement learning design a new end-to-end multi-modal image registration method, by soft actor-critic algorithm SAC driven training, can imitate the step-by-step registration process of human expert, improve the accuracy of high-dimensional registration action.In view of the extremely complex multi-modal environment, there are severe challenges in pixel-level control in three-dimensional space, the present application combines reinforcement learning with planner network, encourages artificial agent to learn more accurate registration action explicitly from the state frame that has been generated, overcomes the challenge from multi-modal and high-dimensional continuous action space with the advantage of space-time dimension, avoids introducing deep neural network with huge amount of parameters, and has strong robustness and generalization ability, can drive model to twist and move image in the correct direction.

The invention relates to a medical image processing technology, and discloses a multi-modal image registration method and system based on representation decoupling. Respectively coding a moving image and a fixed image through a dual-branch multi-scale coding network with independent parameters, and extracting modal-independent features through a multilayer modal-independent contrast loss constraint in training; the features of different scales are input into cascaded multi-scale feature registration modules, registration is carried out step by step according to the scales from small to large, and each module receives the deformation field and the deformation feature of the previous scale, outputs the deformation field and the deformation feature of the current scale and transmits the deformation field and the deformation feature backwards; finally, the deformation field acts on the moving image to obtain a registration result. According to the multi-modal medical image registration method and device, modal irrelevant feature extraction is achieved through double-branch independent coding and multi-layer comparison loss constraint cooperation, error accumulation is relieved through deformation field and deformation feature joint transmission, and the precision and stability of multi-modal medical image registration are remarkably improved.

The invention relates to a visible light image and infrared image registration method, a visible light image and infrared image registration device and an embedded program product, belongs to the technical field of image processing, and realizes the registration of a visible light image and an infrared image through the processing steps of image acquisition, zooming, edge image extraction and translation traversal. In the registration process, a processing strategy of scaling a visible light image according to a scaling factor is provided, normalized thresholding processing is carried out on an edge amplitude image, similarity calculation is more facilitated, image similarity is calculated by adopting a normalized cross-correlation value during traversal, an optimal pixel translation position can be found more accurately, and the registration accuracy is improved. The multi-modal image registration precision can be improved; the registration method has the characteristics of high registration precision and fast processing speed, is convenient to transplant in an embedded platform, and has good application value for multi-modal image fusion.

The invention relates to an infrared-visible light image fusion method based on a deformation field, a storage medium and electronic equipment. The infrared-visible light image fusion method comprises the following steps: acquiring an infrared image and a visible light image of a high-voltage switch cabinet; processing the visible light image by adopting a cross-modal perception style migration network in the multi-modal image registration model to generate a pseudo-infrared image; inputting the pseudo infrared image and the infrared image into a multistage refined registration network of the model to generate a distortion displacement vector deformation field, and performing registration reconstruction on the infrared image to obtain a registered infrared image; and respectively inputting the registered infrared image and visible light image into a feature extraction network, extracting features, fusing the features by adopting a multi-modal image fusion network, and obtaining and outputting a fused image. According to the invention, the deformation field is generated through the pseudo-infrared image to realize the registration and fusion of the infrared and visible light images, the method is constructed and applied to the infrared and visible light image fusion of the switch cabinet, the image registration and fusion are automatically realized, and the manual workload is reduced.