70 results about "Multimodality image registration" patented technology

The invention discloses a surgical robot resection navigation method based on multi-modal image registration. The method comprises the following steps: acquiring a preoperative multi-modal medical image of a patient and reconstructing a three-dimensional model containing a focus and a dangerous structure; acquiring an intraoperative real-time image; extracting cross-modal invariant features from the pre-operative and intra-operative images by using a first deep learning network, and establishing an initial corresponding relation; predicting an initial non-rigid deformation field by using a second deep learning network, based on the pre-operative / intra-operative image and the initial corresponding relation and in combination with physical model constraints; tracking a surgical tool in real time, and updating a non-rigid deformation field in real time by adopting an incremental calculation strategy; applying the updated deformation field to the preoperative three-dimensional model to generate a deformed model; and mapping the tool position to the deformed model, and generating a navigation instruction. According to the method, soft tissue deformation can be accurately compensated, multi-modal image information is effectively fused, navigation information is dynamically updated in real time, and the precision and safety of robot surgery are remarkably improved.

The invention provides a multi-source image registration method based on deep learning, and relates to the technical field of image processing. The method comprises the following steps: firstly, respectively carrying out image preprocessing on infrared and visible light images; then constructing and training a feature point extraction model, and respectively extracting feature points and descriptors from the preprocessed infrared and visible light images; carrying out feature matching on the extracted feature points by adopting a feature point matching model to obtain high-precision matching point pairs; robust matching point screening and registration optimization are carried out on the matching point pairs, and a geometric transformation matrix between the infrared image and the visible light image is calculated; and finally, performing geometric transformation on each pixel point of the infrared image through the geometric transformation matrix between the infrared image and the visible light image, and converting the pixel points into a coordinate system of the visible light image to obtain a final registration result of the infrared image and the visible light image. According to the method, when the modal difference between the multi-modal images is large, a stable registration effect can still be obtained, and the robustness of multi-modal image registration is greatly improved.

The invention relates to a multi-modal image registration method based on modal alignment distillation, and belongs to the technical field of image processing. The method comprises the steps of adopting a distillation learning guide type training process, performing guide type supervision on a multi-modal registration student model by using intermediate features of a same-modal registration teacher model with excellent performance and an output deformation field, migrating modals of the intermediate features of the student model, and effectively reducing modal differences, a high-quality deformation field close to teacher model estimation is output, and the multi-modal registration performance is improved; a deformable mask training strategy is adopted to deal with the problem that a registration result is inconsistent with a label image boundary caused by a black edge effect of spatial transformation in a registration process (namely, a black edge of the registration result and an original boundary of the label image), so that the negative influence on registration is effectively reduced, and the registration performance of the model is improved. According to the invention, a multi-modal image registration task can be realized, registration challenges caused by modal differences are effectively solved, and the effect of multi-modal image space alignment is achieved.

The invention discloses a multi-modal image registration method and system based on feature decoupling and computer equipment, and the method comprises the following steps: 1, obtaining and processing a plurality of multi-modal images, and obtaining a multi-modal image registration data set with diversity; 2, constructing a multi-modal image registration neural network model based on feature decoupling; 3, designing a loss function to supervise training of a multi-modal image registration neural network model, and optimizing a registration process; and 4, processing a to-be-registered multi-modal image by using the trained multi-modal image registration neural network model to obtain an image conversion parameter matrix, and converting the to-be-registered image in the image pair according to the image conversion parameter matrix to obtain a registered image. According to the multi-modal image registration method based on feature decoupling, the performance of multi-modal image registration can be effectively improved, and particularly, good robustness is shown under the condition of large difference between different modals.

The invention discloses a multi-modal image registration method based on self-modal correlation and cross-modal estimation. The method comprises the following steps: firstly, constructing an initial frame model composed of image frequency decomposition, self-modal similarity coding, cross-modal matching estimation, optical flow generation and refinement and an image transformation module, and then performing spatial transformation on registered visible light and thermal infrared images to construct a training set; and training the model to obtain a visible light and thermal infrared registration model, and performing multi-modal image registration on visible light and thermal infrared images through the visible light and thermal infrared registration model. According to the method, the self-modal similarity of the visible light and thermal infrared images can be fully constructed, the accuracy of cross-modal feature matching estimation is improved, and high-precision registration of the visible light and thermal infrared images with remarkable view angle difference and modal difference is realized; and in addition, the visible light and thermal infrared images can still be stably registered under severe environment conditions such as night, rainy days and foggy days, and the robustness is relatively high.

The invention discloses a registration method for multi-modal retina blood flow image processing, which relates to the technical field of image processing, and comprises the following steps of: firstly, receiving retina blood flow image data from different modalities, preparing for subsequent data processing and registration, and after the retina blood flow images from the different modalities are received, carrying out registration on the retina blood flow image data from the different modalities; image preprocessing is carried out, the difference between different modes is reduced, and it is ensured that the images are analyzed and registered under the same standard. According to the method, the dynamic change trend of blood flow is predicted through the support vector machine model, the similarity measurement threshold is dynamically adjusted, local natural deformation is flexibly dealt with, and registration failure caused by blood flow fluctuation is avoided, so that the robustness and precision of multi-modal image registration are improved, more accurate space alignment is realized especially in a blood flow change salient region, and the registration accuracy is improved. And high-quality image data support is provided for retinal disease diagnosis and pathological analysis.

The invention discloses a multi-modal image registration method and system based on deformation adaptation and computer equipment, and the method comprises the steps: collecting a plurality of groups of multi-modal images, carrying out the gray standardization, and constructing a diversified registration data set; building a registration network model comprising a pyramid coding module, a deformation adaptive module, a cross-modal interaction module and a registration parameter estimation module; inputting an image pair into the modules in sequence, respectively extracting basic feature mapping, deformation feature mapping and interaction enhancement feature mapping, and finally outputting an estimation conversion parameter matrix; a training process is supervised through a preset loss function, optimal network parameters are selected, and a trained registration model is obtained; in practical application, an image pair to be registered is input into the trained model, a conversion parameter matrix is obtained, and image registration is completed. The multi-modal image registration performance can be effectively improved, and the method still has good robustness and adaptability especially under the condition that serious geometric distortion and significant modal difference exist.

Disclosed in the present invention are a multi-modal image registration method and system based on iterative optimization. The method comprises: acquiring several low-light images and processing same, in order to obtain a training set and a validation set; constructing a multi-modal image registration neural network model; training the multi-modal image registration neural network model by means of the training set, and using a loss function to calculate loss, in order to obtain a trained multi-modal image registration neural network model; using the validation set to perform iterative optimization on the trained multi-modal image registration neural network model, and determining whether an iteration termination condition is met, in order to obtain an iteratively optimized multi-modal image registration neural network model; and acquiring multi-modal images in a real scene, forming image pairs to be registered, and inputting said image pairs into the iteratively optimized multi-modal image registration neural network model for processing, in order to obtain registered and fused images. The method can improve the robustness of image pairs, which each consist of a reference image and an image to be matched, during a registration process.

The invention relates to the technical field of image processing, in particular to a multi-mode retina fundus image registration method and system. The reference image structural body passes through a first feature encoder, and reference rotation isovariant features of four stages and a total reference rotation isovariant feature graph are extracted; the floating image structural body passes through a second feature encoder, and floating rotation isovariant features of four stages and a total floating rotation isovariant feature graph are extracted; respectively inputting the reference rotation equivariant features and the floating rotation equivariant features of the four stages into a key point detection decoder for feature fusion, and outputting a reference key point prediction map and a floating key point prediction map; respectively inputting the total reference rotation equivariant feature pattern and the total floating rotation equivariant feature pattern into a key point descriptor decoder, and outputting a descriptor of each reference key point and a descriptor of each floating key point; and aligning the floating image to the reference image through a registration module to obtain a registered image. According to the invention, the multi-modal image registration precision is improved.

The invention provides a large-scale multi-modal image registration method and system based on structured enhancement driving, and belongs to the field of image registration, and the method comprises the steps: collecting images of different modals, and obtaining a to-be-registered image data set which comprises a to-be-registered image and a reference image; randomly selecting two images with different modalities from the to-be-registered image data set, and extracting two feature map pairs with different downsampling resolutions; obtaining a rough feature map and a reference anchor point for the feature map pair with one resolution; performing structure enhancement on the rough feature map by using a reference anchor point to obtain an enhanced feature map; calculating a matching probability between the enhanced feature maps, and obtaining a rough matching result; obtaining a fine matching result by combining the rough matching result with the feature map pair of the other resolution; and calculating a transformation matrix of the image by using the fine matching result, transforming the to-be-registered image, and carrying out sub-pixel level alignment on the to-be-registered image and the reference image to obtain a final registration result.

The invention provides a multi-modal image registration method and device, electronic equipment and a storage medium, and the method comprises the steps: carrying out the multi-scale side window filtering of a first to-be-registered image and a second to-be-registered image, and obtaining the multi-scale side window filtering result images of the first to-be-registered image and the second to-be-registered image, based on the side window filtering result image, obtaining a first feature point and a feature vector corresponding to the first feature point and a second feature point and a feature vector corresponding to the second feature point, and based on the feature vectors, performing feature matching on the first feature point and the second feature point twice to obtain a first homonymy feature point pair and a second homonymy feature point pair; and registering the first to-be-registered image and the second to-be-registered image based on the first homonymy feature point pair and the second homonymy feature point pair. According to the multi-modal image registration method and device, the electronic equipment and the storage medium provided by the invention, the accuracy and efficiency of multi-modal image registration are improved, and the robustness of multi-modal image registration is enhanced.

The invention relates to the technical field of image processing, in particular to a cerebrovascular image recognition and analysis system based on deep learning. The system comprises a cerebrovascular image registration module, a CT pixel communication module, a lesion area identification and segmentation module and a lesion risk assessment module, and can obtain a cerebrovascular CT angiography image, perform image enhancement and multi-modal image registration, and generate a cerebrovascular CT multi-modal registration image sequence; acquiring cerebrovascular CT pixel point clouds in the corresponding modals and performing blood vessel cluster connectivity analysis to generate corresponding cerebrovascular CT pixel cluster connectivity graphs in the respective modals; and based on a deep learning multi-branch network architecture, introducing an attention mechanism to carry out lesion region identification and segmentation, obtaining a corresponding cerebrovascular stenosis degree and a lesion thrombus area, and carrying out lesion risk number evaluation on the corresponding cerebrovascular CT image lesion region to obtain a cerebrovascular CT lesion risk number. According to the invention, accurate recognition of the cerebral vascular image lesion condition can be realized.

The invention discloses a multi-modal image registration method and system based on multi-scale feature fusion and Bayesian regression, and the method comprises the steps: a double-flow feature coding module processes a fixed image and a to-be-corrected image, and generates a multi-scale feature map; the Bayesian deformation prediction module generates a deformation field sample by using a Monte Carlo random inactivation mechanism based on decoder features, and directly outputs prediction during training. The multi-scale deformation fusion module integrates deformation fields of different levels, generates a global dense deformation field through resolution alignment, cross-scale attention enhancement and adaptive weight fusion, and balances a global trend and local details. The progressive feature refinement module fuses distortion features, decoder up-sampling features and fixed image features, optimizes feature consistency through dynamic weight distribution and a cross-dimension attention mechanism, and drives iterative optimization. And gradually improving the precision through multi-scale iteration, and finally outputting an optimal registration field. The method has the advantages that the cross-modal feature consistency is enhanced, and the robustness to noise and deformation is improved.

The invention belongs to the field of metallurgical melt jet flow detection and analysis, and discloses a time-space registration method and system for a metallurgical melt jet flow multi-modal image, so as to realize registration of the metallurgical melt jet flow multi-modal image in a complex environment. The method comprises the following steps: synchronously acquiring a metallurgical melt jet multi-modal image by adopting a binocular vision detection device; extracting a metallurgical melt jet flow contour in the multi-modal image, and respectively filtering texture details of the contour to obtain a smooth contour; analyzing the smooth contour of the multi-modal image to extract a contour feature point set, and performing rough matching on the multi-modal contour feature point set; constructing a feature point descriptor for the contour feature point set of the multi-modal image, and performing fine matching on the multi-modal contour feature point set based on the feature point descriptor; and mismatching among the contour feature point sets of the multi-modal image is eliminated, and final registration of the contour feature point sets is realized.

The invention discloses a multi-modal image registration and fusion method and system based on deep learning, and the method comprises the steps: generating a pseudo-infrared image through a cross-modal image generation network, so as to reduce the modal difference between source images, and precisely estimating a deformation field between the images through a multi-scale registration network; collaborative optimization of a registration network and a fusion network is realized by adopting a joint training framework, and semantic information of a fusion image is enhanced by applying a semantic-guided fusion framework. The technology is mainly used in the fields of automatic driving, video monitoring, medical imaging and the like, provides richer and more accurate scene description than a single sensor, and particularly has important application value in the aspects of day and night monitoring and target recognition.

The invention discloses an unsupervised multi-modal image registration method and system based on dual-path training, and the method comprises the following steps: obtaining a plurality of multi-modal image data, so as to construct a multi-modal image pair data set; constructing a multi-modal registration model; based on a registration network and a translation network, performing dual-path collaborative registration on the floating image to obtain a dual-path registration image; inputting the reference image and the dual-path registration image into a discriminator network for discrimination; a composite loss function is constructed, and a target function is established by using a minimum-maximum game method; training a multi-modal registration model based on the target function and the multi-modal image pair data set; according to the method, the dual-path collaborative registration framework is constructed through the registration network and the translation network, and the image alignment problem of different imaging modes is converted into the registration problem in the same mode, so that the problems of semantic gaps and measurement misalignment in cross-modal feature matching are solved, and the problem of cross-modal similarity measurement is avoided.

The invention belongs to the technical field of image processing, and relates to a different-source image registration method, system and device for large and small scenes and a medium, and the method comprises the steps: carrying out the n * n cutting of a large-scene image, and determining a target image block through the correlation index calculated through an NCC algorithm and the number of matching points calculated through a feature matching method, finally, the OS-SIFT algorithm is used for further completing registration of different-source images, the test result of visible light-thermal infrared data sets with different scene sizes shows that both the success rate and the registration precision of the method can meet the current practical application requirements, and the problem of multi-modal image registration under the condition that the scene sizes are greatly different can be solved.

The invention provides a multi-modal image registration method and system based on a deformation field, a storage medium and equipment, and the method comprises the steps: obtaining multi-modal images, including a 2D ultrasonic image and a 3D reconstruction image, and carrying out the coarse registration; obtaining a 2D ultrasonic image sequence of continuous frames, and obtaining rigid change parameters of the organ in a respiratory cycle; acquiring organ contour feature data points of each frame of 2D ultrasonic image, and performing elastic registration to obtain a 2D elastic deformation field of the organ; constructing a 2D-3D deformation function of the organ, and obtaining a 3D deformation field in a three-dimensional space; and updating vertex coordinates of the target organ in the 3D reconstruction image according to the 3D deformation field to obtain a mapped three-dimensional image to represent rigid and elastic changes of the organ. According to the method, by fusing the multi-modal image information and combining rigid and elastic deformation analysis, accurate registration of the ultrasonic image and the CT / MRI image is achieved, and the precision of multi-modal medical image registration is effectively improved.

The application discloses a kind of guided path planning method and system.Method includes following steps:S1.MRI image and ultrasound image are obtained, multi-modal image registration and segmentation are carried out, and MRI image and ultrasound image are fused.S2.real-time modeling of electric field distribution, based on dynamic conductivity model and finite element method, the electric field intensity distribution of cartilage and surrounding tissue after electrode applied pulse is calculated in real time, to provide input for subsequent pulse parameter optimization and path planning.S3.pulse parameter adaptive optimization, through model predictive control algorithm, real-time adjustment pulse voltage and pulse width, so that the electric field intensity of ablation area accurately covers target cartilage, while protecting blood vessel / nerve sensitive area from damage.S4.restrained path search is carried out, and guided path planning is realized.

Systems / techniques that facilitate multi-modal image registration via modality-neutral machine learning transformation are provided. In various embodiments, a system can access a first image and a second image, where the first image can depict an anatomical structure according to a first imaging modality, and where the second image can depict the anatomical structure according to a second imaging modality that is different from the first imaging modality. In various aspects, the system can generate, via execution of a machine learning model on the first image and the second image, a modality-neutral version of the first image and a modality-neutral version of the second image. In various instances, the system can register the first image with the second image, based on the modality-neutral version of the first image and the modality-neutral version of the second image.

A method for aligning multiple depth cameras in an environment based on image data can include accessing, by one or more processors, a first plurality of point cloud data points corresponding to a first pose for a subject and a second plurality of point cloud data points corresponding to a second pose for the subject. The method can include determining, by the one or more processors, a frame of reference for the image data based on at least one of the first pose or the second pose. The method can include transforming and aligning, by the one or more processors, at least one of the first plurality of point cloud data points or the second plurality of point cloud data points to the frame of reference.

The disclosure generally relates to methods and systems for image co-registration of multi-modal temporal sensing. Conventional techniques for image co-registration of multi-modality images focus on either the spatial or temporal domain and thus are not of high accuracy and do not preserve both global and local characteristics for the matching. The present disclosure solves the technical problems in the art for image co-registration of multi-modal temporal sensing using a deep-learning based multi-input-output encoder-decoder network with a Gabor Jet Model. The deep-learning based multi-input-output encoder-decoder network is utilized for the feature extraction. A distinctive Gabor-jet layer of the Gabor Jet Model is utilized for the similarity matching. The Gabor-jet layer generates a Gabor jet graph which provides sparse feature points for matching between matching images and reference images.

The present invention discloses a three-dimensional CT / PET image registration method based on mutual information and L-BFGS optimization, which belongs to the technical field of calculation, inference or counting. The registration method of the present invention is a two-stage cascade process of rigid coarse registration and B-spline elastic fine registration. It utilizes the precision advantage of mutual information measurement and noise reduction preprocessing to solve the problem of lack of precision of traditional multimodal image registration methods due to data modality and grayscale differences. It also adopts L-BFGS to improve the optimization algorithm, which greatly reduces the number of iterative calculations in traditional registration, gives full play to the speed advantage of the second-order convergence optimization algorithm, and greatly reduces the registration time. In summary, the registration method of the present invention shows high precision and speed advantages in the three-dimensional CT / PET multimodal image registration problem.

The invention relates to the technical field of industrial machine vision detection, in particular to an intelligent grading and feedback system based on image recognition. Comprising a multi-modal image acquisition unit used for synchronously acquiring a visible light image, an infrared thermal imaging image and a hyperspectral image of a target object; the dynamic self-adaptive preprocessing unit is in communication connection with the multi-modal image acquisition unit, performs combined preprocessing on the acquired multi-modal image, and comprises a dynamic noise reduction algorithm based on target edge features, a self-adaptive contrast enhancement algorithm and a multi-modal image registration algorithm; and the closed-loop feedback unit is respectively in communication connection with the multi-modal image acquisition unit and the multi-dimensional dynamic threshold grading unit. Through multi-modal image acquisition and attention-multi-scale feature fusion design, the limitation of traditional single-modal detection is effectively broken through, the recognition capability of micro bubbles and hidden cracks of the quartz crucible is remarkably improved, surface stains and internal impurities are accurately distinguished, and the problem of subjective difference of manual visual detection is solved.

The invention discloses an equipment fault source positioning method and device based on multi-modal information fusion, and the method comprises the steps: carrying out the data collection of a power distribution room inspection robot which carries an infrared thermal imager, a visible light camera and a self-adaptive lifting mechanism, carrying out the preprocessing of a collected infrared image, constructing a power distribution room equipment data set, and carrying out the positioning of a power distribution room equipment fault source. The power distribution room equipment data set comprises a plurality of image pairs composed of infrared images and visible light images which are in one-to-one correspondence; performing multi-modal image registration on each image pair based on an improved whale optimization algorithm to obtain a registered image pair; fusing the registered image pair by using a deep learning fusion network containing a coordinate attention module to obtain a fused image; and carrying out fault source positioning based on the relative temperature difference and a Monte Carlo method according to the fused image. According to the invention, accurate identification and grade determination of a fault source are realized through four stages of processes of multi-modal data acquisition, accurate image registration, high-quality image fusion and automatic fault positioning.

The present disclosure provides a method and system for processing multi-modality images. The method may include obtaining multi-modality images; registering the multi-modality images; fusing the multi-modality images; generating a reconstructed image based on a fusion result of the multi-modality images; and determining a removal range with respect to a focus based on the reconstructed image. The multi-modality images may include at least three modalities. The multi-modality images may include a focus.

The present invention provides a multimodal image registration method, apparatus, electronic device, and storage medium. The method includes: performing multi-scale edge window filtering on a first image to be registered and a second image to be registered, respectively, to obtain multi-scale edge window filtered result images of the first image to be registered and the second image to be registered; based on the edge window filtered result images, obtaining a first feature point and a feature vector corresponding to the first feature point, as well as a feature vector corresponding to the second feature point; based on the feature vectors, performing feature matching twice on the first feature point and the second feature point to obtain a first pair of feature points with the same name and a second pair of feature points with the same name; and registering the first image to be registered and the second image to be registered based on the first pair of feature points with the same name and the second pair of feature points with the same name. The present invention provides a multimodal image registration method, apparatus, electronic device, and storage medium, which improve the accuracy and efficiency of multimodal image registration and enhance the robustness of multimodal image registration.

The present invention relates to the technical field of image processing, and particularly relates to a multi-modal retinal fundus image registration method and system. The reference image structure is passed through a first feature encoder to extract the reference rotation-equivariant features in four stages and the total reference rotation-equivariant feature map; the floating image structure is passed through a second feature encoder to extract the floating rotation-equivariant features in four stages and the total floating rotation-equivariant feature map; the reference rotation-equivariant features and the floating rotation-equivariant features in the four stages are respectively input into a key point detection decoder for feature fusion, and a reference key point prediction map and a floating key point prediction map are output; the total reference rotation-equivariant feature map and the total floating rotation-equivariant feature map are respectively input into a key point descriptor decoder to output the descriptor of each reference key point and the descriptor of each floating key point; through a registration module, the floating image is aligned to the reference image to obtain the registered image. The present invention improves the registration accuracy of multi-modal images.

The present invention discloses a multimodal image registration method, system and computer equipment based on feature decoupling, comprising the following steps: 1. Acquire and process a plurality of multimodal images to obtain a multimodal image registration data set with diversity; 2. Build a multimodal image registration neural network model based on feature decoupling; 3. Design a loss function to supervise the training of the multimodal image registration neural network model and optimize the registration process; 4. Use the trained multimodal image registration neural network model to process the multimodal image to be registered to obtain an image conversion parameter matrix, and convert the image to be registered in the image pair according to the image conversion parameter matrix to obtain the registered image. The multimodal image registration method based on feature decoupling proposed by the present invention can effectively improve the performance of multimodal image registration, especially in the case of large differences between different modalities, and exhibits good robustness.