Unlock instant, AI-driven research and patent intelligence for your innovation.

Self-adaptive feature fusion method for multi-mode image

A multi-modal image and feature fusion technology, applied in the field of image processing, can solve the problems of high error rate, time-consuming and labor-intensive, etc., and achieve good discriminative effect

Active Publication Date: 2019-09-10
FUZHOU UNIVERSITY
View PDF5 Cites 5 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Registration needs to find as many corresponding annotation points as possible. Providing annotation points is not only time-consuming and labor-intensive, but also has a high error rate due to the different imaging shapes of different modal directions.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Self-adaptive feature fusion method for multi-mode image
  • Self-adaptive feature fusion method for multi-mode image
  • Self-adaptive feature fusion method for multi-mode image

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0037] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0038] Please refer to figure 1 , the present invention provides a method for adaptive feature fusion of multimodal images. In this embodiment, it will be explained through mammography images and breast B-ultrasound images: the images are mammography images and breast B-ultrasound images. The target area is denoted as I 琨 and I 2 , do the matching manually, as paired images, and reset the image size to 32x32.

[0039] Step S1: Construct the encoder E, input the image, and obtain the feature spaces X and Y of the two modalities respectively;

[0040] The specific structure of the encoder is as follows:

[0041] The first layer: convolutional layer, the input channel is 1, the output channel is 16, the kernel size is (3, 3), using BN regularization, and the activation function is the ReLU function;

[0042] The second layer: the maximum pooling layer...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a self-adaptive feature fusion method for a multi-modal image, and mainly solves the problem of redundancy in fusion of high-level features extracted by a deep network. The method comprises the following specific steps: firstly, constructing an encoder, and respectively obtaining the characteristics of multiple modes; secondly, screening the features of the multiple modes byutilizing a typicality-related feature screening strategy to obtain new features of the multiple modes; thirdly, a decoder is constructed, obtained new features serve as input, and new mode images are obtained respectively; then, constructing a classifier, and updating a self-adaptive feature fusion model by using label consistent loss; and finally, performing cascading operation on the obtainednew features of the multiple modes to obtain fusion features. According to the invention, high-level features of different modalities can be learned in a self-adaptive manner, and better discrimination is achieved.

Description

technical field [0001] The present invention relates to the field of image processing, and more specifically, relates to feature fusion of multimodal images. Background technique [0002] In the field of image processing, images of different modalities have their own advantages and disadvantages. Complementary information can be provided from different aspects, and the fusion of multi-modal images is an important means to improve the performance of classification and segmentation. Multimodal image fusion is often used in the fusion processing of mammography images and B-ultrasound images, and in the fusion processing of infrared and visible light images. There are three main methods of image fusion, which are pixel level, feature level, and decision level. Pixel-level fusion is a relatively mature fusion method at present, but pixel-level fusion depends on registration. Registration needs to find as many corresponding labeling points as possible. Providing labeling points ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06K9/62G06N3/04G06N3/08
CPCG06N3/08G06N3/048G06N3/045G06F18/241G06F18/25
Inventor 余春艳杨素琼
Owner FUZHOU UNIVERSITY