Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Multi-view and multi-mode image segmentation method and system

A multi-modal image and multi-view technology, which is applied in the field of image processing, can solve problems such as errors and does not take into account the multi-view and multi-modal images, and achieve the effect of reduced parameters, high speed and high efficiency

Pending Publication Date: 2022-05-27
SHANDONG NORMAL UNIV
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

On the one hand, this type of method regards all pictures as one type of image, and does not take into account that pictures have the characteristics of multi-view and multi-modal; on the other hand, images of different views and modalities have different characteristics. The result obtained by using a fixed convolution kernel to convolve the image should have a large error

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Multi-view and multi-mode image segmentation method and system
  • Multi-view and multi-mode image segmentation method and system
  • Multi-view and multi-mode image segmentation method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0046] like figure 1 As shown, this embodiment provides a multi-view multi-modal image segmentation method, which specifically includes the following steps:

[0047] S101: Acquire a set of multi-modal multi-view images.

[0048] In this embodiment, the multi-modal multi-view image is a breast image as an example.

[0049] It should be noted here that the multi-view multi-modal image segmentation method provided in this embodiment is applicable to other images, such as heart images, in addition to breast images.

[0050] Breast images include two views: head-to-caudal (CC) and medial-lateral oblique (MLO) views, each of which can obtain low-energy (LE) and high-energy (HE) images in two different modalities, a total of four types of breast pictures such as Figure 3(a)-Figure 3(d) shown.

[0051] The four different types of images are represented as

[0052] in means that contains n i The ith type of breast images of the dataset. Y ij represents the true label of th...

Embodiment 2

[0086] like Figure 5 As shown, this embodiment provides a multi-view multi-modal image segmentation system, which specifically includes the following modules:

[0087] (1) An image acquisition module, which is used to acquire a set of multi-modal multi-view images.

[0088] (2) A first feature map generation module, which is used for encoding each multi-modal and multi-view image to generate a first feature map.

[0089] Specifically, in the first feature map generating module, an encoder is used to encode each multi-modal and multi-view image to generate a first feature map.

[0090] In the encoder, convolution and pooling down-sampling operations are sequentially performed on the multi-modal multi-view image to obtain the first feature map for feature extraction.

[0091] (3) A second feature map generation module, which is used for decoding the first feature map to generate a second feature map.

[0092] Specifically, in the second feature map generation module, a decod...

Embodiment 3

[0100] This embodiment provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the steps in the above-described multi-view and multi-modal image segmentation method.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention belongs to the technical field of image processing, and provides a multi-view multi-modal image segmentation method and system. The method comprises the following steps: acquiring a group of multi-modal multi-view images; each multi-modal multi-view image is coded, and a first feature map is generated; decoding the first feature map to generate a second feature map; selecting any view and modal information in a group of multi-modal multi-view images as prior information, and encoding the prior information as one-hot vectors; performing global pooling operation on the first feature map to obtain an aggregation feature, and splicing the aggregation feature with a one-hot vector; based on the splicing vector and a convolution kernel generator, generating a convolution kernel for segmenting the corresponding view and modal; and performing convolution operation on the second feature map by using the convolution kernel to obtain a segmentation result corresponding to the view and the modal image.

Description

technical field [0001] The invention belongs to the technical field of image processing, and in particular relates to a multi-view and multi-modal image segmentation method and system. Background technique [0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art. [0003] Medical image segmentation is an important research branch of medical image processing. Medical image segmentation is the process of dividing a medical image into several disjoint "connected" regions based on certain similarity features (such as brightness, color texture, etc.) Similarity, but it shows obvious differences in different regions, that is to say, there is some discontinuous characteristic of pixels on the boundary of the region. It is a complex and critical step in the field of medical image processing and analysis. Its purpose is to segment the parts with special meanings in medical images and e...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06T7/00G06T7/10G06T9/00G06N3/04G06N3/08
CPCG06T7/0012G06T7/10G06T9/002G06N3/08G06T2207/20081G06T2207/20084G06T2207/30068G06T2207/30048G06N3/045
Inventor 牛屹杜发文郑元杰隋晓丹
Owner SHANDONG NORMAL UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com