An object material classification method based on multimodal fusion deep learning

A deep learning, multi-modal technology, applied in the field of artificial intelligence and material classification, computer vision

Active Publication Date: 2017-12-12
TSINGHUA UNIV
View PDF6 Cites 23 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, how to effectively combine the visual modality with the tactile modality remains a challenging problem

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • An object material classification method based on multimodal fusion deep learning
  • An object material classification method based on multimodal fusion deep learning
  • An object material classification method based on multimodal fusion deep learning

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0072] The object material classification method based on multi-modal fusion deep learning proposed by the present invention, its flow chart is as follows figure 1 As shown, it is mainly divided into four parts: visual image modality, tactile acceleration modality, tactile sound modality and hybrid network. Include the following steps:

[0073] (1) Let the number of training samples be N 1 , the type of training sample material is M 1 , record the label of each type of material training sample as where 1≤M 1 ≤N 1 , collect all N 1 visual image I of training samples1 , Tactile acceleration A 1 and tactile sound S 1 , to build an I 1 、A 1 and S 1 The data set D 1 , I 1 The image size is 320×480;

[0074] Let the number of objects to be classified be N 2 , the material type of the object to be classified is M 2 , record the label of each class of objects to be classified as where 1≤M 2 ≤M 1 , collect all N 2 A visual image I of an object to be classified 2 ,...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to an object material classification method based on multimodal fusion deep learning and belongs to the technical field of computer vision, artificial intelligence and material classification. The object material classification method based on multimodal fusion deep learning is a multimodal fusion method of an extreme learning machine based on multi-scale local reception fields. The method achieves accurate classification of object materials through fusion of perceptual information (including visual images, tactile acceleration signals and tactile sound signals) of different modes of object materials. The method not only can perform high-representativeness feature extraction on complex materials by using multi-scale local reception fields but also can fuse information of all modes effectively to achieve information complementation between modes. The method can improve the robustness and accuracy of complex material classification, thereby having greater applicability and universality.

Description

technical field [0001] The invention relates to an object material classification method based on multimodal fusion deep learning, and belongs to the technical fields of computer vision, artificial intelligence and material classification. Background technique [0002] There are many kinds of materials in the world, which can be divided into plastics, metals, ceramics, glass, wood, textiles, stone, paper, rubber and foam. Recently, the classification of object materials has greatly attracted the attention of social environmental protection, industry and academia. For example, the classification of materials can be effectively used for the recycling of materials; the four pillars of packaging materials: paper, plastic, metal and glass, require different packaging materials under different market demands. For long-distance transportation but no special requirements for transportation quality, paper, cardboard and box cardboard are generally used; for food packaging, it should...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/62G06N99/00
CPCG06N20/00G06F18/2431
Inventor 刘华平方静刘晓楠孙富春
Owner TSINGHUA UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap