RGB-D image object detection and semantic segmentation method based on deep convolution network

A RGB image and convolutional network technology, applied in the field of deep learning and machine vision, can solve problems such as slowness, complexity, and slow use speed

Active Publication Date: 2017-05-24
深圳市小枫科技有限公司
View PDF4 Cites 161 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In the process of semantic segmentation, the superpixel-based depth features (globe-centered pose) and geometric features (size, shape) use support vector machines to predict the category labels of superpixels, but this method is very slow and uses multi

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • RGB-D image object detection and semantic segmentation method based on deep convolution network
  • RGB-D image object detection and semantic segmentation method based on deep convolution network
  • RGB-D image object detection and semantic segmentation method based on deep convolution network

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0024] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments.

[0025] The present invention will be described from the following aspects: fusion of RGB images and depth images, modified NMS, model training and experimental results.

[0026]The object detection and semantic segmentation method of RGB-D images based on deep convolutional network includes the following steps:

[0027] First, according to the above method, the RGB image and the depth image are fused into an HHG image;

[0028] Second, train the object detection system model;

[0029] There are three training methods of Faster-RCNN: one is Alternating Training, the other is Approximate Joint Training, and the third is Non-approximate Joint Training. This method uses an alternate training scheme. The idea of ​​the alternate training scheme is to make the regional scheme network and Fast-RCNN share the convolutional layer parameters, and fine-...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses an RGB-D image object detection and semantic segmentation method based on a deep convolution network, which belongs to the field of depth learning and machine vision. According to the method provided by the technical scheme of the invention, Faster-RCNN is used to replace the original slow RCNN; Faster-RCNN uses GPU, which is fast in the aspect of feature extracting, and at the same time generates a regional scheme in the network; the whole training process is training from end to end; FCN is used to carry out RGB-D image semantic segmentation; FCN uses a GPU and the deep convolution network to rapidly extract the deep features of an image; deconvolution is used to fuse deep features and shallow features of the image convolution; and the local semantic information of the image is integrated into the global semantic information.

Description

technical field [0001] The invention belongs to the field of deep learning and machine vision, and in particular relates to an object detection and semantic segmentation method including RGB-D images, which has a very wide range of applications in real scenes, such as detecting and tracking pedestrians in surveillance videos, UAV navigation, automatic driving, etc. Background technique [0002] Object detection and semantic segmentation are two important research fields of computer vision. Object detection is mainly used to detect the position and category of objects in the image. There are two main tasks in object detection. One is to find out the region scheme of the object (Region Proposals), the regional scheme is a pre-selected frame, which represents the approximate position of an object in the image; the second is to classify the objects in the pre-selected frame. The problem solved by semantic segmentation is to assign the correct label to each pixel of the image. S...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06N3/04G06N3/08G06K9/00
CPCG06N3/084G06V20/41G06N3/045
Inventor 刘波邓广晖
Owner 深圳市小枫科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap