Pixel-level image segmentation system and method based on multi-modal spectral image

Abstract

The invention discloses a pixel-level image segmentation system based on a multi-modal spectral image. A plurality of visible light cameras and a thermal imaging sensor form a camera array, and the visible light cameras are built into a multi-modal camera array group to form a multi-modal information source; a graphic acquisition card is connected with the multi-modal information source and an acquisition workstation, and stores the acquired image data in the acquisition workstation; and a server carries out registration and post-processing on the graphic data in the acquisition workstation. The invention further discloses a pixel-level image segmentation method based on the multi-modal spectral image. A single-modal image segmentation algorithm is expanded into multi-modal input, and themulti-modal feature images are fused in a network intermediate layer, so that the precision of the Mask-RCNN image segmentation algorithm is improved. And meanwhile, a set of multi-modal spectral image acquisition system is constructed, can be used for constructing a multi-modal data set, is applied to related machine vision tasks such as target detection, image segmentation and semantic segmentation, and has a practical application prospect.

Description

technical field [0001] The present invention relates to the field of image segmentation in computer vision, and more specifically, to a pixel-level image segmentation system and method based on multimodal spectral images. Background technique [0002] Image segmentation is one of the important research topics in the field of computer vision. The so-called image segmentation refers to the process of dividing an image into several disjoint regions according to features such as grayscale, color, and texture. These features exhibit similarities within regions and differences outside regions. In recent years, with the development of deep learning, image segmentation technology has also made great progress. Technologies related to this technology, such as scene object segmentation, human body front and background segmentation, face and human body analysis, and 3D reconstruction, have been used in unmanned, enhanced Reality, security monitoring and other industries have been wide...

AI Technical Summary

Problems solved by technology

However, images in the infrared band are still difficult to provide a reliable source of information about people in hot temperatures, or camouflaged objects that are not heat sources

Images