Multi-source heterogeneous data fusion system based on multi-modal deep transfer learning mechanism

Abstract

The invention discloses a multi-source heterogeneous data fusion system based on a multi-modal deep transfer learning mechanism, including: a signal collector, a processor, and a result output module; the signal collector: used to acquire multi-source detection objects Heterogeneous information; the processor: including a deep feature learning module for extracting source information and target information respectively, a parameter transfer module, and a pre-trained classifier or regressor, through which the classifier detects the object category or passes The regressor detects the value of the object; the result output module is used to output the judgment result of the classifier or regressor. The invention provides a multi-source heterogeneous data fusion system based on a multi-modal deep transfer learning mechanism. The detection process adopts a classifier or a regression device, which can fuse multi-source heterogeneous data, effectively solve the problem of few samples, and automatically extract targets. High-level multimodal features for high accuracy and ease of use.

Description

technical field [0001] The invention relates to artificial intelligence and signal processing technology, in particular to a multi-source heterogeneous data fusion system based on a multi-modal deep transfer learning mechanism. Background technique [0002] At present, most target classification and detection researches involve complex targets. In order to fully detect target information, it is necessary to use multi-source multi-modal sensors for information detection, and then perform fusion to facilitate subsequent classification and regression. Since the multi-source and multi-modal sensors have different mechanisms for detecting information, the detected information has the characteristics of multi-source heterogeneity, and because the target data samples are less in most cases, it is difficult for traditional methods to achieve better results. Therefore, how to realize multi-source heterogeneous data fusion in the case of non-large sample is a hot and difficult point o...

AI Technical Summary

Problems solved by technology

[0004] (1) Most of the existing research is in the stage of theoretical research, and less commercialized;

[0005] (2) It is difficult for various existing fusion methods to achieve high-quality nonlinear transformation of the original information, so as to obtain high-level features to achieve accurate and complete representation of the target

[0006] (3) Existing fusion methods require preprocessing operations such as spatiotemporal registration of multi-source heterogeneous data, which is difficult and requires manual intervention, and cannot achieve end-to-end processing

[0007] (4) Existing fusion methods need to extract features based on empirical knowledge, which are highly subjective, poor in stability, poor in adaptability, and difficult to adapt to classification and regression requirements in complex and changeable situations

[0008] (5) Fusion methods based on traditional deep learning lack the ability to solve small samples, especially multi-modal small sample fusion problems

Traditional transfer learning does not have the ability to achieve high-quality nonlinear transformation of original information

Images