Corrosion source joint de-noising method for multi-source heterogeneous big data

Abstract

The present invention relates to a corrosion source joint de-noising method for multi-source heterogeneous big data. The corrosion source joint de-noising method comprises two models, one model is an HLML (Heterogeneous Linear Metric Learning) model, and the other one model is an MSCD (Multi-source Semi-supervised Collaborative Denoising) model; wherein the HLML model linearly projects multi-source heterogeneous data to a high-dimensional feature isomorphic space by learning a plurality of heterogeneous linear measurements, and complementary information among heterogeneous sources is fully embedded into the high-dimensional feature isomorphic space, thus semantic complementarity and distribution similarity among different sources can be effectively captured. In order to eliminate intra-source and inter-source noise, the MSCD model utilizes an elementary transformation constraint and a gradient energy competition strategy to restore the complementary relation between heterogeneous noise description in the feature isomorphic space learnt by the HLML model, so as to purify corrosion sources of the multi-source heterogeneous data, and be help to obtain an accurate and robust multi-source data evaluation and analysis result.

Description

technical field [0001] The invention belongs to the field of information technology, and aims at the problem of intra-source noise and inter-source noise in a massive multi-source heterogeneous corrosion data environment, and proposes a joint denoising method for corrosion sources of multi-source heterogeneous big data. Background technique [0002] In recent years, with the emergence of a large number of high-tech digital products, the multi-source heterogeneous data (Multi-source Heterogeneous Data) generated by these heterogeneous electronic devices has spread to every corner of people's real life. The so-called multi-source heterogeneous data refers to data that comes from different sources or channels, but expresses similar content, and appears in various styles such as different forms, different modalities, different perspectives, and different backgrounds. For example, Sina Weibo, Tencent WeChat, and Sohu.com report different forms of the same news; the brains of Alzh...

AI Technical Summary

Problems solved by technology

However, the drawback of the KTLQD method is that it cannot handle heterogeneous data

Images