Multi-modal data knowledge information extraction method based on deep-width joint neural network

Abstract

The invention discloses a multi-modal data knowledge information extraction method based on a deep-width joint neural network, and the method comprises the steps: 1) collecting multi-modal data generated by the production of an intelligent manufacturing factory, carrying out the data cleaning preprocessing, and storing the data in a Hadoop distributed file system; 2) performing subdivision table establishment on original data log records stored in the HDFS according to modal properties, processing multi-modal data into single-modal data features, including single-modal data feature tables such as voice, texts and images, and storing the single-modal data features in an HIVE database; and 3) performing feature extraction on the multi-modal data feature table by using the deep-width joint network to obtain corresponding high-level abstract feature knowledge, thereby realizing extraction of multi-modal data knowledge information by the deep-width joint network.

Description

technical field [0001] The present invention relates to technical fields such as deep learning, width learning and multimodal data feature extraction, and in particular refers to a multimodal data knowledge information extraction method based on a deep-width joint neural network. Background technique [0002] With the rapid development of Internet technology and the continuous in-depth transformation of the digital industrial chain, the era of big data has emerged as the times require. Cloud computing, artificial intelligence and other technologies are growing rapidly, and a digital ecological society with big data as the core has been established. Multimodal data from all aspects of the real world are difficult to analyze effectively with the current technical level. The difficulty of processing massive data is also greatly increased. In order to solve these problems, a new method of data analysis and processing is urgently needed. Using the latest AI technology fusion t...

AI Technical Summary

Problems solved by technology

[0003] At present, knowledge feature extraction methods for multi-modal data mainly use machine learning or deep learning methods to achieve the ability to process and understand multi-source modal information, but existing methods are often unable to adaptively realize multiple modal data features Effective Fusion Between (Li Huifang, Zhao Leilei, Hu Guangzheng. An Intelligent Fault Diagnosis Method Based on Multimodal Fusion Deep Learning, 2018.) (Zhong Chongliang. A Multimodal Feature Fusion Based on Convolutional Neural Network Methods and devices, 2019.)

Multimodal learning has gone through multiple development stages and has now fully entered the use of deep learning as the main means of knowledge extraction. However, traditional deep learning methods are time-consuming and laborious, especially when applied in the field of multimodal data. Powerful computing resources are often required. Difficult to meet the needs of industry and academia

Images