A multimodal data representation learning method and system
A data representation and learning method technology, applied in the information field, can solve problems such as missing data, large amount of data, and high computational cost
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
Method used
Image
Examples
Embodiment 1
[0064] A multi-modal data representation learning method disclosed in Embodiment 1 of the present invention is applied to a multi-modal data representation learning system. The flow chart is as follows figure 1 As shown, multimodal data representation learning methods include:
[0065] S101. Receive target multimodal data, and acquire each modality corresponding to the target multimodal data and a feature representation of each modality;
[0066] In the process of executing step S101, according to the target multi-modal data sent by the social media data collection device, each modality corresponding to the target multi-modal data is obtained, and the corresponding feature representation of each modality is obtained according to each acquired modality.
[0067] S102. Obtain a data representation and a dictionary representation of the fusion multimodal feature according to the target multimodal data, feature representation and preset graph random walk model;
[0068] S103. Acc...
Embodiment 2
[0071] Based on the above-mentioned multi-modal data representation learning method disclosed in the first embodiment of the present invention, such as figure 1 In the shown step S101, the target multimodal data is received, and the specific execution process of each modality corresponding to the target multimodal data and the feature representation of each modality is obtained, as shown in figure 2 shown, including:
[0072] S201. Receive target multimodal data, acquire each modality corresponding to the target multimodal data, and extract original features of each modality;
[0073] In the process of executing step S201, the target multimodal data is received, each modality corresponding to the target multimodal data is obtained, and the original features of each modality are extracted, wherein the original features include: visual features, text features and The characteristics of each layer of deep learning neural network.
[0074] S202. Obtain the missing features of e...
Embodiment 3
[0100] Based on a multi-modal data representation learning method disclosed in the second embodiment of the present invention, as image 3 In the shown step S301, the dictionary atom is selected according to the target multimodal data, and the corresponding feature representation of the dictionary atom is extracted according to the feature representation, and the specific execution process of the mode dictionary of each modality is obtained, as shown in Figure 5 shown, including:
[0101] S501, judging whether the target multimodal data has a label;
[0102] S502, if not, select any one of the feature representations as a single mode, perform clustering processing on the target multi-modal data corresponding to the single mode based on the preset center clustering algorithm, and select the second preset of the cluster center point target multimodal data in scope as dictionary atoms;
[0103] Optionally, the preset central clustering algorithm includes: K-Means clustering al...
PUM
Login to View More Abstract
Description
Claims
Application Information
Login to View More