Unlock instant, AI-driven research and patent intelligence for your innovation.

Multi-modal data matching method and device, equipment and storage medium

A multi-modal, data technology, applied in the field of data processing, can solve the problems of consuming manpower and material resources, unable to make full use of single-modal data, etc., to achieve the effect of improving utilization, reducing dimensions, and removing redundant dimensions

Inactive Publication Date: 2021-09-17
BEIJING BIG DATA ADVANCED TECH RES INST
View PDF3 Cites 6 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] The problem in the existing technology is that manually labeling a large amount of cross-modal data consumes manpower and material resources, and cannot make full use of single-modal data

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Multi-modal data matching method and device, equipment and storage medium
  • Multi-modal data matching method and device, equipment and storage medium
  • Multi-modal data matching method and device, equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0067] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

[0068] refer to figure 1 , figure 1 It is a flowchart of a multimodal data matching method proposed by an embodiment of the present application. Such as figure 1 As shown, the method includes the following steps:

[0069] S11: Classify the acquired multiple unimodal data to obtain multiple data sets, wherein the same data set includes multiple unimodal data with the same modality.

[0070] In this embodiment, unimodal data refers to picture data, text data, aud...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The embodiment of the invention relates to the technical field of data processing, in particular to a multi-modal data matching method and device, equipment and a storage medium, and aims to realize quick matching of low-quality cross-modal data. The method comprises the steps of classifying multiple kinds of obtained single-mode data, wherein multiple data sets are obtained, and the same data set comprises multiple pieces of single-mode data with the same mode; performing feature extraction on the single-mode data of each data set in the plurality of data sets to obtain high-level features of the plurality of single-mode data in the data set, and putting the plurality of high-level features in the data set into a feature data set corresponding to the data set; performing reference point-based structural characterization on the high-level features in each feature data set to obtain reference characterization corresponding to the feature data set; and inputting the reference representations corresponding to the plurality of feature data sets into a pre-trained common spatial network to obtain multiple groups of cross-modal matching data.

Description

technical field [0001] The embodiments of the present application relate to the technical field of data processing, and in particular, to a multimodal data matching method, device, device, and storage medium. Background technique [0002] With the large-scale rise of emerging technologies such as the Internet, human society is generating a wide variety of data with huge volumes every day, and data such as text and images are the most common data in Internet data. These data have a wide range of potential correlations and have high utilization value. Discovering the potential correlations between these data is an important research direction in the field of artificial intelligence and information technology. However, text and images are data of different modalities and have very Structured, heterogeneous, and semantically sparse, it is difficult to use these cross-modal data at the same time. In the prior art, in order to make good use of cross-modal data, it is necessary to...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/62G06F16/35G06F40/30G06N3/04G06N3/08
CPCG06F16/35G06F40/30G06N3/08G06N3/045G06F18/22G06F18/23213G06F18/241
Inventor 刘艺郑奇斌刁兴春李蒙蒙秦伟
Owner BEIJING BIG DATA ADVANCED TECH RES INST