Multi-modal data association method and device
A data association and multi-modal technology, applied in the field of data processing, can solve the problems of inability to determine the data association of different modal data, poor universality of data association methods, etc.
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
Method used
Image
Examples
Embodiment 1
[0026] According to an embodiment of the present invention, an embodiment of a multimodal data association method is provided. It should be noted that the steps shown in the flow chart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, Also, although a logical order is shown in the flowcharts, in some cases the steps shown or described may be performed in an order different from that shown or described herein.
[0027] figure 1 is a multimodal data association method according to an embodiment of the present invention, such as figure 1 As shown, the method includes the following steps:
[0028] Step S102, acquiring data to be processed, wherein the types of data to be processed include: video data, picture data and text data;
[0029] Step S104, constructing the semantic graph of the data to be processed;
[0030] Step S106, using the graph convolutional network to calculate the representation vector of the s...
Embodiment 2
[0087] The embodiment of the present invention also provides a multimodal data association device, which is used to execute the multimodal data association method provided in the above content of the embodiment of the present invention, the following is provided by the embodiment of the present invention A detailed introduction to the multimodal data association device.
[0088] Such as image 3 as shown, image 3 It is a schematic diagram of the above-mentioned multimodal data association device, which includes: an acquisition unit 10 , a construction unit 20 , a calculation unit 30 and a determination unit 40 .
[0089] The acquiring unit 10 is configured to acquire data to be processed, wherein the types of the data to be processed include: video data, picture data and text data;
[0090] The construction unit 20 is configured to construct the semantic graph of the data to be processed;
[0091] The calculation unit 30 is configured to calculate the representation vector...
Embodiment 3
[0101] A terminal provided by an embodiment of the present invention includes a memory, a processor, and a computer program stored on the memory and operable on the processor. When the processor executes the computer program, the multimodal data association method in the first embodiment above is implemented. .
[0102] see Figure 4 , the embodiment of the present invention also provides a terminal 100, including: a processor 60, a memory 61, a bus 62 and a communication interface 63, the processor 60, the communication interface 63 and the memory 61 are connected through the bus 62; the processor 60 is used for Executable modules, such as computer programs, stored in the memory 61 are executed.
[0103] Wherein, the memory 61 may include a high-speed random access memory (RAM, Random Access Memory), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory. The communication connection between the system network element and at least...
PUM
Login to View More Abstract
Description
Claims
Application Information
Login to View More