Visual analysis method and system for multi-modal data based on sketch interaction

Abstract

The invention discloses a visual analysis method and system for multi-modal data based on sketch interaction. The method comprises the steps that (1) an original dataset is divided into multiple visual data structures; (2) matching between the visual data structures and visual forms is performed; (3) according to a mapping relation from the original dataset to the visual data structures to the visual forms, the original dataset is decomposed into multiple associated information sides, each information side is displayed in a view through one visual form, and a multi-viewing-angle association view is generated in combination with layout information selected by a user; and (4) sketch symbols drawn by the user who performs sketch selection operation on the multi-viewing-angle association viewaccording to a target analysis demand are recognized, then the implication of a sketch gesture composed of the sketch symbols is analyzed, and a new view is generated according to the implication of the sketch gesture. According to the visual analysis method and system, the user is assisted in performing visual analysis on the multi-modal data based on sketch interaction, the user can be helped toquickly acquire data knowledge, and the data association law can be mined.

Description

technical field [0001] The invention belongs to the field of human-computer interaction, and in particular relates to a multimodal data-oriented visual analysis method and system based on sketch interaction. Background technique [0002] We are in an era of information explosion, and new data are constantly being generated, and the speed of generation is getting faster and the scale is getting bigger and bigger. Multimodality refers to a social symbol in information exchange, which describes data information through various data forms such as images, texts, voices, and videos (Kress, Gunther (2010). Multimodality: A Social Semiotic Approach to Contemporary Communication. New York :Routledge.p.79.ISBN0415320607.). In the past few years, the amount of data has grown rapidly, and the types of data are also increasing. While bringing people rich data resources, it also brings more difficult challenges. How do people quickly understand the information in massive data? , how to ...

AI Technical Summary

Problems solved by technology

The difficulty in analyzing multimodal data lies in, first, multimodal data are often unstructured data, including directly described text data, persistent data that changes over time, image data formed by shooting, etc.; second, these There are often internal correlations between data, and static display is difficult to show the inherent complex structure, relationship and law of data

Images