Multi-modal data set labeling method and device for AI practical training and electronic equipment

Abstract

The invention discloses a multi-modal data set labeling method and device for AI practical training and electronic equipment, and belongs to the field of computer vision. According to the invention, through a scene graph generation algorithm based on a deep learning technology and graph alignment fusion, a first type of scene graph is generated by using weak supervision information of image description, the first type of scene graph is further aligned and fused with a second type of scene graph generated based on an image, and finally a candidate initial scene graph is generated as a reference for manual annotation. And wrong labeling and missing labeling are avoided. According to the multi-modal data marking method and system, intelligent marking prompts can be provided for manual marking of the multi-modal data set, so that only the candidate scene graph needs to be optimized during manual marking, the marking scale and the marking difficulty are greatly reduced, and the marking efficiency of the multi-modal data can be effectively improved.

Description

technical field [0001] The invention belongs to the field of computer vision, and in particular relates to an AI training-oriented multimodal data set labeling method, device and electronic equipment. Background technique [0002] AI training is widely used in the field of online education, such as artificial intelligence courses, specific task training, etc. AI training needs to provide corresponding tutorials and data according to the needs of users. However, with the continuous development of AI technology and the continuous improvement of task complexity, the requirements for the quality and quantity of multimodal data are becoming higher and higher. The model required for AI training relies on high-quality labeled data for training, while the traditional multi-modal data set construction method needs to rely on manual labeling work, and its labeling efficiency and quality are flawed. [0003] In addition, in the prior art, the invention patent with the application numb...

AI Technical Summary

Problems solved by technology

[0005] The purpose of the present invention is to solve the problem of low manual labeling efficiency when constructing multimodal datasets, and provide a multimodal dataset labeling method, device and electronic equipment for AI training

Images