Multi-person multi-mode perception data automatic marking and mutual learning method

Abstract

The invention discloses an automatic marking and mutual learning method for multi-person multi-mode perception data. The method comprises the following steps: step 1, carrying out clock alignment, denoising and other preprocessing on the data; 2, fusing the results of the modal models to segment the data stream; 3, when the scene contains multi-person information, data entity alignment is carriedout by utilizing the static state information and the motion state information of the persons at the same time; 4, automatically marking the data segments by utilizing a designed multi-modal data prediction fusion mechanism; and 5, improving the model understanding capability of each mode by utilizing the automatically marked data. In order to ensure the capability of the updated model, the quality of the automatically marked data is evaluated, i.e., only the data with high quality is selected to update the model. According to the method, automatic segmentation and labeling of the multi-modaldata can be realized by utilizing intrinsic correlation of the multi-modal data, labeled data segments are obtained, the model capability is further improved, and promotion of data understanding related research is facilitated.

Description

Technical field[0001]The invention relates to the field of cross-domain perception, in particular to a method for automatically marking and mutual learning of multi-person multi-modal perception data.Background technique[0002]The wide application of sensing devices (such as smart phones, wearable devices, cameras, and wireless access points, etc.) and the rapid growth of sensing data make intelligent perception a hot research topic. In recent years, there has been quite a lot of research on using machine learning technology to understand various modal data (such as video, audio, motion sensor data, and wireless data). But most of these studies are for single modal data. Monomodal data can only obtain information about the current part of the scene, which may result in low perception accuracy. At the same time, most of the existing models require labeled training data to train. The labeled data itself is a time-consuming and laborious task, and the trained model can only recognize th...

AI Technical Summary

Problems solved by technology

[0004] The purpose of the present invention is to provide a method for automatic marking and mutual learning of multi-modal sensing data, which can automatically segment, align, and mark the sensing data stream, and then learn from each other to achieve higher-precision sensing and solve the difficulty of manually marking data. , the problem of limited awareness of single modality data

Images