Pre-training method and device based on Transform structure

Abstract

The invention discloses a pre-training method and device based on a Transform structure. The method comprises the following steps: firstly, acquiring an image and a video of a target object; for any segmented region in the image and the video, taking the feature symbol of the segmented region as the label of the segmented region; mask processing is carried out on the partial segmentation area of the image and the partial segmentation area of the video to obtain a first training sample and a second training sample; performing supervised prediction learning on the feature symbols of the mask region in the first training sample based on a Transform structure to obtain an initial model; initializing the pre-training model based on the initial model to obtain an initial pre-training model; and finally, performing supervised joint training on the first training sample and the second training sample by using the initial pre-training model to obtain a final pre-training model. Therefore, the model learns the spatial features and the time features of the video data at the same time, and a good pre-training model is provided for downstream tasks.

Description

technical field [0001] The invention relates to the technical field of model training, in particular to a pre-training method and device based on a Transformer structure. Background technique [0002] At present, most tasks based on ultrasound images or videos (such as classification, segmentation or detection tasks) rely on a large amount of data, and the labeling of ultrasound data is cumbersome, and different tasks need to be labeled with different content. Pre-training is a method of putting together a large amount of training data collected at low cost, learning the commonalities through model training, and then "transplanting" the commonalities to specific tasks. Self-supervised learning is a learning style that utilizes the input data itself as supervision and benefits almost all types of downstream tasks. [0003] Transformer is the main network structure in the field of natural language processing and has achieved great success in different Neuro-Linguistic Program...

AI Technical Summary

Problems solved by technology

First, this dynamic nature brings additional difficulties to representation learning; second, letting the model learn video representations from scratch is computationally expensive and requires ve...

Images