The invention discloses a
robot visual touch sensing method and
system based on
transformer and comparative learning, and belongs to the technical field of robots. According to the method, an enhanced Swin Transform model is introduced to serve as a sensing model, the enhanced model takes Swin Transform as a
backbone network, a coordinate attention mechanism is introduced into the
backbone network,
global information is extracted through a sliding window attention mechanism of the
backbone network, local
feature extraction is enhanced through the coordinate attention mechanism, and the
global information is extracted through the sliding window attention mechanism. The global and local features in the visual and tactile data can be effectively extracted and fused. According to the enhanced Swin
Transformer model, in-
modal comparative learning and inter-
modal comparative learning are specifically performed through multi-
modal comparative learning training in a pre-training stage of an earlier stage, and richer and more accurate discrimination features are generated, so that the intelligent sensing and operation capability of the
robot to the environment is effectively improved. According to the method, the features can be effectively extracted in multi-
modal data fusion, and deeper multi-
modal data fusion is realized by utilizing the correlation and synergistic effect of the features among different modals.