A tower crane operating environment perception method based on multi-scale multi-modal feature fusion decoding

Abstract

The present application relates to a tower crane operating environment perception method based on multi-scale multi-modal feature fusion decoding, belonging to the technical field of intelligent construction, specifically comprising that the method is mainly based on sparse point cloud data collected by 4D imaging radar and video image data collected by monitoring camera, through twice feature extraction of fusion of two modal data based on deformable attention mechanism Transformer architecture, while the two modal data are down-sampled, the above feature extraction process is repeated to realize the fusion feature information of multi-modal data from different scales, and finally the multi-scale decoding of the fusion feature information of different scales is carried out, so that the final tower crane environment perception result is obtained, the present application carries out multi-modal fusion analysis of 4D sparse point cloud data and video monitoring image data in different scales, and then the recognition efficiency and accuracy of tower crane operating environment perception problem are significantly improved.

Images