The invention belongs to the field of intelligent traffic and
sight prediction, and discloses a
sight prediction method based on head dynamics and a first-person
view angle video. The method comprises the specific steps that an
inertial measurement unit is used for collecting head
dynamic data, a GoPro camera obtains a first person
view angle video, and
sight line coordinates obtained by a wearable eye tracker serve as training labels; after the resolution of the video is reduced, key object features are extracted by using YOLOv5s,
optical flow features are calculated by using an
optical flow algorithm, and global gray features are extracted and normalized at the same time; two types of data are respectively processed through a video frame
encoder and an IMU
encoder, and after features are spliced, a prediction result and real coordinates are visualized in a video frame through network training. According to the invention, through multi-
modal feature fusion, sight prediction in a micro-
mobile traffic scene is realized, a low-cost and high-performance technical scheme is provided for safety early warning and management of micro-
mobile traffic, and the method can be widely applied to scenes of riding
safety monitoring,
traffic flow optimization and the like.