Medication-taking behavior recognition method based on multi-modal sensor

Abstract

The application discloses a medicine taking behavior recognition method based on a multi-modal sensor, and relates to the technical field of intelligent medical treatment, and comprises the following steps: S1, synchronously collecting multi-modal time sequence data of a user when the user performs an action; S2, respectively extracting feature sequences of the multi-modal time sequence data; S3, performing time sequence alignment and feature interaction through a cross-modal attention fusion model to generate a unified multi-modal fusion feature sequence; S4, based on the multi-modal fusion feature sequence, recognizing a series of continuous sub-behavior categories; and S5, based on an evidence theory decision fusion method, fusing and checking the recognized sub-behavior categories to confirm whether a complete medicine taking behavior occurs or not. The application proposes a collaborative technical scheme based on multi-modal deep time sequence fusion and uncertainty decision, and solves the automatic recognition problem of an accurate, reliable, interpretable and privacy-protected specific complex behavior, i.e., medicine taking.

Images