According to the building management method and system based on the user portraits, user portraits of people entering a building are recognized, user type judgment and trajectory tracking are achieved, an abnormal behavior recognition algorithm is combined, intelligent recognition alarm of abnormal behavior points is conducted, the behavior norms of the people entering the building are guaranteed; when the abnormal behavior is judged to exist, an alarm is given in time, so that the situation that all possible potential safety hazards are handled, solved and eradicated in time before a safety event occurs is ensured, the safety of articles and personnel in the building is ensured, a safe, comfortable and reliable working and living environment is provided for building owners and resident personnel, and the satisfaction degree of property service is improved.