The invention discloses a malicious 
software homology analysis method based on a behavior tree, provides a new definition for behaviors of malicious 
software, is not limited to analysis of API continuous short sequences, describes behavior characteristics of the malicious 
software from each behavior and a relationship between the behaviors, and is richer in behavior 
semantics. The method comprisesthe following steps: firstly, calling a sequence from an API generated during malicious 
software execution, and constructing the behavior tree for reflecting a malicious 
software behavior model by adopting an Inductive Miner 
algorithm; secondly, extracting behavior characteristics from each behavior tree, generating family weighted behavior characteristics, converting the 
behavior trees into similarity vectors based on a similarity 
algorithm, and finally, training a family classification model by applying a naive Bayes classification 
algorithm. According to the method, the problems of lack ofa control structure and existence of 
noise in the API sequence in previous malicious software 
homology analysis based on the API sequence can be solved, and the malicious software family classification capability is improved.