The invention relates to an email filtering using relationship and reputation data. The subject disclosure is directed towards reducing the amount of resources needed to scan email messages for spam. In general, the previous email relationship between a sender and recipient, if any, may be considered in determining how aggressive the filtering level is set for scanning a message for spam, e.g., which filters will be used in the scan. For existing relationships where there has been no previously detected spam (there is good reputation data associated with the relationship), a less aggressive filtering level may be used, thereby saving resources. A relationship may be directly between the sender and recipient, or may be indirect, e.g., via a common third party. Also described is differentiating email from bulk senders from other email messages, for different handling, including spam filtering.