The invention provides a design method of a safety face verification system based on a CNN (convolutional neural network) feature extractor, belongs to the field of biological feature identification, and particularly relates to a method of utilizing the CNN to extract face features and using a Paillier algorithm and an oblivious transfer technique to encrypt. Compared with the SCiFi (secure computation of face identification) system, the method has the advantages that the manually extracted feature is converted into the CNN self-learning feature, and the CNN self-learning feature is performed with binarization to remove the noise effect, so that the identification accuracy is higher; the testing identification rate is 91.48% on a view 2 of an LFW (labeled face wild) base; in the whole identification process, a server will not obtain any feature information of a requester, and only receive the feature ciphertext information, but not decrypt; a client only obtains whether the identification is passed or not, and does not know the other information, including hamming distance; one face picture is expressed by the 320bit feature, and compared with the SCiFi system, the feature data volume is decreased by 2/3, so that the consumption time of encrypting and identification is low, and the real-time performance is high.