The invention discloses an irrelevant label filtering method based on depth feature clustering and semantic measurement. The method comprises the following steps: 1, acquiring an image set by a sensor; 2, establishing a label set corresponding to the image set; 3, extracting depth features of the images in the image set; 4, clustering the depth features to obtain a cluster; 5, constructing a related semantic label set of the clustering cluster; 6, constructing a to-be-measured label set of the clustering cluster; 7, generating a semantic vector; 8, calculating the relevancy of the semantic vectors; and 9, filtering the irrelevant labels according to the relevancy. According to the method, the clustering cluster is obtained by clustering huge sample image data and used for pre-classifying the sample image data, the clustered sample image data is analyzed, higher effectiveness and correctness are achieved, meanwhile, relevancy measurement is conducted on label semantics, and therefore automatic filtering of irrelevant labels is achieved, and the filtering accuracy is improved, the generalization and robustness of the deep network can be improved.