The invention provides a video auditing method and device, a video server and a readable storage medium, and relates to the technical field of Internet. According to the method, after the to-be-audited video is obtained, the picture content and/or the audio content of the to-be-audited video are/is correspondingly extracted to calculate the fragment similarity between the to-be-audited video and each reference verification video recorded in the preset verification video library, and then each calculated fragment similarity is compared with the preset similarity threshold, a corresponding similarity comparison result is obtained; and when the similarity comparison result shows that at least one target verification video of which the corresponding fragment similarity is greater than or equal to a preset similarity threshold exists in a preset verification video library, the to-be-audited video is labeled as a non-original video of the reference target verification video, therefore, the original video condition is effectively audited, so that a video platform limits low-value meaningless non-original videos from flowing into a network, and the video browsing experience of a user is improved.