Method for determining similar character strings, method and system for file duplication checking

Abstract

The invention provides a method for determining similar character strings. The method comprises the steps: obtaining character arrays of sample files and character arrays of target files to be detected; constructing a matrix M, wherein rows and columns of the matrix M are corresponding to the character arrays of the sample files and the character arrays of the target files respectively; looking for a sub-square matrix meeting the similar character string conditions in the matrix M, wherein the similar character string conditions are arranged as following: if the character strings of the rows and columns corresponding to elements (img file='DDA0001205361200000011.TIF' wi='203' he='55' / )..., (img file='DDA0001205361200000012.TIF' wi='70' he='47' / ) are the same in the sub-square matrix, and then the character strings mapped by (img file='DDA0001205361200000013.TIF' wi='203' he='47' / )..., (img file='DDA0001205361200000014.TIF' wi='78' he='47' / ) are determined to be similar character strings, wherein, K represents the order of the sub-square matrix, j1, j2, j3, ... jk is an arrangement of 1, 2, ..., k. According to the method for determining the similar character strings, the fully-finding rate and the precision rate of the files can be improved by finding duplicate files.

Description

technical field [0001] The invention relates to the technical field of plagiarism checking of papers, in particular to a method and system for document plagiarism checking based on word segmentation fuzzy matching. Background technique [0002] At present, paper / file duplication rate detection mainly uses paper detection systems such as PaperPass, Wanfang, and HowNet, and uses string matching algorithms to calculate the similarity ratio of the file to be detected relative to the target file in the file library. [0003] The string matching algorithm uses a piece of text to be completely consistent as the standard for measuring the repetition of papers. However, due to the complexity of the Chinese language and the diversity of expressions, for two paragraphs of text with the same substantive content, there are often some meaningless gaps in the middle. "Stop words" or function words or situations such as subject-verb-object inconsistency, and it is wrongly judged as not belo...

AI Technical Summary

Problems solved by technology

[0003] The string matching algorithm uses a piece of text to be completely consistent as the standard for measuring the repetition of papers. However, due to the complexity of the Chinese language and the diversity of expressions, for two paragraphs of text with the same substantive content, there are often some meaningless gaps in the middle. "Stop words" or function words or situations such as subject-verb-object inconsistency, and it is wrongly judged as not belonging to repeated content, therefore, adopting the character string matching algorithm in the prior art may cause recall rate and precision rate to be inconsistent high

Moreover, the string matching algorithm has strict requirements on the selection of strings, the complexity of the algorithm itself is relatively high, and it requires relatively large resource overhead and long computing time. Therefore, the efficiency of duplicate checking is not high.

Images