A method for tracing the source of text documents

Abstract

The present invention relates to the technical field of text documents, and discloses a method for text document source tracing, comprising: a user obtains an original document from a client; a server segments the original document using the document sentences in the original document as coordinate units; a synonymous watermark is added to the segmented original document according to a preset synonymous watermark generation rule to generate a synonymous watermark document, wherein the synonymous watermark document includes two-dimensional coordinate array data A1; the document is checked according to a preset synonymous watermark extraction rule to obtain two-dimensional array information A2 of document content changes; a similarity threshold is preset, and when the similarity of the checked documents is greater than or equal to the similarity threshold, a synonymous watermark document with a synonymous watermark added is obtained; user registration information is obtained from the synonymous watermark document, the user is confirmed according to the user registration information, and the user is marked as a leaking user for a penalty operation. The present invention is used to solve the problem of adding watermarks to text documents to track and trace document leaks.

Description

Technical Field

[0001] The present invention relates to the technical field of text documents, and in particular to a method for tracing the source of text documents, which is used to solve the problem of adding watermarks to text documents to track and trace the source of document leaks. Background Art

[0002] The Internet started with the display of static text documents. In the era of mobile Internet, the online content of text documents is still an indispensable part of the Internet. For some companies, such as novel websites, paper duplicate checking, news websites, and document library websites, text documents are the basis of corporate profits, and they are often stolen by illegal gray industry practitioners for illegal profits. Taking novel websites as an example, gray industry practitioners use crawlers to crawl paid novels by registering accounts, and then upload them to pirated websites for other users to read for free, and make a lot of profits by placing advertisements, which seriously damages the interests of authors and novel reading websites. The novel website can only report the pirated website and ban it, but the pirates can easily build a website again, which does not fundamentally solve the problem.

[0003] In order to solve the above problems, a method is urgently needed to generate different watermarks for different users, so that when the novel is crawled by the crawler and put on the pirated website, the novel website can detect the watermark traces in the novel, check the account and balance information used by the crawler, and cut off the supply of pirated copies from the source, significantly increasing the cost of piracy. At the same time, this method can also play a very good role in tracking and tracing the source for other companies that have traceability requirements for text documents and protecting corporate interests. Summary of the invention

[0004] The purpose of the present invention is to provide a method for text document traceability, which solves the problem of adding watermarks to text documents for document leakage traceability and has good traceability effect and robustness effect.

[0005] The present invention is implemented by the following technical solution: A method for tracing the source of a text document comprises the following steps:

[0006] Step S1. The user sends a request signal for obtaining the original document on the client, and obtains the original document corresponding to the user registration information of the user;

[0007] Step S2. After receiving the request signal, the server divides the original document using the document sentences in the original document as coordinate units;

[0008] Step S3. Adding a synonymous watermark to the segmented original document according to a preset rule for generating a synonymous watermark to generate a synonymous watermark document, wherein the synonymous watermark document includes two-dimensional coordinate array data A1;

[0009] Step S4. Check the document according to the preset synonymous watermark extraction rule to obtain the two-dimensional array information A2 of the document content change;

[0010] Step S5. Preset a similarity threshold, and when the similarity of the checked documents is greater than or equal to the similarity threshold, obtain a synonymous watermark document with a synonymous watermark added;

[0011] Step S6. Obtain user registration information from the synonymous watermark document, confirm the user according to the user registration information, mark the user as a leaked user, and perform a penalty operation on the leaked user.

[0012] In this technical solution, the user obtains the original text document content to be generated with the watermark, that is, the original document, by request. This technical solution mainly traces the document from two major directions. The first is the watermark generation technical solution, and the second is the watermark extraction technical solution. In the watermark generation, the document is divided into sentences, the Y-axis coordinate is the sequence number of the sentence, and the X-axis coordinate is the position of the word in the sentence. Sentences are randomly selected as the sequence to be watermarked, and the sentence numbers [y1, y2, y3, ..., yn] are recorded. Machine learning is used to perform machine analysis semantics, and the words in the sentence are randomly replaced. The x-coordinate of the position of the replaced word is recorded, and the two-dimensional array information of the modified point is obtained for storage, and finally the modified document is sent to the user.

[0013] Extract the watermark part and divide the document into sentences. The Y-axis coordinate is the sequence number of the sentence, and the X-axis coordinate is the position of the word in the sentence. Compare it with the original document, and get the two-dimensional array information of the inconsistency between the current document and the original document and the stored two-dimensional array information. Perform similarity comparison analysis and obtain the corresponding user registration information. Find the user who maliciously leaks and ban or otherwise punish the user, thereby cutting off the supply of piracy from the source and significantly increasing the cost of piracy.

[0014] In order to better implement the present invention, further, the user registration information in step S1 includes the user's ID number information, the user's mobile phone number information and the user's username information.

[0015] In this technical solution, the user registration information can be bound to the document later, which helps us find crawler users, that is, malicious leakage users. The user registration information here is also different from the user registration information in other document tracing processing methods. Because after the user registration information in this technical solution is bound to the document, a document will generate a random sequence based on different user registration information and will be bound to different two-dimensional array information. The user registration information in other document tracing processing methods will not be bound to the two-dimensional array information in the document.

[0016] In this way, when each user registers and requests a text document, different data effects will be generated.

[0017] In order to better implement the present invention, further, step S2 includes:

[0018] A document sentence coordinate axis is preset, with the number of document sentence lines as the Y-axis coordinate and the position of the document sentence text as the X-axis coordinate; the original document is divided into document lines according to the document sentence coordinate axis.

[0019] In this technical solution, a method of adding watermarks using text content is used to perform machine learning analysis on the semantics in the document content. Before using the machine learning model to replace synonyms, the model must be trained and learned.

[0020] In order to better implement the present invention, further, the rules for generating synonymous watermarks in step S3 include:

[0021] According to the user registration information, a document line is randomly selected from the segmented original document to generate a random replacement position;

[0022] Perform synonym replacement on the text at the random replacement position according to machine learning, and record the synonym replacement information as a synonym watermark;

[0023] Preset a character quantity replacement rate, replace the random replacement position according to the character quantity replacement rate, obtain the two-dimensional coordinate array data A1 of the random replacement position, and obtain a synonymous watermark document;

[0024] The two-dimensional coordinate array data A1 and the corresponding user are bound to each other, and the two-dimensional coordinate array data A1, the registration information of the corresponding user and the binding relationship between the two are stored in the server.

[0025] In this technical solution, the two-dimensional coordinate array data A1 is used as the digital watermark information of the text document, which helps to compare the checked document with the original document on the server to find the leaked document, and the generation of synonymous watermark rules helps to generate watermark information to achieve the purpose of discovering the leaked account. This technical solution uses the principle of image watermarking, which can convert random replacement positions into digital watermarks in text documents, which can be traced very conveniently without affecting the user's reading experience of the text, and increases the cost of illegal acquisition. For novel websites or other text content, there is no need to invest in the inefficient method of banning illegal pirated websites, but directly deal with the source of the leak.

[0026] In order to better implement the present invention, further, the random replacement position includes the text content position and text line number position to be replaced in the segmented original document;

[0027] The text synonym replacement information includes the replaced text position information and the replaced text information.

[0028] The random replacement position in the technical solution includes the replaced text position information and the replaced text information. By randomly screening this information, multiple groups of synonym libraries are supplemented and added to ensure the comprehensiveness and completeness of the subsequent addition of synonym watermarks.

[0029] In order to better implement the present invention, further, performing synonym replacement on the text at the randomly replaced position according to machine learning includes:

[0030] Pre-training of machine learning based on statistical models and rule models.

[0031] In this technical solution, in the process of generating watermarks, replacing the content of the original document with words with similar semantics will change the original semantics, causing obstacles to user reading. In this case, machine learning needs to be used to get as close to the original semantics as possible.

[0032] In practice, there are two main solutions for converting synonymous content: rule-based and statistical-based. The rule-based model relies on linguistic theory. The statistical-based model, such as the Bayesian deep learning model, extracts the correspondence between "words", "sentences", and "semantics" from a large amount of text, and its effect is far better than the rule-based model. The Bayesian deep learning model is used to perform statistical analysis on a large amount of Chinese text data to generate a model of synonym groups for use in generating watermarks.

[0033] In order to better implement the present invention, further, step S4 includes: step S4.1. obtaining the document to be checked;

[0034] Step S4.2. Split the document to be checked into rows using the document sentences as coordinate units;

[0035] Step S4.3. Compare the segmented document to be checked with the unsegmented document to be checked, and obtain the two-dimensional array information A2 of the document content changes.

[0036] In the present technical solution, by screening the documents and comparing the segmented documents to be screened with the unsegmented documents to be screened, a two-dimensional array information A2 of document content changes can be obtained, which can ensure comprehensive document screening. The two-dimensional array information A2 can be regarded as a picture, that is, the document modification content is converted into a picture array, which can better understand the two-dimensional array information A2 of document content changes.

[0037] In order to better implement the present invention, further, step S5 includes: calculating the similarity of the screened documents according to the two-dimensional coordinate array data A1, the two-dimensional array information A2 and the similarity formula;

[0038] Similarity formulas include Euclidean distance formula, cosine distance formula, Hamming distance formula and histogram formula.

[0039] In this technical solution, the similarity formula is used to perform similarity analysis on the two-dimensional coordinate array data A1 and the two-dimensional array information A2 to obtain the document similarity, and then the user analysis is performed based on the document similarity. This method can ensure a certain degree of anti-interference. Even if the leaker modifies the document content, it can be considered that the similarity document within the confidence interval matches the user.

[0040] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0041] (1) The present invention is based on the architecture of the plain text document traceability method and has good robustness. The format of the text document is not limited to TXT format, but also includes other editable document formats such as WORD;

[0042] (2) The present invention is based on the mechanism of generating and verifying watermarks of text documents, which mainly converts the modified content of the document into an image array and then performs similarity analysis. The similarity analysis is not limited to the methods of calculating similarity such as histogram distance, Euclidean distance, and Hamming distance;

[0043] (3) The present invention proposes a method for adding watermarks to text documents, which has good traceability effect and robustness;

[0044] (4) The present invention proposes a method of mapping the content of a text document into an image, reusing the method of image similarity analysis, and tracing the source of the document. It has certain application scenarios for tracing the source of novel websites, paper websites, or other confidential documents;

[0045] (5) The present invention uses machine learning to perform synonym replacement. Through semantic replacement, it can achieve a better tracing effect without affecting the confidentiality of the original text reading. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The present invention is further described in conjunction with the following drawings and embodiments, and all the concepts and innovations of the present invention should be regarded as the disclosed contents and the protection scope of the present invention.

[0047] Figure 1 The present invention discloses a flowchart of a method for text document source tracing. DETAILED DESCRIPTION

[0048] Embodiment 1:

[0049] A text document source tracing method of this embodiment, such as Figure 1As shown, this embodiment proposes a method for tracing the source of text-based document leaks. For pictures, we track the pictures by adding visible digital watermarks or invisible digital watermarks, and we can easily query whether the picture is illegally obtained or directly traced to the person who leaked the picture. However, this method cannot be used for text-based documents such as TXT, DOCX and other formats, unless they are converted into PDF format that cannot be modified and then watermarked. For novel reading websites, they are deeply affected by crawler accounts. After the content is obtained by crawlers, it is uploaded to pirated websites, resulting in damage to corporate interests. It is very inconvenient to circumvent this problem by converting formats to protect the text content from being leaked. The present invention mainly solves the problem of adding watermarks to text-based documents to track and trace document leaks. The current mainstream way to track and trace text documents and prevent piracy is to convert text documents into pictures and then add image watermarks. This method cannot track plain text documents.

[0050] This embodiment proposes a method of adding watermarks in the form of text, which can effectively reduce the impact of text modification and editing. At the same time, in order to prevent the watermark from being destroyed after the text is modified, multi-dimensional watermarks are used for protection, which has strong anti-modification robustness. Its protection effect is similar to that of image watermarks. Compared with image watermarks, document watermarks remove the document's traceability watermark by modifying the document content. Using a program or manually will affect the readability of the document, increasing the cost of hiding for thieves.

[0051] In this embodiment, the user obtains the original text document content to be generated watermark by request, that is, the original document. This embodiment mainly traces the document from two major directions. The first is the watermark generation technology method, and the second is the watermark extraction technology method. In the watermark generation, the document is divided into sentences as units, the Y axis coordinate is the sequence number of the sentence, the X axis coordinate is the position of the word in the sentence, and the sentence number [y1, y2, y3, ..., yn] is randomly selected as the sequence to be watermarked, and the sentence number [y1, y2, y3, ..., yn] is recorded. Machine learning is used to perform machine analysis semantics, and the words in the sentence are randomly replaced. The x coordinate of the position of the replaced word is recorded, and then the two-dimensional array information of the modified point is obtained for storage, and finally the modified document is sent to the user. The watermark part of the sentence is extracted and divided into units, the Y axis coordinate is the sequence number of the sentence, and the X axis coordinate is the position of the word in the sentence. It is compared with the original document to obtain the two-dimensional array information of the inconsistent points between the current document and the original document and the stored two-dimensional array information, and the similarity is calculated for comparative analysis to obtain the corresponding user registration information. Find the user who maliciously leaks the information and ban or otherwise punish him / her, thereby cutting off the supply of pirated content at the source and significantly increasing the cost of piracy.

[0052] Embodiment 2:

[0053] This embodiment is further optimized on the basis of embodiment 1. The user registration information can be bound to the document later, which helps us find crawler users, that is, malicious leaking users. The user registration information here is also different from the user registration information in other document tracing processing methods, because after the user registration information in this embodiment is bound to the document, a document will generate a random sequence according to different user registration information, and will be bound to different two-dimensional array information. The user registration information in other document tracing processing methods will not be bound to the two-dimensional array information in the document. Take the image-based digital watermark processing method as an example. The image-based digital watermark processing method is only a conversion between documents and pictures. The user information here may be displayed through image watermarks and other methods, and will not be bound to the two-dimensional array information in the document. The same is true for the character segmentation processing method. Therefore, only from the role of user registration information, this embodiment is different from the image-based digital watermark processing method and the character segmentation-based text processing method. The method of adding watermarks in the form of text in this embodiment can effectively reduce the impact of text modification and editing. At the same time, in order to prevent the watermark from being destroyed after the text is modified, multi-dimensional watermarks are used for protection, which has strong robustness against modification.

[0054] In this way, when each user registers and requests a text document, different data effects will be generated.

[0055] The other parts of this embodiment are the same as those of Embodiment 1, and thus will not be described in detail.

[0056] Embodiment 3:

[0057] This embodiment is further optimized on the basis of Embodiment 1. It uses a method of adding watermarks using text content to perform machine learning analysis on the semantics in the document content. Without affecting reading, it uses a coordinate axis segmentation method to replace and modify characters in units of sentences and paragraphs, and covertly add text watermarks while retaining the replacement information. It uses the watermarked document and the original document for cross-comparison analysis to restore the watermarked content and achieve the effect of tracing the source.

[0058] The other parts of this embodiment are the same as those of Embodiment 1, and thus will not be described in detail.

[0059] Embodiment 4:

[0060] This embodiment is further optimized on the basis of embodiment 1. The two-dimensional coordinate array data A1 is used as the digital watermark information of the text document, which helps to compare the checked document with the original document on the server to find the leaked document. The generation of synonymous watermark rules helps to generate watermark information to achieve the purpose of discovering the leaked account. This embodiment uses the principle of image watermarking, which can convert the random replacement position into a digital watermark in the text document, which can be traced very conveniently without affecting the user's reading experience of the text, and increases the cost of illegal acquisition. For novel websites or other text content, there is no need to invest in the inefficient method of banning illegal pirated websites, but directly deal with the source of the leak.

[0061] The preset text replacement rate in this embodiment is to ensure readability. If too many words are replaced, it will affect the user's reading experience. For example, a maximum value is set not to exceed 10%. Then, for an article of 1,500 words, at most 150 words are replaced. Although the replacement method using machine learning can make the replacement as close to the original semantics as possible, too many words will still make the reading difficult.

[0062] When adding a watermark in this embodiment, the reading experience of the document will not be affected at all by adding invisible spaces or replacing Chinese and English punctuation marks. At the same time, by replacing similar synonyms through machine learning, the reader will hardly have any reading difficulties. In addition, this embodiment needs to calculate the watermark for the first time. When the document is requested again later, the document with the watermark added can be directly generated according to the stored watermark information. Due to the different random sequences generated according to the information of each user, the watermark information generated by each user is completely different. It is necessary to preset the text replacement rate according to the number of texts in the document to ensure the proportion of replaced and modified content and prevent the problem of reduced document readability caused by too high a modification rate.

[0063] The pre-training of machine learning based on statistical models and rule models in this embodiment refers to the learning model for synonyms. Due to different scenarios of text documents, such as novels, government documents, news, etc., when we replace synonyms, we need to judge the content of the text document and replace it with words as close as possible. A large amount of data training must be used here. For novel websites, you can directly use your own document content and the document content publicly available on the Internet as the data set of your own training model, and pre-process the data set at the same time, process the novel classification label, perform data inspection, identify spelling errors, convert the data format, etc. to ensure the quality of the data set. Then use the pre-processed data to train the model. During the training process, the test set needs to be split, most of it (such as 80%) as the training set, and a small part as the test set. Use the training set to establish a prediction model. After the training is completed, the quality of the model will be verified by the test set, and the model will be tested. At the same time, during the training process, human intervention is required to verify the effect of the model. Without affecting reading, the coordinate axis segmentation method is used to replace and modify characters in units of sentences and paragraphs, and to covertly add text watermarks. At the same time, the replacement information is retained, and the watermarked document is cross-compared with the original document to restore the watermark content and achieve the effect of tracing the source.

[0064] The other parts of this embodiment are the same as those of Embodiment 1, and thus will not be described in detail.

[0065] Embodiment 5:

[0066] This embodiment is further optimized on the basis of the above-mentioned embodiment 4. The random replacement position in this embodiment includes the replaced text position information and the replaced text information. By randomly screening this information, multiple groups of synonym libraries are supplemented and added to ensure the comprehensiveness and completeness of the subsequent addition of synonym watermarks.

[0067] The other parts of this embodiment are the same as those of the above-mentioned embodiment 4, so they will not be described in detail.

[0068] Embodiment 6:

[0069] This embodiment is further optimized on the basis of the embodiment 1. In the process of generating the watermark, replacing the content of the original document with words with similar semantics will change the original semantics, causing obstacles to user reading. In this case, it is necessary to use machine learning to get as close to the original semantics as possible.

[0070] In practice, there are two main solutions for converting synonymous content: rule-based and statistical-based. The rule-based model relies on linguistic theory. The statistical-based model, such as the Bayesian deep learning model, extracts the correspondence between "words", "sentences", and "semantics" from a large amount of text, and its effect is far better than the rule-based model. The Bayesian deep learning model is used to perform statistical analysis on a large amount of Chinese text data to generate a model of synonym groups for use in generating watermarks.

[0071] This embodiment uses machine learning to perform synonym replacement, assisted by invisible symbols such as space and tab keys, Chinese and English symbols such as commas, periods, exclamation marks, etc., and records text replacement position information and replaced text.

[0072] The other parts of this embodiment are the same as those of Embodiment 1, and thus will not be described in detail.

[0073] Embodiment 7:

[0074] This embodiment is further optimized on the basis of embodiment 1. By checking the documents, the segmented documents to be checked and the unsegmented documents to be checked are compared to obtain a two-dimensional array information A2 of document content changes, which can ensure comprehensive document checking. The two-dimensional array information A2 can be regarded as a picture, that is, the document modification content is converted into a picture array, which can better understand the two-dimensional array information A2 of document content changes.

[0075] The other parts of this embodiment are the same as those of Embodiment 1, and thus will not be described in detail.

[0076] Embodiment 8:

[0077] This embodiment is further optimized on the basis of embodiment 1, and uses a similarity formula to perform similarity analysis on the two-dimensional coordinate array data A1 and the two-dimensional array information A2 to obtain the document similarity, and then performs user analysis based on the document similarity. This method can ensure a certain degree of anti-interference. Even if the leaker modifies the document content, it can be considered that the similarity document within the confidence interval matches the user.

[0078] This embodiment selects a simple and effective histogram calculation method as a demonstration of similarity formula calculation, and the similarity formula is as follows: ;

[0079] A1 is the two-dimensional coordinate array data A1 in the synonymous watermark document, A2 is the two-dimensional array information A2 of the document content change, and I is the data of each two-dimensional coordinate axis point. Each point is calculated in turn. If it has not been tampered with by those pirated websites, the distance should be 0.

[0080] The similarity threshold in this embodiment is actually the variance distance calculated according to the similarity formula, which is used to measure the difference between two two-dimensional arrays.

[0081] The other parts of this embodiment are the same as those of Embodiment 1, and thus will not be described in detail.

[0082] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any simple modification or equivalent change made to the above embodiment based on the technical essence of the present invention shall fall within the protection scope of the present invention.

Claims

1. A method for tracing the source of a text document, characterized in that: The following steps are involved: Step S1. The user sends a request signal for obtaining an original document on the client, and obtains the original document corresponding to the user registration information of the user; Step S2. After receiving the request signal, the server segments the original document using the document sentences in the original document as coordinate units; Step S3. Adding a synonymous watermark to the segmented original document according to a preset synonymous watermark generation rule to generate a synonymous watermark document, wherein the synonymous watermark document includes two-dimensional coordinate array data A1; Step S4. Check the document according to the preset synonymous watermark extraction rule to obtain the two-dimensional array information A2 of the document content change; Step S5. Preset a similarity threshold. When the similarity of the checked documents is greater than or equal to the similarity threshold, obtain a synonymous watermark document with a synonymous watermark added. Step S6. Obtain user registration information from the synonymous watermark document, confirm the user according to the user registration information, mark the user as a leaked user, and punish the leaked user. The step S2 includes: presetting a document sentence coordinate axis, with the number of document sentence lines as the Y-axis coordinate and the position of the document sentence text as the X-coordinate; dividing the original document into document lines according to the document sentence coordinate axis; The rules for generating synonymous watermarks in step S3 include: randomly selecting the document line in the segmented original document to generate a random replacement position according to the user registration information; replacing the text at the random replacement position with a synonym according to machine learning, and recording the text synonym replacement information as a synonymous watermark; presetting a text quantity replacement rate, replacing the random replacement position according to the text quantity replacement rate, obtaining the two-dimensional coordinate array data A1 of the random replacement position, and obtaining a synonymous watermark document; binding the two-dimensional coordinate array data A1 and the corresponding user, and storing the two-dimensional coordinate array data A1, the registration information of the corresponding user and the binding relationship between the two to the server.

2. A method for tracing the source of a text document according to claim 1, characterized in that: The user registration information in step S1 includes the user's ID number information, the user's mobile phone number information and the user's username information.

3. A method for tracing the source of a text document according to claim 1, characterized in that: include: The random replacement position includes the text content position and text line number position to be replaced in the segmented original document; The text synonym replacement information includes the replaced text position information and the replaced text information.

4. A method for tracing the source of a text document according to claim 1, characterized in that: The replacing the text at the randomly replaced position with a synonym according to machine learning includes: performing pre-training of machine learning according to a statistical model and a rule model.

5. A method for tracing the source of a text document according to claim 1, characterized in that: The step S4 includes: step S4.

1. obtaining the document to be checked; step S4.

2. dividing the document to be checked into rows using the document sentences as coordinate units; step S4.

3. comparing the divided document to be checked with the undivided document to be checked, and obtaining the two-dimensional array information A2 of the document content changes.

6. A method for tracing the source of a text document according to claim 1, characterized in that: The step S5 comprises: calculating the similarity of the checked documents according to the two-dimensional coordinate array data A1, the two-dimensional array information A2 and the similarity formula; The similarity formulas include Euclidean distance formula, cosine distance formula, Hamming distance formula and histogram formula.

Images