Domain-oriented scientific and technological project duplication checking method and system

Abstract

The application discloses a kind of field-oriented scientific and technological project duplication checking method and system, the method includes: respectively constructing project duplication checking contrast library, field dictionary and field stop word library;At least according to field dictionary and field stop word library, the inverse document frequency of each word in corpus is calculated;According to the inverse document frequency of each word in corpus, the keywords of materials in project duplication checking contrast library and declaration library are extracted;The repetition rate of the project to be checked and the project in duplication checking contrast library is calculated based on the extracted keywords, and the project with high similarity is determined according to the repetition rate.The field-oriented scientific and technological project duplication checking method and system of the application can help enterprise technology management personnel to compare the file to be checked with a large number of historical project information, find out the project with highly similar research content, can avoid the waste of enterprise technology resources caused by repeated project establishment, and ensure the fairness of project establishment.

Description

Technical Field

[0001] This invention relates to the field of plagiarism detection technology, and in particular to a method for detecting plagiarism in technology projects within a specific field. Background Technology

[0002] In recent years, the number of science and technology projects initiated by tobacco companies has continued to grow, accumulating a large amount of data on project applications and completions. However, due to the lack of effective plagiarism detection methods, science and technology management personnel find it difficult to conduct plagiarism checks on projects quickly and accurately using manual methods.

[0003] Domestic literature retrieval platforms such as CNKI and Wanfang provide paper plagiarism detection services by comparing data with massive amounts of papers, patents, and other literature in their databases. However, there are no reports on plagiarism detection for scientific and technological projects. Research projects in the tobacco industry have specific characteristics and include many ongoing projects and projects involving classified information that cannot be disclosed within a certain timeframe. Therefore, it is essential to develop plagiarism detection methods and systems specifically for scientific and technological projects in the tobacco industry.

[0004] Therefore, there is an urgent need for a domain-specific method and system for detecting plagiarism in scientific and technological projects. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for checking the duplication of scientific and technological projects in a specific field, so as to solve the problems in the prior art, avoid the waste of enterprise scientific and technological resources caused by duplicate project approvals, and ensure the fairness of scientific and technological project approvals.

[0006] This invention provides a method for detecting plagiarism in scientific and technological projects within a specific field, comprising:

[0007] Construct a project plagiarism comparison database, a domain dictionary, and a domain stop word database respectively;

[0008] Based at least on the domain dictionary and the domain stop word database, the inverse document frequency of each word in a corpus consisting of scientific and technological literature and historical scientific and technological projects in the tobacco field is calculated, wherein the inverse document frequency of each word is used to characterize the general importance of the word.

[0009] Based on the inverse document frequency of each word in the corpus, keywords are extracted from the materials in the project plagiarism comparison database and the application database.

[0010] The duplication rate between the item to be checked and the items in the duplication comparison database is calculated based on the extracted keywords, and highly similar items are determined based on the duplication rate calculation results.

[0011] The domain-oriented technology project plagiarism detection method described above, preferably, includes the following steps: constructing a project plagiarism comparison library, a domain dictionary, and a domain stop word library, respectively.

[0012] Based on the application materials of completed projects, the completion materials, and the application materials of ongoing projects, a project plagiarism comparison database was constructed.

[0013] Constructing domain-specific dictionaries based on academic papers;

[0014] Construct a domain-specific stop word library based on domain-general vocabulary and words without substantive meaning.

[0015] The step of calculating the inverse document frequency of each word in a corpus composed of materials from scientific and technological literature and historical scientific and technological projects in the tobacco field, based at least on the domain dictionary and the domain stop word database, specifically includes:

[0016] Key fields are extracted from the corpus, wherein the scientific and technological documents in the corpus include at least one of Chinese and foreign language papers and patent documents; the key fields in Chinese and foreign language papers include at least one of title, abstract, and keyword fields; the key fields in patent documents include at least one of name and abstract fields; the historical scientific and technological projects in the corpus include application forms and technical reports; the key fields in application forms include at least one of project name and project objective fields; and the key fields in technical reports include at least one of abstract, project objective, and conclusion fields.

[0017] At least based on the domain dictionary and the domain stop word library, a preset word segmentation algorithm is used to segment non-keyword fields in the corpus, wherein the non-keyword fields include fields other than the paper keywords;

[0018] Calculate the inverse document frequency of each word in the word segmentation results.

[0019] In the domain-oriented scientific and technological project plagiarism detection method described above, preferably, the non-keyword fields in the corpus are segmented using a preset word segmentation algorithm based at least on the domain dictionary and the domain stop word database. The non-keyword fields include fields other than the paper's keywords, specifically including:

[0020] The Conditional Random Field algorithm is used to segment non-keyword fields in the corpus;

[0021] Based on domain characteristics, set the word length threshold to Len. max ;

[0022] Len was filtered out after word segmentation when its length exceeded the word length threshold. max The word;

[0023] In the word segmentation results, the reverse maximum matching method is adopted. Based on the domain dictionary, multiple adjacent words are combined into a word. If the word formed by combining multiple adjacent words exists in the domain dictionary, then the adjacent words are combined into a word.

[0024] Based on the stop word library, words in the stop word library after word segmentation are removed to obtain the word segmentation result.

[0025] The calculation of the inverse document frequency of each word in the word segmentation result specifically includes:

[0026] Based on the total number of documents in the corpus and the number of documents containing word i, the inverse document frequency of each word in the segmentation results is calculated using the following formula.

[0027]

[0028] Among them, IDF i Let represent the inverse document frequency of word i, D represent the total number of documents in the corpus, and d represent the number of documents containing word i.

[0029] The above-described field-oriented technology project plagiarism detection method, preferably, involves extracting keywords from the project plagiarism comparison database and the application database based on the inverse document frequency of each word in the corpus, specifically including:

[0030] Extract keywords from documents in the project's plagiarism comparison database;

[0031] Extract keywords from the application materials of the project to be checked for plagiarism.

[0032] The extraction of keywords from documents in the project's plagiarism comparison database specifically includes:

[0033] The document structure parsing and field content extraction method is used to extract the fields to be checked from each document in the project deduplication comparison library;

[0034] The extracted fields to be checked for plagiarism are segmented using a preset word segmentation algorithm;

[0035] Extract keywords from documents in the project's plagiarism comparison database based on the word segmentation results.

[0036] The extraction of keywords from the application materials of the project to be checked for plagiarism specifically includes:

[0037] The document structure parsing and field content extraction method is used to extract the fields to be checked for plagiarism in the application form of the project to be checked;

[0038] The pre-defined word segmentation algorithm is used to segment the fields to be checked for plagiarism in the application form of the project to be checked.

[0039] Extract keywords from the documents in the application materials of the project to be checked for plagiarism based on the word segmentation results.

[0040] The above-described domain-oriented technology project plagiarism detection method preferably employs a document structure parsing and field content extraction method to extract the fields to be checked from each document in the project plagiarism comparison database, specifically including:

[0041] For documents with fixed templates, each document in the project plagiarism comparison library is parsed according to the pre-configured field names to be extracted. By keyword matching, the corresponding keywords are found in the document, and the text content corresponding to the current keyword and the next keyword is extracted. The extracted text content is used as the field to be checked for plagiarism.

[0042] For documents without a fixed format, based on the visual features of the text, the system parses each document in the project plagiarism comparison database, predicts the titles and corresponding body text of each chapter, forms a document directory structure, and matches the names of fields to be extracted in the identified chapter titles. The text content between the first and next title is then extracted and used as the field to be checked for plagiarism.

[0043] The method employs document structure parsing and field content extraction to extract the fields to be checked in the application form of the project to be checked for plagiarism, specifically including:

[0044] Based on the pre-configured field names to be extracted, the document of the project application form to be checked for plagiarism is parsed. Through keyword matching, the corresponding keywords are found in the document, and the text content corresponding to the current keyword and the next keyword is extracted. The extracted text content is used as the field to be checked for plagiarism in the project application form.

[0045] The above-described field-oriented technology project plagiarism detection method preferably includes, in part, segmenting the extracted fields to be checked using a preset word segmentation algorithm, specifically comprising:

[0046] The Conditional Random Field algorithm is used to segment non-keyword fields in the corpus;

[0047] Based on domain characteristics, set the word length threshold to Len. max ;

[0048] Len was filtered out after word segmentation when its length exceeded the word length threshold. max The word;

[0049] In the word segmentation results, the reverse maximum matching method is adopted. Based on the domain dictionary, multiple adjacent words are combined into a word. If the word formed by combining multiple adjacent words exists in the domain dictionary, then the adjacent words are combined into a word.

[0050] Based on the stop word library, words in the stop word library after word segmentation are removed to obtain the word segmentation result.

[0051] The extraction of keywords from documents in the project plagiarism comparison database based on word segmentation results specifically includes:

[0052] Based on the word segmentation results, words existing in the domain dictionary are selected as candidate keywords;

[0053] Keywords from documents in the project plagiarism comparison database are extracted using a preset keyword extraction method from the candidate keywords.

[0054] The above-described field-oriented technology project plagiarism detection method preferably includes the following: The step of extracting keywords from the project plagiarism comparison database using a preset keyword extraction method specifically includes:

[0055] The word frequency of word i in document j is calculated using the following formula, based on the number of times word i appears in document j and the total number of words in document j.

[0056]

[0057] Among them, TF ij n represents the frequency of word i in document j. i,j N represents the number of times word i appears in document j. j This represents the total number of words in document j.

[0058] Based on the term frequency of word i in document j and the inverse document frequency of word i, the term frequency - inverse document frequency of word i relative to document j is calculated using the following formula:

[0059] TFIDF ij =TF i,j *IDF i (3)

[0060] Among them, TFIDF i,j This represents the TF-IDF value of word i relative to document j, where IDF is the TF-IDF value. i Indicates the inverse document frequency of word i;

[0061] Extract keywords from each document based on the term frequency-inverse document frequency of each word.

[0062] The above-described domain-oriented plagiarism detection method for scientific and technological projects preferably includes, in part, extracting keywords from a document based on the term frequency-inverse document frequency of each word in the document, specifically comprising:

[0063] Extract the N words with the highest term frequency-inverse document frequency from each document as the keywords for that document.

[0064] Alternatively, different weights can be assigned to words appearing in different parts of the document, and the weighted sum of term frequency-inverse document frequency (TFM) can be sorted in descending order. The N words with the highest TFM-inverse document frequency in each document can then be extracted as the keywords for that document.

[0065] Alternatively, an improved TextRank algorithm can be used to extract keywords from the document, and the initial weights of nodes and the transition probabilities between nodes can be optimized based on positional and semantic information. Specifically, this includes:

[0066] The nouns, verbs, and adjectives in the word segmentation results are retained to form a candidate keyword list. Using candidate keywords as nodes, if two words appear in the same sentence and co-occur within a sliding window of size 3, an edge is created between the nodes, constructing a network graph G = (V, E), where V = {v1, v2, ..., v...}. n Let E be a set of nodes, where E = {e1, e2, ..., e} m} is the set of edges;

[0067] Based on the importance of content at different locations within the document, the document is divided into first, second, and third positions, with the importance decreasing sequentially from first to third position.

[0068] The following formula assigns different weights to words appearing in different positions in a document.

[0069]

[0070] Among them, Position it Let p1 represent the weight of the position of word i in the document at the t-th occurrence, and p1 > p2 > 1;

[0071] Based on the weights of all occurrences of word i in the document, the sum of the weights of each occurrence of word i in document j is calculated using the following formula.

[0072] P ij =∑ t Position it (5)

[0073] Among them, P ij This represents the sum of the positional weights of word i in document j for each occurrence;

[0074] Based on the word's positional information, the term frequency-inverse document frequency of word i relative to document j is optimized using the following formula, and used as the starting weight for nodes in TextRank.

[0075]

[0076]

[0077] TFIDF i,j =TF i,j *IDF i (8)

[0078] Among them, P tj TF represents the sum of the position weights of each occurrence of the t-th word in document j. ij Let IDF represent the frequency of word i in document j, D represent the total number of documents in the corpus, and d represent the number of documents containing word i. i The inverse document frequency of word i, TF-IDF i,j This represents the TF-IDF value of word i relative to document j;

[0079] The following formula is used to normalize words that co-occur in a document, serving as a term frequency feature among them.

[0080]

[0081] Among them, F(w) x ,w y ) indicates the word w x and word w y The word frequency features between them, count(w x ,w y ) indicates the word w x and word w y The number of co-occurrences within a sliding window of a preset size, count(w) x ) indicates the word w x The number of times it appears in the document, count(w y ) indicates the word w y Number of times it appears in the document;

[0082] The cosine similarity between vectors corresponding to words that co-occur in a document is calculated using the following formula, and used as a semantic feature between words.

[0083]

[0084] Where X represents the word w x The corresponding word vector, Y represents word w y The corresponding word vectors, where m represents the dimension of the word vectors;

[0085] Based on word frequency and semantic features, the weights of edges between nodes in TextRank are calculated using the following formula.

[0086]

[0087] Where w(x,y) represents node vx and node v y The weight of the edges between them;

[0088] The weights of each node are iteratively updated according to the following formula (12) until convergence.

[0089]

[0090] Wherein, S(v x ) represents node v x The weights, N(v) x ) indicates the relationship with node v x A set of connected nodes, where d represents the damping coefficient, which characterizes the probability of the current node jumping to a connected node;

[0091] In the above-described domain-oriented technology project plagiarism detection method, a preferred approach is to sort the nodes according to their weights and select the top N as the document's keywords.

[0092] The process of calculating the similarity rate between the item to be checked and the items in the plagiarism comparison database based on the extracted keywords, and determining highly similar items based on the calculation results of the similarity rate, specifically includes:

[0093] The application form of the project to be checked for plagiarism is compared with the documents in the project plagiarism comparison database, and the document duplication rate is calculated.

[0094] Sort the calculation results in descending order of repetition rate;

[0095] Items with a duplication rate greater than or equal to a preset similarity threshold are selected as highly similar items and used as the result of the duplication check.

[0096] The process of comparing the application form of the project to be checked with documents in the project plagiarism comparison database and calculating the document duplication rate specifically includes:

[0097] Let A be the r-th field to be checked for plagiarism extracted from the application form of the project S to be checked for plagiarism. r The fth item in the project plagiarism comparison database is the project material from a historical project Pro. f The g-th field to be checked for plagiarism is Pro fg ;

[0098] If A r With Pro fg If fields belong to the same type, then according to the first preset keyword matching rule, A will be matched sequentially. r Each keyword in Pro fg The keywords in the text are matched, and if a match is found, the Pro will be... fg With A r count(pro)fg Add 1, and calculate Pro using the following formula. fg With A r The repetition rate

[0099]

[0100] Among them, RF (Pro fg ) indicates Pro fg With A r The repetition rate, count(pro) fg ) indicates Pro fg In and A r The number of matching keywords, Keywords(pro) fg ) indicates Pro fg The total number of keywords in Pro fg If there are no keywords in the text, then RF (Pro) fg If A is 0, the first preset keyword matching rule is: if A r The keyword in Pro is equal to f Keywords or containing Pro f If the keywords in the text are not found, they are considered to match the rules.

[0101] If A r With Pro fg Fields that do not belong to the same type will not have their duplication rate calculated.

[0102] For each field to be checked in the application form of the project to be checked for plagiarism, and for the f-th project material Pro in the project plagiarism comparison database, f Perform the above operation on each field to be checked for plagiarism in the application form S, and obtain the results of checking each field to be checked for plagiarism in the application form S and Pro. f The repetition rate of each field of the same type to be checked for duplicates, let Pro f If the number of fields of the same type to be checked for duplication in Project S's application is M, then Pro f The similarity rate with the item S to be checked is taken as Pro f The maximum repetition rate of all fields to be checked for duplicates, expressed by the following formula:

[0103] RF (Pro fg ) = max(RF(Pro f1 ),RF(Pro f2 ),……,RF(Pro fM )) (14),

[0104] Alternatively, the weights of each field to be checked for duplication can be pre-set, and the duplication rate of each field can be weighted and averaged to obtain the f-th project material Pro from a historical project Pro in the project duplication comparison database. f The similarity rate with the item S to be checked for plagiarism is expressed by the following formula:

[0105] RF (Pro f )=∑ f,g RF(pro fg )·w c (15)

[0106] Among them, w c Indicates Pro f The weight of the c-th field to be checked for duplicates;

[0107] Let D be the number of project materials for historical project Pro in the project plagiarism comparison database. Then, the duplication rate between project Pro and the project to be checked for duplication S is the maximum value of the duplication rates of all project materials of project P and the project to be checked for duplication S:

[0108] RF(Pro)=max(RF(Pro1),RF(Pro2),…,RF(Pro D )) (16),

[0109] Alternatively, let A be the r-th field to be checked extracted from the application form of the project S to be checked for plagiarism. r The fth item in the project plagiarism comparison database is the project material from a historical project Pro. f The g-th field to be checked for plagiarism is Pro fg If A r With Pro fg If fields belong to the same type, then according to the second preset keyword matching rule, A will be matched sequentially. r Each keyword in Pro fg The keywords in the text are matched, and if a match is found, the Pro will be... fg With A r degree of matching fg Add this keyword to the fth item in the Pro historical project list. f The weight value in Pro is the term frequency-inverse document frequency value, calculated using the following formula: f With A r The repetition rate

[0110] degree(pro fg )=∑ h∈H weight(h) (17)

[0111]

[0112] Where H represents Pro fg With A r The set of matched keywords, weight(h) represents the h-th keyword in the f-th material of historical item Pro. f The weights in the degree(Pro) fg ) indicates Pro fg With A r The degree of matching, K represents Pro fg All keywords in the set, the matching rule for the second keyword is, if A r The keyword in Pro is equal to f Keywords or containing Pro f If the keywords in the text are not found, they are considered to match the rules.

[0113] If A r With Pro fg Fields that do not belong to the same type will not have their duplication rate calculated.

[0114] For each field to be checked in the application form of the project S to be checked for plagiarism, and for the f-th material Pro of project P in the project plagiarism comparison database... f Perform the above operation on each field to be checked for plagiarism in the application form S, and obtain the results of checking each field to be checked for plagiarism in the application form S and Pro. f The repetition rate of each field of the same type to be checked for duplicates, let Pro f If the number of fields of the same type to be checked for duplication in Project S's application is M, then Pro f The similarity rate with the item S to be checked is taken as Pro f The maximum repetition rate of all fields to be checked for duplicates, expressed by the following formula:

[0115] RF (Pro f ) = max(RF(Pro f1 ),RF(Pro f2 ),……,RF(Pro fM )) (19),

[0116] Alternatively, the weights of each field to be checked for plagiarism can be pre-set, and the Pro value can be obtained by weighted averaging the repetition rates of each field. f The similarity rate with the item S to be checked for plagiarism is expressed by the following formula:

[0117] RF(Prof f )=∑ f,g RF(pro fg )·w c (20)

[0118] Among them, wc Indicates Pro f The weight of the c-th field to be deduplicated.

[0119] Let D be the number of project materials for historical project Pro in the project plagiarism comparison database. Then, the duplication rate between project P and the project S to be checked is the maximum value of the duplication rates of all materials of Pro and S, and is expressed by the following formula:

[0120] RF(Pro)=max(RF(Pro1),RF(Pro2),…,RF(Pro D )) (twenty one),

[0121] Alternatively, text similarity methods can be used to calculate document duplication rates, specifically including:

[0122] The word embedding method is used to train word vectors in all documents in the project plagiarism comparison database to obtain the vector representation of each word.

[0123] Extract the content of all fields to be checked in the application form of the project S to be checked for plagiarism, and filter out sentences containing keywords to form a set of key sentences for the project S to be checked for plagiarism.

[0124] Extract the f-th item material from the historical item Pro in the item plagiarism comparison database. f The content of all fields to be checked for plagiarism is filtered to form sentences containing keywords. f A collection of key sentences;

[0125] Calculate the set of key sentences for the project S to be checked for plagiarism and the Pro selected from the f-th project material of the historical project Pro in the project plagiarism comparison database. f The similarity of key sentence pairs containing the same keywords in the key sentence set includes:

[0126] The vector of the key sentence is obtained by averaging the word vectors of each word in the key sentence using a weighted average of word frequency and inverse document frequency, and is expressed by the following formula:

[0127]

[0128] Where m represents the number of words in the key sentence, vec d This represents the word vector of the d-th word in the key sentence, and weight(d) represents the weight of the d-th word, which is the TF-IDF value of the d-th word relative to the current document.

[0129] Select the set of key sentences for the project S to be checked for plagiarism and the f-th project material Pro from the historical project Pro in the plagiarism comparison database. fThe set of key sentences contains key sentences with the same keywords. The cosine similarity of key sentence pairs is calculated using the following formula, where the cosine similarity of the key sentence pairs is used to characterize the similarity between sentence pairs:

[0130]

[0131] Where s1 represents the sentence vector of the key sentence in the application form of the project S to be checked for plagiarism, and s2 represents the f-th project material Pro from the historical project Pro in the project plagiarism comparison database. f The sentence vectors of the key sentences in the text, sim(s1,s2) represent the similarity between sentence vectors s1 and s2;

[0132] The average similarity of each sentence pair is taken as Pro. f The rate of overlap with the item S to be checked for plagiarism;

[0133] Let D be the number of materials in the historical project Pro in the project plagiarism comparison database. Then, the duplication rate between project Pro and the project to be checked S is the maximum value of the duplication rates of all materials of Pro and the project to be checked S, and is expressed by the following formula:

[0134] RF(Pro)=max(RF(Pro1),RF(Pro2),…,RF(Pro D )) (twenty four).

[0135] This invention also provides a field-specific plagiarism detection system for scientific and technological projects, comprising:

[0136] The database construction module is used to build the project plagiarism comparison database, the domain dictionary, and the domain stop word database, respectively.

[0137] The inverse document frequency calculation module is used to calculate the inverse document frequency of each word in a corpus consisting of scientific and technological literature and historical scientific and technological projects in the tobacco field, based at least on the domain dictionary and the domain stop word database. The inverse document frequency of each word is used to characterize the general importance of that word.

[0138] The keyword extraction module is used to extract keywords from the materials in the project plagiarism comparison database and the application database based on the inverse document frequency of each word in the corpus.

[0139] The duplication rate calculation module is used to calculate the duplication rate between the item to be checked and the items in the duplication comparison database based on the extracted keywords, and to determine the highly similar items based on the duplication rate calculation results.

[0140] This invention provides a method and system for checking the duplication of scientific and technological projects in a specific field. Based on a field dictionary and a field stop word database, the system calculates the inverse document frequency of each word in the corpus. Then, it extracts keywords from materials in a project duplication comparison database and an application database. Finally, based on these keywords, it calculates the repetition rate between the project to be checked and projects in the comparison database to identify highly similar projects. This helps enterprise science and technology managers compare the documents to be checked with a large amount of historical project information to find projects with highly similar research content, thus avoiding the waste of enterprise science and technology resources caused by duplicate project approvals and ensuring the fairness of scientific and technological project approvals. Attached Figure Description

[0141] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described below with reference to the accompanying drawings, wherein:

[0142] Figure 1 A flowchart illustrating an embodiment of the field-oriented scientific and technological project plagiarism detection method provided by the present invention;

[0143] Figure 2 This is a structural block diagram of an embodiment of the field-oriented scientific and technological project plagiarism detection system provided by the present invention. Detailed Implementation

[0144] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. The descriptions of the exemplary embodiments are merely illustrative and are in no way intended to limit the present disclosure or its application or use. The present disclosure may be implemented in many different forms and is not limited to the embodiments described herein. These embodiments are provided so that the present disclosure will be thorough and complete, and will fully express the scope of the disclosure to those skilled in the art. It should be noted that, unless specifically stated otherwise, the relative arrangement of components and steps, the composition of materials, numerical expressions, and values ​​set forth in these embodiments should be interpreted as exemplary only and not as limiting.

[0145] The terms “first,” “second,” and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different parts. Terms such as “including” or “contains” mean that the element preceding the term encompasses the element listed after it, and do not exclude the possibility of encompassing other elements as well. Terms such as “above” and “below” are used only to indicate relative positional relationships; when the absolute position of the described object changes, this relative positional relationship may also change accordingly.

[0146] In this disclosure, when a specific component is described as being located between a first component and a second component, an intermediary component may or may not be present between the specific component and the first or second component. When a specific component is described as connecting to other components, the specific component may be directly connected to the other components without having an intermediary component, or it may not be directly connected to the other components but may have an intermediary component.

[0147] All terms used in this disclosure (including technical or scientific terms) have the same meaning as understood by one of ordinary skill in the art to which this disclosure pertains, unless otherwise specifically defined. It should also be understood that terms defined in a general dictionary, such as a dictionary, should be interpreted as having a meaning consistent with their meaning in the context of the relevant art, and not as having an idealized or highly formalized meaning, unless expressly defined herein.

[0148] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, they should be considered part of the specification.

[0149] The method for detecting plagiarism in scientific and technological projects according to this invention is illustrated using the tobacco industry as an example. Those skilled in the art will understand that this method is also applicable to project plagiarism detection in other fields. Figure 1 As shown, the domain-oriented technology project plagiarism detection method provided in this embodiment includes the following steps in actual implementation:

[0150] Step S1: Construct the project plagiarism comparison library, the domain dictionary, and the domain stop word library respectively.

[0151] In one embodiment of the field-oriented scientific and technological project plagiarism detection method of the present invention, step S1 may specifically include:

[0152] Step S11: Based on the application materials of completed projects, the completion materials, and the application materials of ongoing projects, construct a project plagiarism comparison database.

[0153] Specifically, the project compiles application materials and completion documents for completed projects, as well as application materials for ongoing projects, to construct a project plagiarism comparison database. Application materials include application forms and implementation plans, while completion materials include work reports, technical reports, appraisal certificates, acceptance certificates, third-party evaluation reports, and other completion documents required by regulations in fields such as tobacco breeding, tobacco cultivation, tobacco plant protection, tobacco curing, tobacco technology, tobacco chemistry, tobacco product design, tobacco quality supervision, tobacco informatization, and tobacco metrology. The application and completion materials can be in doc, docx, or pdf formats.

[0154] Step S12: Construct a domain dictionary based on the paper.

[0155] Since there is no obvious boundary between words in the Chinese context, word segmentation has a great impact on the accuracy of text analysis. Applying general word segmentation tools to specific fields often fails to achieve satisfactory results due to the lack of out-of-vocabulary words in the field. Therefore, constructing a dictionary for the tobacco field and applying it to the word segmentation of project documents can improve the accuracy of word segmentation, thereby improving the accuracy of subsequent duplicate checking.

[0156] Since the number of scientific and technological papers in the field is much larger than the number of scientific and technological projects, and the scientific and technological papers in a specific field are closely related to scientific and technological projects, constructing a field dictionary based on papers can greatly expand the scale of the field dictionary. In one implementation manner of the present invention, the method for constructing a field dictionary may be to extract the keyword fields in Chinese and foreign papers related to tobacco, so as to construct a field dictionary. In another implementation manner of the present invention, it may also be to construct a field dictionary on the basis of extracting the keyword fields in Chinese and foreign papers related to tobacco and combining the national standards of tobacco term series. In yet another implementation manner of the present invention, it may also be to construct a field dictionary on the basis of extracting the keyword fields in Chinese and foreign papers related to tobacco, the terms in the national standards of tobacco term series, and combining tobacco-related books, such as "Tobacco Science and Technology Word System", etc.

[0157] Step S13: Construct a field stop word library according to the general vocabulary and meaningless words in the field.

[0158] Specifically, the words in the field stop word library may be general tobacco vocabulary (such as: tobacco, cigarette, etc.) that appear in various tobacco literatures, general vocabulary (such as: project objectives, research status, development trends, implementation plans, technical routes, etc.) in document templates such as application forms, technical reports, appraisal certificates, acceptance certificates, etc., and other meaningless words such as "of", "is", "in", etc.

[0159] It should be noted that the present invention does not specifically limit the order of execution of steps S11, S12, and S13.

[0160] Step S2: Calculate the inverse document frequency (IDF) of each word in the corpus composed of scientific and technological literatures in the tobacco field and materials of historical scientific and technological projects at least according to the field dictionary and the field stop word library, wherein the inverse document frequency of each word is used to characterize the general importance degree of the word.

[0161] Inverse Document Frequency (IDF) measures the general importance of a word by counting the number of documents containing that word in a corpus. The more documents containing a word, the less information that word contains. Since the number of science and technology projects is relatively small, and science and technology literature in a specific field is closely related to science and technology projects, calculating the IDF value based on field-specific science and technology projects and literature is more accurate in measuring the general importance of a word. In one embodiment of the field-oriented science and technology project plagiarism detection method of the present invention, step S2 may specifically include:

[0162] Step S21: Extract key fields from the corpus, wherein the scientific and technological documents in the corpus include at least one of Chinese and foreign language papers and patent documents; the key fields in Chinese and foreign language papers include at least one of title, abstract, and keyword fields; the key fields in patent documents include at least one of name and abstract fields; the historical scientific and technological projects in the corpus include application forms and technical reports; the key fields in application forms include at least one of project name and project objective fields; the key fields in technical reports include at least one of abstract, project objective, and conclusion fields.

[0163] In this invention, based on Chinese and foreign language papers, patents and other scientific and technological literature in the tobacco field, as well as application forms and technical reports of historical scientific and technological projects in the tobacco field, the title, abstract and keyword fields of the papers, the name and abstract fields of the patents, the project name and project objective fields of the project application forms, and the abstract, project objective and conclusion fields of the technical reports are extracted.

[0164] In some embodiments of the present invention, the key fields extracted may also include other fields from scientific and technological literature and historical scientific and technological project materials in the tobacco field, such as the full text of papers, the name of standards, the invention content of patents, the main research content, implementation schemes, and technical routes in project materials, etc., which are not limited here.

[0165] Step S22: At least based on the domain dictionary and the domain stop word library, use a preset word segmentation algorithm to segment the non-keyword fields in the corpus, wherein the non-keyword fields include fields other than the paper keywords.

[0166] In one embodiment of the field-oriented scientific and technological project plagiarism detection method of the present invention, step S22 may specifically include:

[0167] Step S221: Use the Conditional Random Field (CRF) algorithm to segment non-keyword fields in the corpus.

[0168] Conditional random field algorithms have a strong ability to identify out-of-vocabulary words, thus improving the accuracy of word segmentation.

[0169] Step S222: Based on domain characteristics, set the word length threshold to Len. max .

[0170] In one embodiment of the invention, Len max It can be set to 7, and Len will adjust according to changes in the technology project data. max Adjustments can be made, but this invention does not impose specific limitations on them.

[0171] Step S223: Filter out words whose length exceeds the word length threshold after word segmentation (Len). max The word.

[0172] Step S224: In the word segmentation results, the reverse maximum matching method is used to combine multiple adjacent words into a word based on the domain dictionary. If the word formed by combining multiple adjacent words exists in the domain dictionary, then the adjacent words are combined into a word.

[0173] If a combination of several adjacent words exists in the domain dictionary, meaning that the combined word has practical significance in the tobacco field, then adjacent words are combined into a single word. For example, the adjacent words "tobacco leaves" and "maintenance" are combined into "tobacco leaf maintenance," and the adjacent words "tobacco shreds" and "expansion" are combined into "tobacco shred expansion," etc.

[0174] Step S225: Based on the stop word library, remove the words in the stop word library after word segmentation to obtain the word segmentation result.

[0175] Step S23: Calculate the reverse document frequency of each word in the word segmentation results.

[0176] Specifically, based on the total number of documents in the corpus and the number of documents containing word i, the inverse document frequency of each word in the segmentation results is calculated using the following formula:

[0177]

[0178] Among them, IDF i Let represent the inverse document frequency of word i, D represent the total number of documents in the corpus, and d represent the number of documents containing word i.

[0179] Step S3: Based on the inverse document frequency of each word in the corpus, extract the keywords from the materials in the project plagiarism comparison database and the application database.

[0180] In one embodiment of the field-oriented scientific and technological project plagiarism detection method of the present invention, step S3 may specifically include:

[0181] Step S31: Extract keywords from the documents in the project plagiarism comparison database.

[0182] In one embodiment of the field-oriented scientific and technological project plagiarism detection method of the present invention, step S31 may specifically include:

[0183] Step S311: Using document structure parsing and field content extraction methods, extract the fields to be checked from each document in the project plagiarism comparison database.

[0184] The fields to be checked for plagiarism can be selected according to business needs. For example, they can include fields such as project name, project objectives, project summary, main research content, key technologies, innovative points, and implementation plan from the application form; project summary, technical route, objectives, and research results from the technical report; and achievement name, brief technical description, achieved technical indicators, and evaluation opinions from the appraisal certificate. No specific limitations are set here. Some project documents have fixed templates, such as application forms and appraisal certificates, while others do not, such as work reports and technical reports.

[0185] In one embodiment of the field-oriented scientific and technological project plagiarism detection method of the present invention, step S311 may specifically include:

[0186] Step S3111: For documents with fixed templates, parse each document in the project plagiarism comparison library according to the pre-configured field names to be extracted, find the corresponding keywords in the document through keyword matching, extract the text content corresponding to the current keyword and the next keyword, and use the extracted text content as the field to be checked for plagiarism.

[0187] For example, if the field to be extracted is set to "X. Project Objectives", then the corresponding keyword will be found in the document, and the content between "X. Project Objectives" and "XI. Main Contents of the Project" will be extracted.

[0188] Step S3112: For documents without a fixed format, based on the visual features of the text, parse each document in the project plagiarism comparison library, predict the title and corresponding body content of each chapter in the document, form a document directory structure, and match the field name to be extracted in the title of each chapter, extract the text content between the title and the next title, and use the extracted text content as the field to be checked for plagiarism.

[0189] The visual features of the text can include font size, thickness, paragraph features, etc. In the specific implementation, the visual features of the text are used to predict which are the titles of each chapter and which are the main text, forming a document directory structure. Then, the names of the fields to be extracted are matched in the identified titles to extract the text content between the titles.

[0190] Step S312: Use a preset word segmentation algorithm to segment the extracted fields to be checked for duplicates.

[0191] In one embodiment of the field-oriented scientific and technological project plagiarism detection method of the present invention, step S312 may specifically include:

[0192] Step S3121: Use the Conditional Random Field algorithm to segment non-keyword fields in the corpus.

[0193] Conditional random field algorithms have a strong ability to identify out-of-vocabulary words, thus improving the accuracy of word segmentation.

[0194] Step S3122: Based on domain characteristics, set the word length threshold to Len. max .

[0195] In one embodiment of the invention, Len max It can be set to 7, and Len will adjust according to changes in the technology project data. max Adjustments can be made, but this invention does not impose specific limitations on them.

[0196] Step S3123: Filter out words whose length exceeds the word length threshold after word segmentation. max The word.

[0197] Step S3124: In the word segmentation results, the reverse maximum matching method is adopted. Based on the domain dictionary, multiple adjacent words are combined into a word. If the word formed by combining multiple adjacent words exists in the domain dictionary, then the adjacent words are combined into a word.

[0198] If a combination of several adjacent words exists in the domain dictionary, meaning that the combined word has practical significance in the tobacco field, then adjacent words are combined into a single word. For example, the adjacent words "tobacco leaves" and "maintenance" are combined into "tobacco leaf maintenance," and the adjacent words "tobacco shreds" and "expansion" are combined into "tobacco shred expansion," etc.

[0199] Step S3125: Based on the stop word library, remove the words in the stop word library after word segmentation to obtain the word segmentation result.

[0200] Step S313: Extract keywords from the documents in the project plagiarism comparison database based on the word segmentation results.

[0201] In one embodiment of the field-oriented scientific and technological project plagiarism detection method of the present invention, step S313 may specifically include:

[0202] Step S3131: Based on the word segmentation results, select words that exist in the domain dictionary as candidate keywords.

[0203] Step S3132: Use a preset keyword extraction method to extract keywords from the documents in the project plagiarism comparison database from the candidate keywords.

[0204] The keyword extraction method can be based on term frequency–inverse document frequency (TF-IDF), where TF represents the frequency of a word in a document, and TF-IDF combines TF and IDF to evaluate the importance of a word to a document in a corpus. The TF-IDF value of each word is calculated based on the IDF value obtained using formula (1). In one embodiment of the field-oriented scientific and technological project plagiarism detection method of the present invention, step S3132 may specifically include:

[0205] Step S31321: Based on the number of times word i appears in document j and the total number of words in document j, calculate the word frequency of word i in document j using the following formula.

[0206]

[0207] Among them, TF ij n represents the frequency of word i in document j. i,j N represents the number of times word i appears in document j. j This represents the total number of words in document j.

[0208] Step S31322: Based on the term frequency of word i in document j and the inverse document frequency representing word i, calculate the term frequency - inverse document frequency of word i relative to document j using the following formula:

[0209] TFIDF ij =TF i,j *IDF i (3)

[0210] Among them, TFIDF i,j This represents the TF-IDF value of word i relative to document j, where IDF is the TF-IDF value. i Indicates the inverse document frequency of word i.

[0211] Step S31323: Extract the keywords of each document based on the word frequency-inverse document frequency of each word in each document.

[0212] Specifically, in one embodiment of the present invention, the N words with the highest word frequency-inverse document frequency in each document are extracted as the keywords of that document.

[0213] In another embodiment of the present invention, the keyword extraction method can be based on the calculated TF-IDF value, with weights applied to different parts of the document as needed. For example, different weights can be assigned to words appearing in different parts of the document, and the weighted sum of term frequency-inverse document frequency (TF-IDF) can be sorted in descending order. The N words with the highest TF-IDF in each document can be extracted as the keywords of that document. For example, words appearing in the project name can be assigned a first weight, words appearing in the project summary can be assigned a second weight, and words appearing in the project goal can be assigned a third weight. The magnitude of the weight can be set according to its importance to the project's plagiarism detection, and the present invention does not specifically limit this.

[0214] In another embodiment of this invention, an improved TextRank algorithm is used to extract keywords from the document. The initial weights of nodes and the transition probabilities between nodes in TextRank are optimized based on positional and semantic information. TextRank is a graph-based word importance ranking algorithm that extracts keywords using the co-occurrence relationships between words in a document. However, the extraction results are easily affected by high-frequency words in the document, leading to low accuracy. This invention introduces positional and semantic information to optimize the initial weights of nodes and the transition probabilities between nodes in TextRank. In one embodiment of the domain-oriented scientific and technological project plagiarism detection method of this invention, step S31323 may specifically include:

[0215] Step S313231: Retain the nouns, verbs, and adjectives from the word segmentation results to form a candidate keyword table. Using candidate keywords as nodes, if two words appear in the same sentence and co-occur within a sliding window of size 3, then establish an edge between the nodes to construct a network graph G = (V, E), where V = {v1, v2, ..., v...} n Let E be a set of nodes, where E = {e1, e2, ..., e} m Let} be the set of edges.

[0216] Step S313232: Based on the importance of content at different locations in the document, divide the document into first position, second position, and third position, with the importance of the first position, second position, and third position decreasing in that order.

[0217] Step S313233: Assign different weights to words appearing in different positions in the document using the following formula.

[0218]

[0219] Among them, Position it Let p1 represent the weight of the position of word i in the document at the t-th occurrence, and p1 > p2 > 1.

[0220] Considering that words appearing in different positions in the literature have different importance, in one embodiment of the present invention, different weights can be assigned to words in the first position such as project name or achievement name, topic name, title, abstract or introduction, and research content or main text according to formula (4). For example, in one embodiment of the present invention, p1 can be set to 5 and p2 can be set to 3.

[0221] Step S313234: Based on the weights of all occurrences of word i in the document, calculate the sum of the weights of each occurrence of word i in document j using the following formula.

[0222] P ij =∑ t Position it (5)

[0223] Among them, P ij This represents the sum of the positional weights of word i in document j for each occurrence.

[0224] Step S313235: Based on the word's position information, optimize the term frequency-inverse document frequency of word i relative to document j using the following formula, and use it as the starting weight of the node in TextRank.

[0225]

[0226]

[0227] TFIDF i,j =TF i,j *IDF i (8)

[0228] Among them, P tj TF represents the sum of the position weights of each occurrence of the t-th word in document j. ij Let IDF represent the frequency of word i in document j, D represent the total number of documents in the corpus, and d represent the number of documents containing word i. i The inverse document frequency of word i, TF-IDF i,j The TF-IDF value of word i relative to document j.

[0229] TF-IDF combines word frequency (TF) and word universal importance (IDF) in a document to evaluate the importance of a word to a document in a corpus. This invention integrates positional information to optimize TF-IDF and uses it as the starting weight of a node in TextRank.

[0230] Step S313236: Normalize the words that co-occur in the document using the following formula to obtain the word frequency features between words.

[0231]

[0232] Among them, F(w) x ,w y ) indicates the word w x and word w y The word frequency features between them, count(w x ,w y ) indicates the word w x and word w y The number of co-occurrences within a sliding window of a preset size, count(w) x ) indicates the word w x The number of times it appears in the document, count(w y ) indicates the word w y Number of times it appears in the document.

[0233] The more times two words co-occur in a document, the closer their relationship is. By normalizing the frequency of word co-occurrence, the influence of high-frequency words can be weakened.

[0234] Step S313237: Calculate the cosine similarity between the vectors corresponding to words that co-occur in the document using the following formula, as a semantic feature between words.

[0235]

[0236] Where X represents the word w x The corresponding word vector, Y represents word w y The corresponding word vector, m represents the dimension of the word vector, for example, it can be set to 128.

[0237] Word2vec is a word vector model where two words with similar contexts have similar word vector representations. This invention uses word2vec to obtain word vectors and then calculates the cosine similarity between the vectors.

[0238] Step S313238: Based on the word frequency and semantic features between words, calculate the weights of edges between nodes in TextRank using the following formula.

[0239]

[0240] Where w(x,y) represents node v x and node v y The weight of the edges between them.

[0241] This invention improves the transition probability between nodes in TextRank by combining word frequency features and semantic features.

[0242] Step S313239: Iteratively update the weights of each node according to the following formula (12) until convergence.

[0243]

[0244] Wherein, S(v x ) represents node v x The weights, N(v) x ) indicates the relationship with node v x There is a set of connected nodes, where d represents the damping coefficient, which characterizes the probability that the current node will jump to a connected node. For example, it can be set to 0.85.

[0245] Step S3132310: Sort the nodes according to their weights and select the top N as the keywords of the document.

[0246] Step S32: Extract keywords from the application materials of the project to be checked for plagiarism.

[0247] In one embodiment of the field-oriented scientific and technological project plagiarism detection method of the present invention, step S32 may specifically include:

[0248] Step S321: Use document structure parsing and field content extraction methods to extract the fields to be checked in the application form of the project to be checked for plagiarism.

[0249] Specifically, based on the pre-configured field names to be extracted, the document of the project application form to be checked for plagiarism is parsed, and the corresponding keywords are found in the document through keyword matching. The text content corresponding to the current keyword and the next keyword is extracted, and the extracted text content is used as the field to be checked for plagiarism in the project application form.

[0250] The fields to be checked for plagiarism in the project application form can be selected according to business needs, such as: project name, project objectives, project abstract, main research content, key technologies, innovative points, and implementation plan. No specific limitations are imposed here. In one embodiment of this invention, the document structure parsing and field content extraction method is as follows: based on manually configured field names to be extracted, document parsing is achieved through keyword matching. For example, if the field to be extracted is set to "X. Project Objectives", then the corresponding keywords are found in the document, and the content between "X. Project Objectives" and "XI. Main Contents of the Project" is extracted.

[0251] Step S322: Use a preset word segmentation algorithm to segment the fields to be checked for plagiarism in the extracted application form.

[0252] Specifically, the Conditional Random Field (CRF) algorithm, which has strong capabilities for identifying out-of-vocabulary words, can be used for word segmentation. Depending on the domain characteristics, a word length threshold of Len can be set.max After filtering out word segments, the length exceeds Len max The word. In this scheme, Len max Set to 7, Len will adjust his settings based on changes in the technology project data. max Adjustments are possible. For the word segmentation results, a reverse maximum matching method is used, based on a domain dictionary. If a combination of multiple adjacent words exists in the domain dictionary (i.e., has practical significance in the tobacco field), then the adjacent words are combined into one word. For example, the adjacent words "tobacco leaves" and "maintenance" are combined into "tobacco leaf maintenance," and the adjacent words "tobacco shreds" and "expansion" are combined into "tobacco shred expansion," etc. Subsequently, based on a stop word library, words that appear in the stop word library after segmentation are removed to obtain the word segmentation results. Based on the word segmentation results, words in the domain dictionary are selected as candidate keywords.

[0253] Step S323: Extract keywords from the documents in the application materials of the project to be checked for plagiarism based on the word segmentation results.

[0254] Among them, keyword extraction methods are used to extract keywords for the items to be checked for plagiarism. Keyword extraction methods can be based on term frequency–inverse document frequency (TF-IDF). Based on the previously obtained IDF values, the TF-IDF values ​​of words are obtained according to formulas (2) and (3).

[0255] Extract the N words with the highest TF-IDF values ​​from each document as the keywords for that document.

[0256] Alternatively, based on the calculated TF-IDF value, different parts of the document can be weighted as needed. For example, words appearing in the project name can be assigned the first weight, words appearing in the project summary the second weight, and words appearing in the project objectives the third weight. The weight can be set according to their importance to the project's plagiarism detection. The weighted TF-IDF values ​​are then sorted from high to low, and the N words with the highest TF-IDF values ​​in each document are extracted as the keywords for that document.

[0257] Keyword extraction can also be performed using a modified TextRank algorithm. The specific method is the same as that used to extract keywords from documents in the project plagiarism comparison database, and will not be repeated here.

[0258] Step S4: Calculate the duplication rate between the item to be checked and the items in the duplication comparison database based on the extracted keywords, and determine the highly similar items based on the duplication rate calculation results.

[0259] The application form of the project to be checked for plagiarism is compared with documents in the project plagiarism comparison database to calculate the document similarity rate. The results are sorted from high to low similarity rate. Based on the set similarity threshold, projects with a similarity rate greater than or equal to the similarity threshold are selected and judged as highly similar projects, which are the results of the project plagiarism check. The plagiarism detection method can be based on keyword matching. In one embodiment of the field-oriented scientific and technological project plagiarism detection method of the present invention, step S4 may specifically include:

[0260] Step S41: Compare the application form of the project to be checked with the documents in the project plagiarism comparison database, and calculate the document duplication rate.

[0261] In one embodiment of the field-oriented scientific and technological project plagiarism detection method of the present invention, step S41 may specifically include:

[0262] Step S411: Let A be the r-th field to be checked extracted from the application form of the project S to be checked for plagiarism. r The fth item in the project plagiarism comparison database is the project material from a historical project Pro. f The g-th field to be checked for plagiarism is Pro fg .

[0263] The fields to be checked for plagiarism, extracted from the application form of project S, can include, for example, project objectives and project summary. The project materials from historical projects in the plagiarism comparison database can include application forms, implementation plans, work reports, technical reports, appraisal certificates, acceptance certificates, third-party evaluation reports, and other completion documents required by fields such as tobacco breeding, tobacco cultivation, tobacco plant protection, tobacco curing, tobacco technology, tobacco chemistry, tobacco product design, tobacco quality supervision, tobacco informatization, and tobacco metrology.

[0264] Step S412, if A r With Pro fg If fields belong to the same type, then according to the first preset keyword matching rule, A will be matched sequentially. r Each keyword in Pro fg The keywords in the text are matched, and if a match is found, the Pro will be... fg With A r count(pro) fg Add 1, and calculate Pro using the following formula. fg With A r The repetition rate

[0265]

[0266] Among them, RF (Pro fg ) indicates Pro fg With A rThe repetition rate, count(pro) fg ) indicates Pro fg In and A r The number of matching keywords, Keywords(pro) fg ) indicates Pro fg The total number of keywords in Pro fg If there are no keywords in the text, then RF (Pro) fg If A is 0, the first preset keyword matching rule is: if A r The keyword in Pro is equal to f Keywords or containing Pro f Keywords in the text are considered to match the matching rules.

[0267] A r With Pro fg The meaning of fields belonging to the same type can be understood as follows: the fields describing the project name in the application form and the topic name in the application form at different stages, the project name in the technical report, the achievement name in the appraisal certificate, etc. are the same type of fields; the fields describing the project research objectives in the application form and the project objectives in the application form at different stages, the overall project objectives in the work report, the overall project objectives in the technical report, etc. are the same type of fields.

[0268] Step S413, if A r With Pro fg Fields that do not belong to the same type will not have their duplication rate calculated.

[0269] Step S414: For each field to be checked in the application form of the project to be checked, compare it with the f-th project material Pro in the project comparison database. f Perform the above operation on each field to be checked for plagiarism in the application form S, and obtain the results of checking each field to be checked for plagiarism in the application form S and Pro. f The repetition rate of each field of the same type to be checked for duplicates, let Pro f If the number of fields of the same type to be checked for duplication in Project S's application is M, then Pro f The similarity rate with the item S to be checked is taken as Pro f The maximum repetition rate of all fields to be checked for duplicates, expressed by the following formula:

[0270] RF (Pro fg ) = max(RF(Pro f1 ),RF(Pro f2 ),……,RF(Pro fM (14).

[0271] In another embodiment of the field-oriented scientific and technological project plagiarism detection method of the present invention, step S41 may specifically include:

[0272] Step S411': Pre-set the weight of each field to be checked for plagiarism, and calculate the weighted average of the repetition rates of each field to obtain the f-th project material Pro from a historical project Pro in the project plagiarism comparison database. f The similarity rate with the item S to be checked for plagiarism is expressed by the following formula:

[0273] RF (Pro f )=∑ f,g RF(pro fg )·w c (15)

[0274] Among them, w c Indicates Pro f The weight of the c-th field to be checked for plagiarism.

[0275] Step S412': Let D be the number of project materials for historical project Pro in the project duplication comparison database. Then, the duplication rate between project Pro and the project to be checked S is taken as the maximum value of the duplication rates between all project materials of project P and the project to be checked S.

[0276] RF(Pro)=max(RF(Pro1),RF(Pro2),…,RF(Pro D )) (16).

[0277] In another embodiment of the field-oriented scientific and technological project plagiarism detection method of the present invention, step S41 may specifically include:

[0278] Step S411”, let A be the r-th field to be checked extracted from the application form of the project S to be checked. r The fth item in the project plagiarism comparison database is the project material from a historical project Pro. f The g-th field to be checked for plagiarism is Pro fg If A r With Pro fg If fields belong to the same type, then according to the second preset keyword matching rule, A will be matched sequentially. r Each keyword in Pro fg The keywords in the text are matched, and if a match is found, the Pro will be... fg With A r degree of matching fg Add this keyword to the fth item in the Pro historical project list. f The weight value in Pro is the term frequency-inverse document frequency value, calculated using the following formula:f With A r The repetition rate

[0279] degree(pro fg )=∑ h∈H weight(h) (17)

[0280]

[0281] Where H represents Pro fg With A r The set of matched keywords, weight(h) represents the h-th keyword in the f-th material of historical item Pro. f The weights in the degree(Pro) fg ) indicates Pro fg With A r The degree of matching, K represents Pro fg All keywords in the set, the matching rule for the second keyword is, if A r The keyword in Pro is equal to f Keywords or containing Pro f Keywords in the text are considered to match the matching rules.

[0282] The fields to be checked for plagiarism can include, for example, project objectives and project abstracts. Project materials can include application forms, implementation plans, work reports, technical reports, appraisal certificates, acceptance certificates, third-party evaluation reports, and other completion documents required by the fields of tobacco breeding, tobacco cultivation, tobacco plant protection, tobacco curing, tobacco technology, tobacco chemistry, tobacco product design, tobacco quality supervision, tobacco informatization, and tobacco metrology.

[0283] A r With Pro fg Fields belonging to the same category can be understood as follows: fields describing project names in the application form, project names in different stages of the application, project names in technical reports, and achievement names in appraisal certificates are considered to be of the same category; similarly, fields describing project research objectives in the application form, project objectives in different stages of the application, overall project objectives in work reports, and overall project objectives in technical reports are considered to be of the same category.

[0284] Step S412”, if A r With Pro fg Fields that do not belong to the same type will not have their duplication rate calculated.

[0285] Step S413”, each field to be checked in the application form of the project to be checked and the f-th material Pro of project P in the project plagiarism comparison database. fPerform the above operation on each field to be checked for plagiarism in the application form S, and obtain the results of checking each field to be checked for plagiarism in the application form S and Pro. f The repetition rate of each field of the same type to be checked for duplicates, let Pro f If the number of fields of the same type to be checked for duplication in Project S's application is M, then Pro f The similarity rate with the item S to be checked is taken as Pro f The maximum repetition rate of all fields to be checked for duplicates, expressed by the following formula:

[0286] RF (Pro f ) = max(RF(Pro f1 ),RF(Pro f2 ),……,RF(Pro fM )) (19).

[0287] In another embodiment of the field-oriented scientific and technological project plagiarism detection method of the present invention, step S41 may specifically include:

[0288] Step S411”: Pre-set the weight of each field to be checked for plagiarism, and obtain Pro by weighted average of the repetition rates of each field to be checked for plagiarism. f The similarity rate with the item S to be checked for plagiarism is expressed by the following formula:

[0289] RF(Prof f )=∑ f,g RF(pro fg )·w c (20)

[0290] Among them, w c Indicates Pro f The weight of the c-th field to be checked for plagiarism.

[0291] Step S412”': Let D be the number of project materials for historical project Pro in the project duplication comparison database. Then, the duplication rate between project P and the project to be checked for duplication S is the maximum value of the duplication rates of all materials of Pro and S, and is expressed by the following formula:

[0292] RF(Pro)=max(RF(Pro1),RF(Pro2),…,RF(Pro D )) (twenty one).

[0293] In one embodiment of the present invention, the plagiarism detection method can also be based on text similarity. In another embodiment of the field-specific scientific and technological project plagiarism detection method of the present invention, the document duplication rate is calculated based on text similarity, and step S41 may specifically include:

[0294] Step S411””: Use word embedding method to train word vectors in all documents in the project plagiarism comparison library to obtain the vector representation of each word.

[0295] Specifically, based on all documents in the project plagiarism comparison database, word vectors are trained using word embedding methods such as word2vec, glove, and bert to obtain the vector representation corresponding to each word.

[0296] Step S412””: Extract the content of all fields to be checked in the application form of the project to be checked S, and filter out sentences containing keywords to form a set of key sentences for the project to be checked S.

[0297] Step S413””, Extract the f-th project material Pro from the historical project Pro in the project duplication comparison database. f The content of all fields to be checked for plagiarism is filtered to form sentences containing keywords. f A collection of key sentences.

[0298] The project materials may include application forms, implementation plans, work reports, technical reports, appraisal certificates, acceptance certificates, third-party evaluation reports, and other completion documents required by the fields of tobacco breeding, tobacco cultivation, tobacco plant protection, tobacco curing, tobacco technology, tobacco chemistry, tobacco product design, tobacco quality supervision, tobacco informatization, and tobacco metrology.

[0299] Step S414””, calculate the set of key sentences for the project S to be checked for plagiarism and the Pro selected from the f-th project material of the historical project Pro in the project plagiarism comparison database. f The similarity of key sentence pairs containing the same keywords in the key sentence set.

[0300] In another embodiment of the field-oriented scientific and technological project plagiarism detection method of the present invention, the document duplication rate is calculated based on the text similarity method, and step S414" specifically may include:

[0301] Step S4141: Based on the word vector of each word in the key sentence, calculate the average weighted by word frequency minus inverse document frequency to obtain the vector of the key sentence, which is expressed by the following formula:

[0302]

[0303] Where m represents the number of words in the key sentence, vec d This represents the word vector of the d-th word in the key sentence, and weight(d) represents the weight of the d-th word, which is the TF-IDF value of the d-th word relative to the current document.

[0304] Step S4142””, Select the set of key sentences for the item to be checked for plagiarism S and the f-th item material Pro from the historical item Pro in the plagiarism comparison database. f The set of key sentences contains key sentences with the same keywords. The cosine similarity of key sentence pairs is calculated using the following formula, where the cosine similarity of the key sentence pairs is used to characterize the similarity between sentence pairs:

[0305]

[0306] Where s1 represents the sentence vector of the key sentence in the application form of the project S to be checked for plagiarism, and s2 represents the f-th project material Pro from the historical project Pro in the project plagiarism comparison database. f The sentence vectors of the key sentences in the text, sim(s1,s2) represent the similarity between sentence vectors s1 and s2.

[0307] Step S4143””, take the average similarity of each sentence pair as Pro f The similarity rate with the item S to be checked for plagiarism.

[0308] Step S4144”: Let D be the number of materials in the historical project Pro in the project duplication comparison database. Then, the duplication rate between project Pro and the project to be duplicated S is the maximum value of the duplication rates of all project materials of Pro and the project to be duplicated S, and is expressed by the following formula:

[0309] RF(Pro)=max(RF(Pro1),RF(Pro2),…,RF(Pro D )) (twenty four).

[0310] Step S42: Sort the calculation results in descending order of repetition rate;

[0311] Step S43: Filter out items with a duplication rate greater than or equal to the preset similarity threshold, identify them as highly similar items, and use them as the result of the item duplication check.

[0312] The domain-oriented technology project plagiarism detection method provided in this invention calculates the inverse document frequency of each word in the corpus based on a domain dictionary and a domain stop word database. It then extracts keywords from materials in the project plagiarism comparison database and the application database. Based on these keywords, it calculates the repetition rate between the project to be checked and projects in the comparison database to identify highly similar projects. This method helps enterprise technology managers compare the documents to be checked with a large amount of historical project information to find projects with highly similar research content, thus avoiding the waste of enterprise technology resources due to duplicate project approvals and ensuring the fairness of technology project approvals.

[0313] Accordingly, such as Figure 2 As shown, the present invention also provides a field-specific technology project plagiarism detection system, comprising:

[0314] Database construction module 1 is used to build the project plagiarism comparison database, the domain dictionary, and the domain stop word database, respectively.

[0315] The reverse document frequency calculation module 2 is used to calculate the reverse document frequency of each word in a corpus composed of scientific and technological literature and historical scientific and technological projects in the tobacco field, based at least on the domain dictionary and the domain stop word database. The reverse document frequency of each word is used to characterize the general importance of that word.

[0316] Keyword extraction module 3 is used to extract keywords from the materials in the project plagiarism comparison database and the application database based on the inverse document frequency of each word in the corpus.

[0317] The duplication rate calculation module 4 is used to calculate the duplication rate between the item to be checked and the items in the duplication comparison database based on the extracted keywords, and to determine the highly similar items based on the duplication rate calculation results.

[0318] Furthermore, in some embodiments of the present invention, a field-oriented scientific and technological project plagiarism detection system further includes an application form upload module, a similarity threshold setting module, a keyword management module, a project data viewing module, and a plagiarism detection result export module, wherein,

[0319] The application form upload module is used to upload application forms. Users can select local application forms and data types to upload, and click the confirmation button to proceed to the next step. In specific implementations, it can support batch uploading of application forms via compressed packages.

[0320] The similarity threshold setting module is used to set the similarity threshold to a value between 0 and 1. After setting, the system will filter items with a duplication rate greater than or equal to the similarity threshold as the duplication detection result.

[0321] The keyword management module is used to modify the document keywords extracted by the system. Specifically, it includes: modifying the weight of keywords, deleting keywords that were extracted incorrectly, and adding new keywords and their weights. The system will calculate the repetition rate based on the keywords and their weights modified by the user.

[0322] The project information viewing module is used to view project information. Users can view application forms, technical reports, and other materials for similar projects in the system.

[0323] The plagiarism detection result export module is used to export plagiarism detection results. For example, the first part of the exported content includes the current project application information (number, project name, project category, undertaking unit, collaborating unit, application date), and the second part displays a list of similar projects (project category, project name, undertaking unit, collaborating unit, application date, project start and end dates). This invention does not specifically limit the specific exported results and export format.

[0324] The domain-oriented technology project plagiarism detection system provided in this invention calculates the inverse document frequency of each word in the corpus based on a domain dictionary and a domain stop word database. It then extracts keywords from materials in the project plagiarism comparison database and the application database. Based on these keywords, it calculates the duplication rate between the project to be checked and projects in the comparison database to identify highly similar projects. This system helps enterprise technology managers compare the documents to be checked with a large amount of historical project information to find projects with highly similar research content. It can avoid the waste of enterprise technology resources caused by duplicate project approvals and ensure the fairness of technology project approvals.

[0325] The embodiments of this disclosure have now been described in detail. To avoid obscuring the concept of this disclosure, some details known in the art have not been described. Those skilled in the art can fully understand how to implement the technical solutions disclosed herein based on the above description.

[0326] While specific embodiments of this disclosure have been described in detail by way of examples, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of this disclosure. Those skilled in the art should understand that modifications can be made to the above embodiments or equivalent substitutions can be made to some technical features without departing from the scope and spirit of this disclosure. The scope of this disclosure is defined by the appended claims.

Claims

1. A method for detecting plagiarism in domain-specific scientific and technological projects, characterized in that, include: Construct a project plagiarism comparison database, a domain dictionary, and a domain stop word database respectively; Based at least on the domain dictionary and the domain stop word database, the inverse document frequency of each word in a corpus consisting of scientific and technological literature and historical scientific and technological projects in the tobacco field is calculated, wherein the inverse document frequency of each word is used to characterize the general importance of the word. Based on the inverse document frequency of each word in the corpus, keywords are extracted from the materials in the project plagiarism comparison database and the application database. The duplication rate between the item to be checked and the items in the duplication comparison database is calculated based on the extracted keywords, and highly similar items are determined based on the duplication rate calculation results. The step of extracting keywords from the project plagiarism comparison database and the application database based on the inverse document frequency of each word in the corpus specifically includes: Extract keywords from documents in the project's plagiarism comparison database; Extract keywords from the application materials of the project to be checked for plagiarism. The extraction of keywords from the application materials of the project to be checked for plagiarism specifically includes: The document structure parsing and field content extraction method is used to extract the fields to be checked for plagiarism in the application form of the project to be checked; The pre-defined word segmentation algorithm is used to segment the fields to be checked for plagiarism in the application form of the project to be checked. Keywords are extracted from the documents in the application materials of the project to be checked for plagiarism based on the word segmentation results. The extraction of keywords from documents in the project's plagiarism comparison database specifically includes: The document structure parsing and field content extraction method is used to extract the fields to be checked from each document in the project deduplication comparison library; The extracted fields to be checked for plagiarism are segmented using a preset word segmentation algorithm; Extract keywords from documents in the project's plagiarism comparison database based on the word segmentation results. The extraction of keywords from documents in the project plagiarism comparison database based on word segmentation results specifically includes: Based on the word segmentation results, words existing in the domain dictionary are selected as candidate keywords; Keywords from documents in the project plagiarism comparison database are extracted using a preset keyword extraction method from the candidate keywords. The method of extracting keywords from the candidate keywords to extract documents in the project plagiarism comparison database using a preset keyword extraction method specifically includes: The word frequency of word i in document j is calculated using the following formula, based on the number of times word i appears in document j and the total number of words in document j. （1） in, This indicates the frequency of word i in document j. This indicates the number of times word i appears in document j. This represents the total number of words in document j. Based on the term frequency of word i in document j and the inverse document frequency of word i, the term frequency - inverse document frequency of word i relative to document j is calculated using the following formula: （2） in, This represents the TF-IDF value of word i relative to document j. Indicates the inverse document frequency of word i; Extract keywords from each document based on the term frequency-inverse document frequency of each word. The step of extracting keywords from a document based on the term frequency-inverse document frequency of each word in the document specifically includes: extracting keywords from the document using an improved TextRank algorithm, and optimizing the initial weights of nodes and the transition probabilities between nodes in TextRank based on positional and semantic information. An improved TextRank algorithm is used to extract keywords from the document. The initial weights of nodes and the transition probabilities between nodes in TextRank are optimized based on positional and semantic information. Specifically, this includes: The nouns, verbs, and adjectives in the word segmentation results are retained to form a candidate keyword list. Using candidate keywords as nodes, if two words appear in the same sentence and co-occur within a sliding window of size 3, an edge is created between the nodes, constructing a network graph G=(V,E), where V={ } is a set of nodes, E={ } is the set of edges; Based on the importance of content at different locations within the document, the document is divided into first, second, and third positions, with the importance decreasing sequentially from first to third position. The following formula assigns different weights to words appearing in different positions in a document. （3） in, Let represent the weight of the position of word i at the t-th occurrence in the document, and ; Based on the weights of all occurrences of word i in the document, the sum of the weights of each occurrence of word i in document j is calculated using the following formula. （4） in, This represents the sum of the positional weights of word i in document j for each occurrence; Based on the word's positional information, the term frequency-inverse document frequency of word i relative to document j is optimized using the following formula, and used as the starting weight for nodes in TextRank. （5） （6） （7） in, This represents the sum of the position weights of the t-th word in document j for each occurrence. Let represent the frequency of word i in document j, D represent the total number of documents in the corpus, and d represent the number of documents containing word i. Indicates the inverse document frequency of word i. This represents the TF-IDF value of word i relative to document j; The following formula is used to normalize words that co-occur in a document, serving as a term frequency feature among them. （8） in, Indicator Harmony Word frequency features between Indicator Harmony The number of co-occurrences within a preset-size sliding window Indicator Number of times it appears in the document Indicator Number of times it appears in the document; The cosine similarity between vectors corresponding to words that co-occur in a document is calculated using the following formula, and used as a semantic feature between words. （9） Where X represents a word The corresponding word vector, Y represents the word The corresponding word vectors, where m represents the dimension of the word vectors; Based on word frequency and semantic features, the weights of edges between nodes in TextRank are calculated using the following formula. （10） in, Represents a node and nodes The weight of the edges between them; The weights of each node are iteratively updated according to the following formula (11) until convergence. （11） in, Represents a node The weights, Represents nodes There exists a set of connected nodes. This represents the damping coefficient, which characterizes the probability of the current node jumping to a connected node; Sort the nodes according to their weights and select the top N as the keywords for the document.

2. The domain-oriented scientific and technological project plagiarism detection method according to claim 1, characterized in that, The construction of the project plagiarism comparison library, the domain dictionary, and the domain stop word library specifically includes: Based on the application materials of completed projects, the completion materials, and the application materials of ongoing projects, a project plagiarism comparison database was constructed. Constructing domain-specific dictionaries based on academic papers; Construct a domain-specific stop word library based on domain-general vocabulary and words without substantive meaning. The step of calculating the inverse document frequency of each word in a corpus composed of materials from scientific and technological literature and historical scientific and technological projects in the tobacco field, based at least on the domain dictionary and the domain stop word database, specifically includes: Key fields are extracted from the corpus, wherein the scientific and technological documents in the corpus include at least one of Chinese and foreign language papers and patent documents; the key fields in Chinese and foreign language papers include at least one of title, abstract, and keyword fields; the key fields in patent documents include at least one of name and abstract fields; the historical scientific and technological projects in the corpus include application forms and technical reports; the key fields in application forms include at least one of project name and project objective fields; and the key fields in technical reports include at least one of abstract, project objective, and conclusion fields. At least based on the domain dictionary and the domain stop word library, a preset word segmentation algorithm is used to segment non-keyword fields in the corpus, wherein the non-keyword fields include fields other than the paper keywords; Calculate the inverse document frequency of each word in the word segmentation results.

3. The domain-oriented scientific and technological project plagiarism detection method according to claim 2, characterized in that, The step involves segmenting non-keyword fields in the corpus using a preset word segmentation algorithm based at least on the domain dictionary and the domain stop word database. The non-keyword fields include fields other than the paper's keywords, specifically including: The Conditional Random Field algorithm is used to segment non-keyword fields in the corpus; Based on domain characteristics, the word length threshold is set to... ; After filtering out words whose length exceeds the word length threshold, the filtering process is complete. The word; In the word segmentation results, the reverse maximum matching method is adopted. Based on the domain dictionary, multiple adjacent words are combined into a word. If the word formed by combining multiple adjacent words exists in the domain dictionary, then the adjacent words are combined into a word. Based on the stop word library, words in the stop word library after word segmentation are removed to obtain the word segmentation result. The calculation of the inverse document frequency of each word in the word segmentation result specifically includes: Based on the total number of documents in the corpus and the number of documents containing word i, the inverse document frequency of each word in the segmentation results is calculated using the following formula. （12） in, Let represent the inverse document frequency of word i, D represent the total number of documents in the corpus, and d represent the number of documents containing word i.

4. The domain-oriented scientific and technological project plagiarism detection method according to claim 1, characterized in that, The method employs document structure parsing and field content extraction to extract the fields to be checked from each document in the project plagiarism comparison database, specifically including: For documents with fixed templates, each document in the project plagiarism comparison library is parsed according to the pre-configured field names to be extracted. By keyword matching, the corresponding keywords are found in the document, and the text content corresponding to the current keyword and the next keyword is extracted. The extracted text content is used as the field to be checked for plagiarism. For documents without a fixed format, based on the visual features of the text, the system parses each document in the project plagiarism comparison database, predicts the titles and corresponding body text of each chapter, forms a document directory structure, and matches the names of fields to be extracted in the identified chapter titles. The text content between the first and next title is then extracted and used as the field to be checked for plagiarism. The method employs document structure parsing and field content extraction to extract the fields to be checked in the application form of the project to be checked for plagiarism, specifically including: Based on the pre-configured field names to be extracted, the document of the project application form to be checked for plagiarism is parsed. Through keyword matching, the corresponding keywords are found in the document, and the text content corresponding to the current keyword and the next keyword is extracted. The extracted text content is used as the field to be checked for plagiarism in the project application form.

5. The domain-oriented scientific and technological project plagiarism detection method according to claim 1, characterized in that, The step of segmenting the extracted fields to be checked for plagiarism using a preset word segmentation algorithm specifically includes: The Conditional Random Field algorithm is used to segment non-keyword fields in the corpus; Based on domain characteristics, the word length threshold is set to... ; After filtering out words whose length exceeds the word length threshold, the filtering process is complete. The word; In the word segmentation results, the reverse maximum matching method is adopted. Based on the domain dictionary, multiple adjacent words are combined into a word. If the word formed by combining multiple adjacent words exists in the domain dictionary, then the adjacent words are combined into a word. Based on the stop word library, words in the stop word library are removed after word segmentation to obtain the word segmentation result.

6. The domain-oriented scientific and technological project plagiarism detection method according to claim 1, characterized in that, The step of extracting keywords from a document based on the term frequency-inverse document frequency of each word in the document specifically includes: Extract the N words with the highest term frequency-inverse document frequency from each document as the keywords for that document. Alternatively, different weights can be assigned to words appearing in different parts of the document, and the weighted sum of term frequency-inverse document frequency can be sorted in descending order. The N words with the highest term frequency-inverse document frequency in each document can be extracted as the keywords of that document.

7. The domain-oriented scientific and technological project plagiarism detection method according to claim 1, characterized in that, The process of calculating the similarity rate between the item to be checked and the items in the plagiarism comparison database based on the extracted keywords, and determining highly similar items based on the calculation results of the similarity rate, specifically includes: The application form of the project to be checked for plagiarism is compared with the documents in the project plagiarism comparison database, and the document duplication rate is calculated. Sort the calculation results in descending order of repetition rate; Items with a duplication rate greater than or equal to a preset similarity threshold are selected as highly similar items and used as the result of the duplication check. The process of comparing the application form of the project to be checked with documents in the project plagiarism comparison database and calculating the document duplication rate specifically includes: Let the r-th field to be checked for plagiarism be extracted from the application form of the project S to be checked. The fth project material from a historical project Pro in the project plagiarism comparison database. The g-th field to be checked for duplicates is ; like and Fields of the same type will be matched sequentially according to the first preset keyword matching rule. Each keyword in The keywords in the text are matched, and if a match is found, then... and Match count Add 1, and calculate using the following formula. and The repetition rate （13） in, express and The repetition rate express Zhongyu The number of matching keywords, express The total number of keywords in the text, if If there are no keywords, then If the value is 0, the first preset keyword matching rule is: if The keywords in the text are equal to Keywords or contents If the keywords in the text are not found, they are considered to match the rules. like and Fields that do not belong to the same type will not have their duplication rate calculated. For each field to be checked in the application form of project S, and the f-th project material of project Pro in the project plagiarism comparison database. Perform the above operation on each field to be checked for plagiarism in the declaration form of the project S to obtain the results. The repetition rate of each field of the same type to be checked for duplicates, let's assume... If the number of fields of the same type to be checked for duplication in the application form of project S is M, then... The similarity rate with the item S to be checked is taken as The maximum repetition rate of all fields to be checked for duplicates, expressed by the following formula: （14）， Alternatively, the weights of each field to be checked for duplication can be pre-set, and the duplication rate of each field can be weighted and averaged to obtain the f-th project material of a historical project Pro in the project duplication comparison database. The similarity rate with the item S to be checked for plagiarism is expressed by the following formula: （15） in, express The weight of the c-th field to be checked for duplicates; Let D be the number of project materials for historical project Pro in the project plagiarism comparison database. Then, the duplication rate between project Pro and the project to be checked for duplication S is the maximum value of the duplication rates of all project materials of project P and the project to be checked for duplication S: （16）， Alternatively, let the r-th field to be checked for plagiarism be extracted from the application form of the project S to be checked. The fth project material from a historical project Pro in the project plagiarism comparison database. The g-th field to be checked for duplicates is ,like and Fields belonging to the same type will be matched sequentially according to the second preset keyword matching rule. Each keyword in The keywords in the text are matched, and if a match is found, then... and degree of matching Add this keyword to the fth project material in the historical project Pro. The weight value in the text is the term frequency-inverse document frequency value, calculated using the following formula. and The repetition rate （17） （18） Where H represents and The set of matched keywords, This indicates that the h-th keyword appears in the f-th material of the historical project Pro. The weights in express and The degree of matching, K represents All keywords in the second keyword, the matching rule for the second keyword is, if The keywords in the text are equal to Keywords or contents If the keywords in the text are not found, they are considered to match the rules. like and Fields that do not belong to the same type will not have their duplication rate calculated. For each field to be checked in the application form of project S and the f-th material of project P in the project plagiarism comparison database, Perform the above operation on each field to be checked for plagiarism in the declaration form of the project S to obtain the results. The repetition rate of each field of the same type to be checked for duplicates, let's assume... If the number of fields of the same type to be checked for duplication in the application form of project S is M, then... The similarity rate with the item S to be checked is taken as The maximum repetition rate of all fields to be checked for duplicates, expressed by the following formula: （19）， Alternatively, the weight of each field to be checked for plagiarism can be preset, and the weighted average of the repetition rates of each field can be obtained. The similarity rate with the item S to be checked for plagiarism is expressed by the following formula: （20） in, express The weight of the c-th field to be deduplicated. Let D be the number of project materials for historical project Pro in the project plagiarism comparison database. Then, the duplication rate between project P and the project S to be checked is the maximum value of the duplication rates of all materials of Pro and S, and is expressed by the following formula: （21）， Alternatively, text similarity methods can be used to calculate document duplication rates, specifically including: The word embedding method is used to train word vectors in all documents in the project plagiarism comparison database to obtain the vector representation of each word. Extract the content of all fields to be checked in the application form of the project S to be checked for plagiarism, and filter out sentences containing keywords to form a set of key sentences for the project S to be checked for plagiarism. Extract the f-th project material from the historical project Pro in the project plagiarism comparison database. The content of all fields to be checked for plagiarism is filtered to form sentences containing keywords. A collection of key sentences; Calculate the set of key sentences for the project S to be checked for plagiarism and the selected key sentences from the f-th project material of the historical project Pro in the project plagiarism comparison database. The similarity of key sentence pairs containing the same keywords in the key sentence set includes: The vector of the key sentence is obtained by averaging the word vectors of each word in the key sentence using a weighted average of word frequency and inverse document frequency, and is expressed by the following formula: （22） Where m represents the number of words in the key sentence, This represents the word vector of the d-th word in the key sentence. The weight of the d-th word is represented by its TF-IDF value relative to the current document. Select the set of key sentences from the project S to be checked for plagiarism and the f-th project material from the historical project Pro in the plagiarism comparison database. The set of key sentences contains key sentences with the same keywords. The cosine similarity of key sentence pairs is calculated using the following formula, where the cosine similarity of the key sentence pairs is used to characterize the similarity between sentence pairs: （23） in, This represents the sentence vector of the key sentence in the application document of the project S to be checked for plagiarism. This refers to the f-th item in the historical project Pro database used for plagiarism detection. The sentence vector of the key sentence in the text. Sentence vector With sentence vectors Similarity between them; The average similarity of each sentence pair is used as... The rate of overlap with the item S to be checked for plagiarism; Let D be the number of materials in the historical project Pro in the project plagiarism comparison database. Then, the duplication rate between project Pro and the project to be checked S is the maximum value of the duplication rates of all materials of Pro and the project to be checked S, and is expressed by the following formula: （24）。 8. A field-oriented scientific and technological project plagiarism detection system employing the method described in any one of claims 1-7, characterized in that, include: The database construction module is used to build the project plagiarism comparison database, the domain dictionary, and the domain stop word database, respectively. The inverse document frequency calculation module is used to calculate the inverse document frequency of each word in a corpus consisting of scientific and technological literature and historical scientific and technological projects in the tobacco field, based at least on the domain dictionary and the domain stop word database. The inverse document frequency of each word is used to characterize the general importance of that word. The keyword extraction module is used to extract keywords from the materials in the project plagiarism comparison database and the application database based on the inverse document frequency of each word in the corpus. The duplication rate calculation module is used to calculate the duplication rate between the item to be checked and the items in the duplication comparison database based on the extracted keywords, and to determine the highly similar items based on the duplication rate calculation results.

Images