Geological report duplicate checking method and device

Abstract

The invention relates to a geological report duplicate checking method and device, and belongs to the technical field of text duplicate checking. The method comprises the following steps: firstly, constructing a specimen library containing geological report structure analysis, professional term standardization and keyword extraction; after similar preprocessing is carried out on the to-be-duplicate-checked report, a sample thematic database is established according to the geological work type, degree and region; performing content comparison through a multi-stage hybrid algorithm, wherein the content comparison comprises Jaccard coarse screening, fixed-length text segment Hash hit operation and TF-IDF cosine similarity actuarial calculation; and finally, calculating a repetition rate by adopting a weighting formula based on content importance, and performing threshold screening and special field ablation to optimize a result. The device correspondingly comprises a specimen library construction module, a preprocessing module, a thematic library management module, a similarity calculation module, a result optimization module and a report generation module. According to the method, the problems of geological terminology misjudgment, poor algorithm adaptability and inflexible threshold value are effectively solved, and the duplicate checking accuracy, efficiency and result scientificity are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of text plagiarism detection technology, specifically to a method and apparatus for plagiarism detection in geological reports. Background Technology

[0002] With the continuous advancement of academic integrity construction, text plagiarism detection technology has been widely applied in various fields. Currently, the mainstream plagiarism detection technologies mainly include: Text fingerprinting technology: This uses a sliding window algorithm to generate text fingerprints, mapping text fragments to unique identifiers through a hash function, effectively handling cases of word order changes and partial modifications. Semantic analysis technology: Based on artificial intelligence, semantic models can understand the deep meaning of text, identifying content with the same meaning but different expressions through word embedding and attention mechanisms. String matching technology: This directly compares the similarity of text strings, including exact matching and fuzzy matching. It is simple to implement and computationally fast, but cannot identify semantic similarity.

[0003] As a crucial outcome document comprehensively reflecting the results of geological work, geological reports possess unique textual characteristics and plagiarism detection requirements: **Dense Terminology:** Geological reports are characterized by a high concentration of specialized terminology, including both purely technical and general terms, which are frequently misidentified as plagiarism in existing plagiarism detection systems. **Relatively Fixed Text Structure:** Most geological reports adhere to specific format and structural requirements, exhibiting numerous standardized and normative expressions, making them easily flagged as duplicate content by existing plagiarism detection systems. **Complex Citation Relationships:** Geological reports require citing numerous previous research findings, industry standards, and normative documents, and existing systems are insufficient in distinguishing between legitimate citations and inappropriate plagiarism.

[0004] Issues with terminology recognition: The lack of a geological terminology database makes it easy for standard expressions and normative terms in geology to be misjudged as duplicate content. Insufficient algorithm adaptability: Existing plagiarism detection algorithms are mainly designed for general text and do not consider the professional characteristics and structural features of geological reports, leading to inaccurate results. Insufficient flexibility of fixed thresholds: Using fixed thresholds for duplicate detection lacks flexibility and cannot be dynamically adjusted according to the specific characteristics of geological reports.

[0005] Therefore, it is necessary to develop a plagiarism detection method and device specifically designed for the characteristics of geological reports, in order to improve the accuracy and reliability of plagiarism detection and ensure the quality and originality of geological reports. Summary of the Invention

[0006] To address the problems of existing technologies, this invention provides a method and apparatus for checking geological reports for duplicate content. It solves the problem of inaccurate identification of geological terminology in existing duplication detection systems, avoiding misjudging standard terms as duplicate content; improves the poor adaptability of existing duplication detection algorithms to the structural features of geological reports, thus increasing the accuracy of duplication results; and addresses the problem of inflexible duplication threshold settings, enabling more scientific and reasonable calculation and judgment of duplication rates.

[0007] To solve the above-mentioned technical problems, the present invention is achieved through the following technical solution: Firstly, a method for checking the duplication of geological reports, comprising the following steps:

[0008] The specimen library construction steps are as follows: Based on the geological data in the collection, a basic specimen library for geological report duplication detection is constructed, including structural analysis of geological reports in the specimen library, extraction and preprocessing of main text content, standardization of professional terms, and extraction of content summary and keyword information;

[0009] Preprocessing steps for the plagiarism check report: perform structural analysis, extract and standardize the main text content of the geological report to be checked;

[0010] Steps for establishing a sample thematic database: Classify the reports in the specimen database based on at least one of geological work type, work level, region, and keywords to establish a sample thematic database;

[0011] Content comparison steps: The preprocessed report to be checked for plagiarism is compared with the reports in the sample topic library for text similarity calculation. The text similarity calculation includes multi-level screening calculations performed sequentially.

[0012] Results filtering and optimization steps: Based on preset thresholds and special text field elimination rules, the duplicate results obtained from the content comparison step are filtered and optimized;

[0013] Comprehensive analysis and calculation steps: Based on the optimized duplication results, calculate the overall, single-article, and segment duplication rates of the duplication report to be checked, and generate the duplication report.

[0014] In one specific implementation of the first aspect, the multi-level screening calculation in the content comparison step includes:

[0015] First-level calculation: The Jaccard algorithm is used to calculate the coarse-grained similarity between the report to be checked and the reports in the sample topic library. If the similarity is lower than the first preset threshold, the subsequent calculation is terminated.

[0016] Second-level calculation: For report pairs that pass the first-level calculation, the text is truncated into fixed-length text segments, a hash mapping is established based on the text segments and a hit operation is performed to calculate the text repetition rate. If the text repetition rate meets the second preset condition, the subsequent calculation is terminated.

[0017] Third-level calculation: For report pairs that do not meet the second preset condition, the improved TF-IDF and cosine similarity algorithms are used to perform region content matching calculation.

[0018] In one specific implementation of the first aspect, the second preset condition is: the text repetition rate is less than 10% or greater than 20%; or, the text repetition rate is less than 20% and the repetition rate within every consecutive 200 words does not exceed 40%.

[0019] In one specific implementation of the first aspect, the third-level calculation targets report pairs with a text repetition rate between 10% and 20%, and with a repetition rate of 40% or more within 200 consecutive words.

[0020] In one specific implementation of the first aspect, the formula for calculating the overall repetition rate in the comprehensive analysis and calculation step is as follows:

[0021]

[0022] in:

[0023] Repetition is the overall percentage of duplicate entries;

[0024] weight is the content importance weight of the i-th repeated segment;

[0025] sectionLength is the length of the i-th repeated segment;

[0026] totalLength is the total length of the report;

[0027] The importance weights of the content are pre-set based on the different content sections of the geological report.

[0028] In one specific implementation of the first aspect, the weighting of the content importance includes: a weight of 0.6 for the basic information section of the project, a weight of 1.0 for the description of important achievements, a weight of 0.5 for the project quality management section, and a weight of 0.5 for the project funding usage section.

[0029] In one specific implementation of the first aspect, in the result screening and optimization step, the preset threshold is 2%, and only the results with a similarity rate greater than or equal to 2% between the report to be checked and a single report in the specimen library are summarized; the elimination of special text fields includes eliminating duplicate text fields such as the name of the report writing unit and the content of the references.

[0030] Secondly, a geological report plagiarism detection device includes:

[0031] The specimen library construction module is used to build a basic specimen library for geological report duplication checking based on the geological data in the collection;

[0032] The preprocessing module is used to perform structural analysis, text content extraction, and standardization processing on the geological report to be checked for plagiarism.

[0033] The thematic database management module is used to classify reports in the specimen database based on at least one of the following: geological work type, work level, region, and keywords, and to establish a sample thematic database.

[0034] The similarity calculation module is used to perform multi-level filtering and text similarity calculation between the preprocessed plagiarism check report and the reports in the sample topic library;

[0035] The results optimization module is used to filter and optimize the similarity calculation results based on preset thresholds and special text field elimination rules;

[0036] The report generation module is used to calculate the duplication rate based on the optimized results and generate a duplication report.

[0037] In one specific implementation of the second aspect, the similarity calculation module includes:

[0038] The first-level computing unit is configured to use the Jaccard algorithm for coarse-grained similarity calculation and filtering;

[0039] The second-level computing unit is configured to perform text duplication rate calculation and filtering based on hash hit operation of fixed-length text segments;

[0040] The third-level computing unit is configured to perform region content matching calculations using an improved TF-IDF and cosine similarity algorithm.

[0041] In one specific implementation of the second aspect, the report generation module is configured to calculate the overall duplication rate using a weighted algorithm, and the calculation formula is as follows:

[0042] The weight is set according to the different content sections of the geological report.

[0043] The beneficial effects of this invention are as follows:

[0044] 1. By constructing a standardized dictionary of geological terminology and unifying synonyms, the problem of existing systems misjudging standard terms and normative expressions as duplicate content was effectively solved. Combined with a weighted duplication rate calculation model targeting the structural characteristics of geological reports (such as basic project information and descriptions of important results), the duplication detection results more accurately reflect the duplication of the core content of the report, significantly improving the accuracy and reliability of the results.

[0045] 2. This method is deeply adapted to the characteristics of geological report text. Through a customized process of "report structure parsing - sample thematic library classification - multi-level mixed comparison (Jaccard, hash hit, TF-IDF cosine similarity)", the entire plagiarism detection system has excellent adaptability to the professional attributes, writing structure and expression habits of geological reports, overcoming the shortcomings of general algorithms that are not suitable for this field.

[0046] 3. By establishing a sample thematic database based on multiple dimensions such as geological work type, work level, and region, and combining it with keyword-based correlation matching, relevant comparison samples can be quickly identified. Experimental results show that this can reduce unnecessary computational resource consumption by more than 60%. Employing a multi-level filtering calculation strategy (such as setting dynamic thresholds of 30% and 10%-20%), deep calculations for obviously dissimilar or low-risk text pairs can be automatically skipped, significantly improving the processing efficiency of large-scale data comparison while ensuring accuracy.

[0047] 4. This invention introduces a weighting system based on content importance (e.g., the weight of important achievements is 1.0, and the weight of basic project information is 0.6), through the formula... A comprehensive evaluation is conducted. Simultaneously, a 2% aggregation threshold is set, and special text fields such as references and institution names are automatically filtered. This ensures that the final plagiarism report not only provides a numerical similarity rate but also scientifically reflects the quality and distribution of duplicated content, offering more instructive judgment criteria for manual review. Attached Figure Description

[0048] Figure 1 This is a schematic diagram of the method flow of the present invention. Detailed Implementation

[0049] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0050] like Figure 1 As shown, the core innovation of this invention lies in its comprehensive plagiarism detection method for geological reports, tailored to the specialized characteristics and structural features of these reports. This method not only focuses on traditional text similarity comparison but, more importantly, fully considers the uniqueness of the specialized terminology and structure of geological reports.

[0051] (I) Specimen bank construction technology

[0052] The construction of a specimen repository is a fundamental step in the plagiarism detection method. It is mainly based on the existing geological data reports in the collection to establish a basic specimen repository for geological report plagiarism detection.

[0053] 1. Report structure parsing technology.

[0054] Geological reports have relatively fixed structural features, including a cover, table of contents, and main text. The main text is largely organized according to the following sections: basic information on geological work, basic geographical and geological conditions, major achievements of geological work, quality management of geological work, and the use of geological work funds. This invention employs a rule-based template-based report parsing method to broadly divide the geological report according to its overall content, providing a data foundation for assigning weights to different segments.

[0055] 2. Text content extraction and preprocessing techniques.

[0056] Extracting the main text content requires considering the characteristics of the geological report text. Text preprocessing includes:

[0057] Unified encoding: Unified character encoding for handling special characters;

[0058] Sentence segmentation and word segmentation: Segmenting the entire geological report into words;

[0059] Synonym unification: unifying synonyms into a standard expression.

[0060] 3. Technical standardization of professional terminology.

[0061] Geological reports contain a large number of technical terms, and the same concept may have multiple expressions. This invention establishes a standardized dictionary of geological terminology, including standard expressions for each term. During the construction of the specimen database, the system associates terms with their standard expressions to improve the accuracy of deduplication detection.

[0062] 4. Extract the summary and keywords from the geological report.

[0063] Construct a geological terminology dictionary; extract content summaries and keyword information from geological reports based on metadata information of geological data in the collection; segment the content summaries of geological reports into words, perform statistical analysis based on professional relevance and word frequency, and link them with the terminology dictionary.

[0064] (II) Preprocessing of geological reports to be checked for duplication.

[0065] Preprocessing of geological reports to be checked for plagiarism is a crucial step in ensuring the accuracy of plagiarism calculations. It mainly involves structural analysis, text content processing, and standardization of the geological reports to be checked, based on relevant technologies for constructing specimen libraries.

[0066] (III) Techniques for establishing a sample thematic database.

[0067] The establishment of a sample topic database is an important part of the entire plagiarism detection method and a key step in improving plagiarism detection efficiency.

[0068] 1. Classification based on geological reports

[0069] The key to establishing a sample thematic database lies in a reasonable classification system. This invention adopts a multi-dimensional classification method based on geological work classification standards and report content characteristics. The classification system includes:

[0070] Geological work can be categorized by type as follows: regional geological survey, mineral geological exploration, geophysical exploration, geochemical exploration, hydrogeological exploration, engineering geological exploration, environmental geological survey, etc.

[0071] Classified by work level: 1:50000, 1:100000, 1:200000, 1:250000, 1:500000, 1:1000000, preliminary survey, general survey, detailed survey, exploration, etc.

[0072] Classified by region: different provinces, different map sheets, different metallogenic belts, etc.

[0073] 2. Relationship matching based on keywords and categories

[0074] To improve the efficiency of content duplication calculation, based on the characteristics of geological reports, including report content summary, keywords, work type, engineering level, and mineral information, a preliminary screening of all specimen database texts is conducted. Enhanced matching is then performed based on basic geological information and geographical location information. According to current measurements, this method can solve more than 60% of the computational resources on average.

[0075] (iv) Content comparison technology.

[0076] Content comparison is the core component of this plagiarism detection method.

[0077] 1. Text similarity calculation technology.

[0078] Traditional text similarity calculation methods mainly include string matching-based methods and semantic-based methods. This invention proposes a hybrid similarity calculation method that takes into account the professional characteristics of geological reports.

[0079] First, the system uses the Jaccard algorithm for coarse-grained matching. The Jaccard algorithm compares the text content of the report to be compared with the text content in the specimen library, calculating their similarity to preliminarily measure the similarity between the two texts. For text pairs with a similarity of less than 30%, subsequent calculations are automatically skipped to improve the overall computational efficiency of text content comparison.

[0080] Secondly, the system truncates both the specimen database text and the text to be checked for plagiarism into 13-character segments. A memory hash mapping is established based on the specimen database text segments. Then, a matching operation is performed on the text segments to be checked for plagiarism. The text duplication rate is analyzed based on the matching operation results. Texts with a duplication rate less than 10% and greater than 20%, or with a duplication rate less than 20% and a duplication rate not exceeding 40% per 200 characters, are automatically skipped from subsequent calculations to improve the overall efficiency of text content comparison.

[0081] Third, for texts with a repetition rate between 10% and 20%, and with 200 words having a repetition rate greater than or equal to 40%, TF-IDF and cosine similarity methods are used for regional content matching. TF-IDF vectors can reflect the lexical features of geological report texts, while cosine similarity can calculate the degree of similarity between text content in the vector space. Considering the characteristics of geological reports, the system further refines and improves the TF-IDF algorithm through professional terminology optimization and word vector models to further enhance the accuracy of the calculation.

[0082] 2. Technique for calculating the percentage of duplicate content.

[0083] The calculation of the percentage of duplicate content should consider not only the absolute number of duplicate content, but also the distribution characteristics and importance weight of the duplicate content.

[0084] (1) Weight calculation based on importance.

[0085] Different types of content have different levels of importance in geological reports, and the system assigns different weights to these different types of content:

[0086] Project basic information: Weight is 0.6;

[0087] Key findings: weighted at 1.0;

[0088] Project quality management: weight 0.5;

[0089] Project funding usage: weighted at 0.5;

[0090] The determination of repetition is based on statistical analysis of a large number of geological report instances and can be configured to be dynamically optimized through machine learning methods so that the calculation results more accurately reflect the importance of repetitive content.

[0091] (2) Calculation algorithm for overall proportion

[0092] The formula for calculating the overall percentage is:

[0093]

[0094] in:

[0095] Repetition: Overall percentage of repetitions;

[0096] weight: The importance weight of the i-th repeated segment;

[0097] sectionLength: The length of the i-th repeating segment;

[0098] totalLength: Total length of the report.

[0099] (v) Results screening and optimization techniques.

[0100] To avoid excessive low-value duplicate information interfering with the plagiarism detection results, the system adopts a multi-level filtering mechanism.

[0101] 1. Threshold adjustment technology.

[0102] The system uses a fixed 2% threshold for result screening and optimization. Since the average geological report is about 100,000 words, reports with a plagiarism rate of less than 2% (total duplicate text of about 2,000 words) in the report to be checked and reports in the specimen database are automatically ignored when calculating the overall plagiarism rate. Only the relevant results with a plagiarism rate of 2% or higher in the report to be checked and reports in the specimen database are summarized.

[0103] 2. Elimination of special text fields.

[0104] For the filtered duplicate content, the system will also focus on special text fields for duplicate text ablation, such as the name of the reporting unit and references. After removing special duplicate text fields, the overall text duplication rate will be recalculated according to the above algorithm.

[0105] (vi) Comprehensive analysis and calculation techniques.

[0106] Comprehensive analysis and calculation is the final step in the entire plagiarism detection method, and its goal is to provide users with comprehensive and accurate plagiarism detection results.

[0107] 1. Multidimensional analysis technology

[0108] The system analyzes the plagiarism detection results from multiple dimensions:

[0109] Overall duplication rate analysis: Calculates the overall duplication rate of the report;

[0110] Single-article duplication rate analysis: Analyze the duplication rate of the report to be checked and the single reports in the specimen library, and identify and mark the single reports with a duplication rate higher than 20%;

[0111] Segmented duplication rate analysis: Analyze the duplication rate of different parts such as basic project information, description of important results, project quality management, and use of project funds.

[0112] 2. Plagiarism report generation technology

[0113] The system generates a duplicate report for the geological report. The report includes the following:

[0114] Basic information: Report name, project, project code, project leader, plagiarism checking organization, plagiarism checking time, plagiarism checking number, etc.;

[0115] Overall results: overall duplication rate, single-article duplication rate, etc.;

[0116] Detailed analysis: Information such as the content and source of each repeated segment.

[0117] II. Equipment Technical Solution.

[0118] The geological report duplication detection device provided by this invention adopts a modular design.

[0119] (I) Parallel computing framework.

[0120] The software system employs a distributed computing framework, supporting large-scale parallel processing. Framework features include:

[0121] 1. Task decomposition: Break down large plagiarism detection tasks into multiple subtasks and allocate them to different computing threads.

[0122] 2. Fault tolerance: When a thread fails, the task is automatically transferred to another thread or a new thread is created.

[0123] 3. Results Summary: Summarize and integrate the calculation results from each thread.

[0124] (ii) Professional algorithm library.

[0125] The software system integrates a geological professional algorithm library, containing various plagiarism detection algorithm implementations. The algorithm library mainly includes word segmentation, various text similarity calculation methods, matching, and other total calculation methods.

[0126] (III) User Interface System.

[0127] The software system provides a user-friendly interface that supports the needs of different user roles.

[0128] 1. The interface includes:

[0129] Administrator interface: System management, user management, data management, etc.;

[0130] Operator interface: submitting plagiarism detection tasks, monitoring progress, and viewing results.

[0131] 2. The following principles should be considered in the interface design:

[0132] Simple and easy to use: The interface is simple and the operation is intuitive;

[0133] Fully functional: Covers all necessary features.

[0134] (iv) Core functional modules.

[0135] 1. Report preprocessing module.

[0136] This module is responsible for preprocessing the input geological report to prepare for subsequent deduplication checks.

[0137] Features include:

[0138] Format recognition: Identifies the file format of the report, such as PDF, TXT, etc.

[0139] Text cleanup: Cleaning up noisy information in the report, such as page numbers;

[0140] Unified encoding: Unified character encoding for handling special characters.

[0141] 2. Feature Information Extraction Module

[0142] This module is responsible for extracting various features from the preprocessed report for similarity calculation.

[0143] Features include:

[0144] Text content extraction: Extract all text content and record information such as pagination;

[0145] Structural content extraction: Based on the report text content, the text is segmented into different parts (before, during, and after) and associated with different weights to facilitate text similarity calculation.

[0146] Professional feature extraction: Extract geological professional feature keywords, such as stratigraphic information and mineral information.

[0147] 3. Similarity calculation module.

[0148] This module is responsible for calculating the similarity between reports and is the core module for plagiarism detection.

[0149] Features include:

[0150] Text similarity calculation: Multiple algorithms are used to calculate the text similarity between different text segments;

[0151] Comprehensive similarity calculation: The final similarity score is obtained by combining various similarity results.

[0152] 4. Report generation module.

[0153] This module is responsible for generating the final plagiarism report. Its main function is to generate report content based on the plagiarism check results.

[0154] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for checking the duplication of geological reports, characterized in that: Includes the following steps: The specimen library construction steps are as follows: Based on the geological data in the collection, a basic specimen library for geological report duplication detection is constructed, including structural analysis of geological reports in the specimen library, extraction and preprocessing of main text content, standardization of professional terms, and extraction of content summary and keyword information; Preprocessing steps for the plagiarism check report: perform structural analysis, extract and standardize the main text content of the geological report to be checked; Steps for establishing a sample thematic database: Classify the reports in the specimen database based on at least one of geological work type, work level, region, and keywords to establish a sample thematic database; Content comparison steps: The preprocessed report to be checked for plagiarism is compared with the reports in the sample topic library for text similarity calculation. The text similarity calculation includes multi-level screening calculations performed sequentially. Results filtering and optimization steps: Based on preset thresholds and special text field elimination rules, the duplicate results obtained from the content comparison step are filtered and optimized; Comprehensive analysis and calculation steps: Based on the optimized duplication results, calculate the overall, single-article, and segment duplication rates of the duplication report to be checked, and generate the duplication report.

2. The method for checking the duplication of geological reports according to claim 1, characterized in that: The multi-level filtering calculation in the content comparison step includes: First-level calculation: The Jaccard algorithm is used to calculate the coarse-grained similarity between the report to be checked and the reports in the sample topic library. If the similarity is lower than the first preset threshold, the subsequent calculation is terminated. Second-level calculation: For report pairs that pass the first-level calculation, the text is truncated into fixed-length text segments, a hash mapping is established based on the text segments and a hit operation is performed to calculate the text repetition rate. If the text repetition rate meets the second preset condition, the subsequent calculation is terminated. Third-level calculation: For report pairs that do not meet the second preset condition, the improved TF-IDF and cosine similarity algorithms are used to perform region content matching calculation.

3. The method for checking the duplication of geological reports according to claim 2, characterized in that: The second preset condition is: the text repetition rate is less than 10% or greater than 20%; or, the text repetition rate is less than 20% and the repetition rate within every consecutive 200 words does not exceed 40%.

4. The method for checking the duplication of geological reports according to claim 2, characterized in that: The third level of calculation targets report pairs with a text repetition rate between 10% and 20%, and which have a repetition rate of 40% or more within 200 consecutive words.

5. The method for checking the duplication of geological reports according to claim 1, characterized in that: In the comprehensive analysis and calculation step, the formula for calculating the comprehensive repetition rate is: in: Repetition is the overall percentage of duplicate entries; weight is the content importance weight of the i-th repeated segment; sectionLength is the length of the i-th repeated segment; totalLength is the total length of the report; The importance weights of the content are pre-set based on the different content sections of the geological report.

6. The method for checking the duplication of geological reports according to claim 5, characterized in that: The weighting of the content importance includes: 0.6 for the basic information section of the project, 1.0 for the description of important achievements, 0.5 for the quality management section of the project, and 0.5 for the use of project funds.

7. The method for checking the duplication of geological reports according to claim 1, characterized in that: In the result screening and optimization step, the preset threshold is 2%, and only the relevant results with a plagiarism rate greater than or equal to 2% between the report to be checked and a single report in the specimen library are summarized. The elimination of special text fields includes eliminating duplicate text fields such as the name of the report-writing unit and the content of the references.

8. A geological report plagiarism detection device, characterized in that, include: The specimen library construction module is used to build a basic specimen library for geological report duplication checking based on the geological data in the collection; The preprocessing module is used to perform structural analysis, text content extraction, and standardization processing on the geological report to be checked for plagiarism. The thematic database management module is used to classify reports in the specimen database based on at least one of the following: geological work type, work level, region, and keywords, and to establish a sample thematic database. The similarity calculation module is used to perform multi-level filtering and text similarity calculation between the preprocessed plagiarism check report and the reports in the sample topic library; The results optimization module is used to filter and optimize the similarity calculation results based on preset thresholds and special text field elimination rules; The report generation module is used to calculate the duplication rate based on the optimized results and generate a duplication report.

9. A geological report plagiarism detection device according to claim 8, characterized in that: The similarity calculation module includes: The first-level computing unit is configured to use the Jaccard algorithm for coarse-grained similarity calculation and filtering; The second-level computing unit is configured to perform text duplication rate calculation and filtering based on hash hit operation of fixed-length text segments; The third-level computing unit is configured to perform region content matching calculations using an improved TF-IDF and cosine similarity algorithm.

10. A geological report plagiarism detection device according to claim 8, characterized in that: The report generation module is configured to use a weighted algorithm to calculate the overall duplication rate, and the calculation formula is as follows: The weight is set according to the different content sections of the geological report.

Images