Text similarity calculation method and system based on multi-algorithm fusion

Abstract

The invention discloses a text similarity calculation method and system based on multi-algorithm fusion. The text similarity calculation method comprises the following steps: step 1, performing structured processing on a to-be-checked project document; step 2, similarity calculation; step 3, outputting a result; the text similarity calculation system comprises an approval project library module, a content repetition review module and a report management module. According to the method, the multi-algorithm fusion design is adopted, the duplicate checking accuracy is effectively improved, an algorithm threshold regulation and control mechanism is introduced, efficient utilization of computing power resources is achieved, and a quantitative decision basis is provided for project review through intelligent repeated review.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing technology, and in particular relates to a text similarity calculation method and system based on multi-algorithm fusion. Background Technology

[0002] In the current government project application system, the problem of duplicate application content has long existed and is difficult to eradicate. For example, different regions or departments often submit duplicate applications for projects with similar functions but different descriptions, resulting in inefficient use of fiscal funds.

[0003] Traditional content plagiarism detection methods have the following drawbacks: (1) Relying on traditional keyword matching technology cannot identify equivalent expressions of government terminology, such as “one-stop online service” and “full-process online service”, “intelligent transportation system” and “intelligent travel platform”, “intelligent” and “AI”, which presents a semantic understanding gap; (2) It is difficult to balance computing resources and response efficiency. General-purpose large-scale deep semantic algorithms cannot meet the needs of large-scale real-time applications. Although deep semantic similarity calculation based on large pre-trained models can effectively solve some semantic problems, its model parameters are huge, the inference process is computationally complex, and it requires a lot of GPU resources and computing time. In the CPU server environment commonly used in government information systems, the time taken to perform full database comparison for a single project may be tens of minutes or even longer. When facing the peak period of project application, it is necessary to handle a large number of concurrent deduplication requests. Limited hardware resources will become a huge bottleneck, resulting in slow system response or even crash, and a very poor user experience. It is difficult to achieve "overuse" in the actual production environment. (3) Lack of domain specialization: The general model has insufficient accuracy in recognizing specific government terminology and policy expressions. The training data of the general model mainly comes from general Internet texts, which is insufficient for learning highly specialized and policy-oriented terminology and fixed expressions in the government field. For example, for terms with specific policy connotations such as "three guarantees expenditure" and "special bond project", the general model may not be able to accurately capture their core concepts, either failing to recognize them or causing ambiguity. This lack of "domain adaptability" leads to poor performance in the vertical field of government affairs, and the calculated similarity deviates from the actual judgment of experts, resulting in low professional credibility of the final output. (4) Data unstructured bottleneck: The original document data has not been effectively transformed into computable structured data. Traditional plagiarism detection methods often directly apply to the entire unstructured application document (PDF format, Word format), treating it as a "black box text". However, the project application is a highly structured text, containing different chapters such as basic project information, construction goals, functional list, and technical route. Traditional plagiarism detection methods do not perform intelligent parsing and information extraction on the application document, nor do they structurally separate the content from different dimensions, resulting in a coarse-grained and unfocused plagiarism detection process that cannot achieve accurate repetition analysis.

[0004] Existing patent document 1, "Semantic-based text similarity calculation method" (publication number CN103617157A), discloses that "when calculating the similarity between texts, the similarity of keyword pairs and the similarity of concept pairs are calculated separately, thereby overcoming the impact of the high dimensionality and sparsity of the vector space model on text similarity calculation." However, it lacks sensitivity to government terminology and ignores the precise matching of government terminology.

[0005] Existing patent document 2, "A Method for Calculating Patent Text Similarity" (publication number CN108536677A), discloses that "it comprehensively considers the structural characteristics of patents and the semantic relationships between words, integrates the unique structure of patent texts such as IPC classification numbers, abstracts, and claims into the text similarity calculation method, which is more targeted than general text similarity calculation methods, and calculates the semantic relationships between words through word2vec." However, it lacks semantic understanding and has a low accuracy rate in recognizing domain-specific expressions. Summary of the Invention

[0006] The purpose of this invention is to solve the above-mentioned technical problems existing in the prior art, and to provide a text similarity calculation method and system based on multi-algorithm fusion. By adopting a multi-algorithm fusion design, the accuracy of plagiarism detection is effectively improved. By introducing an algorithm threshold control mechanism, the efficient utilization of computing resources is achieved. Through intelligent duplicate review, a quantitative decision-making basis is provided for project review.

[0007] To solve the above-mentioned technical problems, the present invention employs the following technical solution: The text similarity calculation method based on multi-algorithm fusion is characterized by the following steps: Step 1: Structure processing of project documents to be reviewed: (1.1) Obtaining documents: Obtain the project documents to be reviewed as the comparison documents; obtain historical project documents and use the functional descriptions in the historical project documents as the comparison text; (1.2) Processing documents: Perform structured processing on the documents to be compared in (1.1) to obtain structured data; extract functional descriptions from the structured data, and use each functional description as the text to be compared to form a set of texts to be compared; Step 2, Similarity Calculation: Use HanLP combined with cosine similarity algorithm and large model deep semantic understanding similarity algorithm to calculate the similarity between the text to be compared in (1.2) and the text compared in (1.1), and obtain the final similarity value S. final The calculation steps are as follows: (2.1) Use HanLP’s word segmentation function, which combines HanLP with cosine similarity algorithm, to perform word segmentation on the text to be compared in (1.2) and the text to be compared in (1.1) respectively, to complete the preprocessing of the text to be compared and the text to be compared. (2.2) Based on the TF-IDF method, the preprocessed text to be compared and the text to be compared in (2.1) are converted into corresponding feature vectors to obtain two feature vectors; (2.3) Call the HanLP combined with cosine similarity algorithm to calculate the two feature vectors in (2.2) and obtain the HanLP combined with cosine similarity value S. cos Then compare S cos The comparison results with the values ​​of W1 and W2 are as follows: If S cos <W1 or S cos >W2, set the final similarity value S final =S cos ; If W1≤S cos If W2 ≤ W2, the large model deep semantic understanding similarity algorithm is called to calculate the similarity between the text to be compared in (1.2) and the text compared in (1.1), and the large model deep semantic understanding similarity value S is obtained. sem Set the final similarity value S final =S sem ; Step 3: Output Results: Output the final similarity value S final .

[0008] A text similarity calculation system based on multi-algorithm fusion is characterized by including: a project database module, a content duplication review module, and a report management module; The project database module is used to parse uploaded project documents awaiting review, extract key information and functional descriptions from the parsed project documents, and form structured data. The content duplication review module is used to calculate the text similarity between the project documents to be reviewed input from the project library module and each document in the historical project document set, based on the functional description. The module obtains the corresponding similarity value and outputs it. Then, it compares each similarity value with the preset substantive duplication threshold and displays the duplicate content and comparison results based on the comparison results. The report management module is used to convert the data output from the content duplication review module into review reports.

[0009] The present invention, by adopting the above-described technical solution, has the following beneficial effects: 1. Employing a multi-algorithm fusion design effectively improves the accuracy of plagiarism detection. This invention integrates the advantages of HanLP combined with cosine similarity algorithm and large-model deep semantic understanding similarity algorithm. Through the complementarity between algorithms, it effectively overcomes the limitations of a single algorithm, retaining the efficiency of literal matching while utilizing the semantic understanding's ability to recognize diverse expressions. Thus, it achieves more comprehensive and accurate plagiarism detection in complex government text environments.

[0010] 2. An algorithm threshold adjustment mechanism is introduced to achieve efficient utilization of computing resources. This invention allows users to customize the threshold application range of the deep semantic understanding similarity algorithm for large models based on actual resource conditions. When computing resources are scarce, the threshold range can be appropriately narrowed to prioritize the rapid response of core tasks; when computing resources are sufficient, the threshold range can be expanded to conduct more comprehensive and in-depth semantic comparisons. This adjustment mechanism significantly improves the flexibility and environmental adaptability of text similarity calculation methods, maximizing the effectiveness of limited computing resources.

[0011] 3. The establishment of a structured project database laid a solid foundation for data intelligence. The project database module uses automatic parsing and extraction technology to transform unstructured project documents awaiting review into standard structured data. This process not only significantly reduces the workload of manual data entry and improves data processing efficiency, but also forms unified and standardized project data assets, providing a reliable data foundation for subsequent data analysis, decision support, and knowledge management.

[0012] 4. Intelligent duplicate review provides a quantitative basis for project evaluation. The content duplicate review module can efficiently and accurately calculate the text similarity of the project to be reviewed and the historical project set in terms of functional descriptions, and automatically generate review results based on configurable thresholds. This function provides project reviewers with scientific and objective quantitative indicators, effectively assisting them in identifying the risk of duplicate project approval, and significantly improving the accuracy, objectivity and efficiency of project evaluation.

[0013] 5. Generate professional review reports to ensure effective communication and persistent retention of review conclusions. The report management module can automatically convert data results from repeated reviews into well-structured, professionally formatted review reports. It supports rapid report generation, flexible customization, convenient viewing, and export, not only achieving efficient communication of review conclusions but also ensuring the transparency of the analysis process and the traceability of results, providing strong support for knowledge accumulation and audit trails. Attached Figure Description

[0014] The present invention will be further described below with reference to the accompanying drawings: Figure 1 This is a flowchart of text similarity calculation in this invention; Figure 2 This is a flowchart of the calculation of cosine similarity value using HanLP in this invention; Figure 3 This is a flowchart of the calculation of the similarity value for deep semantic understanding of the large model in this invention; Figure 4 This is a schematic diagram of the text similarity calculation system based on multi-algorithm fusion in this invention. Detailed Implementation

[0015] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0016] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0017] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion.

[0018] Definitions of technical terms described in this invention: Apache PDFBox is an open-source Java library designed specifically for PDF document processing.

[0019] Apache POI is an open-source Java API designed for processing Microsoft Office documents, supporting read and write operations for various file formats.

[0020] XWPFParagraph is a core component of the Apache POI library used to process paragraphs in Word documents, supporting text editing, style setting, and content replacement.

[0021] HanLP is a multilingual natural language processing toolkit.

[0022] TF-IDF is a commonly used text vectorization method in natural language processing, used to evaluate the importance of words in a document.

[0023] The text similarity calculation method based on multi-algorithm fusion includes the following steps: Step 1: Structure processing of project documents to be reviewed: (1.1) Obtaining documents: Obtain the project documents to be reviewed from the project database as the comparison documents; if the project documents to be reviewed are in PDF format, use Apache PDFBox to parse the content of the project documents to be reviewed, convert them to Word format, and retain the original layout tags under the PDF format of the project documents to be reviewed.

[0024] Retrieve historical project documents from the database and use the functional descriptions in these documents as the comparison text.

[0025] (1.2) Document processing: (1.2.1) Parsing the document: Use Apache POI to read the format-converted document to be compared in (1.1), and perform structured parsing of the document to be compared according to the following hierarchy: ① Directory structure: The paragraphs are traversed using XWPFDocument.getParagraphs(), and a tree-like directory is generated by combining styles (such as Heading1 / Heading2); ② Paragraph information: Extract the text content, font style (bold / indented), and list number (e.g., “1.1”, “(a)”) of each XWPFParagraph.

[0026] (1.2.2) Information extraction: Use regular expressions to extract the following key information from the document to be compared after parsing (1.2.1): ① Project Name: Recognizes "Project Name: XXX" or title-style text, or file name; ②Applicant: Identify "Applicant: XXX" or header / footer information; ③ Application Date: Identify "Application Date: YYYY-MM-DD" or a synonym (such as "Submission Date").

[0027] Extract each functional description from the document to be compared after parsing (1.2.1), and use each functional description as the text to be compared to form a set of texts to be compared.

[0028] (1.2.3) Results: The structured data after structured processing is obtained, and the key information in (1.2.2) and the set of texts to be compared are stored in the database.

[0029] Step 2, Similarity Calculation: like Figures 1 to 3 As shown, the similarity between the text to be compared in (1.2.2) and the text compared in (1.1) is calculated by calling HanLP, which combines the cosine similarity algorithm and the large model deep semantic understanding similarity algorithm, and the final similarity value S is obtained. final .

[0030] Among them, the large model of the deep semantic understanding similarity algorithm is the DeepSeek-R1 logical reasoning model. The DeepSeek-R1 logical reasoning model is an existing technology and will not be described in detail here.

[0031] The calculation steps are as follows: (2.1) The HanLP word segmentation function, which combines HanLP with cosine similarity algorithm, is used to perform word segmentation on the text to be compared in (1.2.2) and the text to be compared in (1.1): unify lowercase, remove empty characters, filter stop words, and expand synonyms to complete the preprocessing of the text to be compared and the text to be compared.

[0032] The implementation of the synonym expansion relies on the semantic resources of the synonym expansion library, which adds a real-time government terminology library on top of the basic thesaurus.

[0033] Dynamic updates to the real-time government terminology database: New government terms are captured by manually monitoring government gazettes, such as by inspecting official website announcements three times a day (morning, noon, and evening). After being approved by reviewers, the new government terms are added to the real-time government terminology database.

[0034] (2.2) Based on the TF-IDF method, the preprocessed text to be compared and the text to be compared in (2.1) are converted into corresponding feature vectors to obtain two feature vectors.

[0035] (2.3) Call the HanLP combined with cosine similarity algorithm to calculate the two feature vectors in (2.2) and obtain the HanLP combined with cosine similarity value S. cos S cos The numerical range of S is 0-1, then compare S.cos The magnitude of W1 and W2.

[0036] Where W1 < W2, the numerical range of W1 and W2 is 0-1, and W1 and W2 are both thresholds for calling the large model deep semantic understanding similarity algorithm; when the text similarity value calculated by HanLP combined with the cosine similarity algorithm falls within the range of [W1, W2], the text similarity calculation method in this invention calls the large model deep semantic understanding similarity algorithm to calculate the text similarity.

[0037] When W1=0.2 and W2=0.8, the comparison results are as follows: If S cos <0.2 or S cos >0.8, set the final similarity value S final =S cos ; If 0.2≤S cos If the similarity is ≤0.8, the large model deep semantic understanding similarity algorithm is called to calculate the similarity between the text to be compared in (1.2.2) and the text compared in (1.1). The steps are as follows: ① Deploy large models: Design prompt words to ensure that large models output stable and parsable results; ② Perform synonym expansion on the text to be compared in (1.2.2) and the text to be compared in (1.1), respectively. Then, call the large model to calculate the similarity between the text to be compared and the text to be compared after the synonym expansion, and obtain the deep semantic understanding similarity value S of the large model. sem S sem The numerical range is 0-1, and the final similarity value S is set. final =S sem .

[0038] The values ​​of W2 and W1 can be dynamically adjusted according to the actual resource situation to avoid the performance degradation of the text similarity calculation method in this invention due to the increased number of calls to the large model and the increased resource requirements caused by the larger range of [W1, W2].

[0039] First, HanLP combined with the cosine similarity algorithm is called, which can filter out most of the cases that are very similar or very dissimilar, greatly saving costs and time. Then, a large-scale deep semantic understanding similarity algorithm is called to accurately judge the cases that are fuzzy and similar through the large model, and the results are more reliable.

[0040] Step 3: Output Results: Output the final similarity value S final And determine the result.

[0041] Case 1: Documents to be compared: Return function, including initiating the return process and agreeing to the return process; when an event is returned to the city level in the self-developed system or still needs to be processed after reaching the circulation period, the return operation needs to be performed synchronously on the provincial platform.

[0042] Documents being compared: Initiating the return process means that when an event is returned to the city level in the self-developed system or still needs to be processed after reaching the circulation period, the return operation needs to be carried out on the provincial platform.

[0043] S is calculated by calling HanLP combined with the cosine similarity algorithm. cos =0.8036; Output S final =0.8036; Judgment result: The content is substantially duplicated.

[0044] Case 2: Documents to be compared: This system provides a natural language-based document drafting assistance function. Users can describe their intentions verbally or by entering text, and the system can automatically generate drafts that conform to the format of government documents, including notices, reports, letters, etc., and supports content polishing and typo checking.

[0045] The document being compared is equipped with an AI document assistant that supports both voice and text input. It intelligently creates initial drafts of official documents, covering common document types, and has the ability to correct grammatical errors and review standardization.

[0046] S is calculated by calling HanLP combined with the cosine similarity algorithm. cos =0.2041; The S value is calculated by calling the deep semantic understanding similarity algorithm of the large model. sem =0.9; Output S final =0.9; Judgment result: The content is substantially duplicated.

[0047] Case 3: Document to be compared: Implements Single Sign-On (SSO) integration, allowing users to access all authorized applications using unified identity authentication.

[0048] Documents being compared: Provides a unified portal entry point, allowing users to jump to various business subsystems after a single login without needing to authenticate again.

[0049] S is calculated by calling HanLP combined with the cosine similarity algorithm. cos =0.2505; The S value is calculated by calling the deep semantic understanding similarity algorithm of the large model. sem =0.8; Output S final =0.8; Judgment result: The content is substantially duplicated.

[0050] Case 4: Documents to be compared: Built-in AI document assistant, supports voice and text input, intelligently creates initial drafts of official documents, covers common document types, and has the ability to correct grammatical errors and review standardization.

[0051] The document being compared: provides a login function, allowing users to log in to the system and authorize access to applications after entering their account and password.

[0052] S is calculated by calling HanLP combined with the cosine similarity algorithm. cos =0.0917; Output S final =0.0917; Judgment result: There is no substantial duplication of content.

[0053] like Figure 4 As shown, the text similarity calculation system based on multi-algorithm fusion in this invention includes: a project database module, a content duplication review module, and a report management module.

[0054] The project database module is used to parse uploaded project documents awaiting review, extract key information and functional descriptions from the parsed documents and form structured data, enabling two-way interactive viewing of the original text and the parsed results, effectively improving the efficiency of project management and review.

[0055] The project database module includes a text acquisition module and a text processing module. The text acquisition module is used to obtain the project documents to be reviewed from the project database and output the project documents to the text processing module. The text processing module is used to perform structured processing on the project documents to be reviewed input by the text acquisition module to obtain structured data.

[0056] The text processing module includes a format conversion module, a text parsing module, and an information extraction module. The format conversion module uses Apache PDFBox to convert the PDF project documents to Word format while retaining the original layout tags in the PDF format. The text parsing module uses Apache POI to perform structured parsing on the converted project documents to obtain the table of contents and paragraph information. The information extraction module is used to extract key information and functional descriptions such as project name, applicant organization, and application date from the parsed project documents and outputs the key information and functional descriptions to the database for storage.

[0057] The processing flow of the project initiation database module is as follows: a. Document Acquisition: Supports uploading common formats such as Word and PDF, and supports single file or batch upload; b. Intelligent parsing: Automatically extracts basic information from the project documents to be reviewed and parses the functional descriptions of the project documents to be reviewed; c. Linked Viewing: Displays the original document and structured data side by side, supporting bidirectional click positioning and highlighting, facilitating verification and correction.

[0058] Users select a project document to be reviewed from the project database and one or more historical project documents from the database. The content duplication review module calls HanLP, which combines cosine similarity algorithm and large-scale deep semantic understanding similarity algorithm, to calculate the text similarity of the functional description of each document in the project database and the historical project document set. The corresponding similarity value is obtained and output. Then, each similarity value is compared with a preset substantial duplication threshold. Data that meets the threshold is stored in the database and the duplicate content and comparison results are presented in list form. Users can drill down to view detailed text comparison highlights, clearly displaying the review results.

[0059] The content duplication review module's processing flow is as follows: a. Selection of projects pending review: Provides a visual interface, supports searching, and allows users to select a project document that needs to be reviewed from the project library; b. Comparison Project Selection: Supports querying historical projects by project application date range (e.g., the past three years), project application unit, and other dimensions, and allows selecting multiple historical project documents for comparison; c. Call HanLP, which combines cosine similarity algorithm and large model deep semantic understanding similarity algorithm, to calculate the text similarity of the functional description of each document in the project document to be reviewed and the historical project document set. Then, compare each calculated similarity value with the preset substantial duplication threshold and output the risk items that need to be focused on. d. Display the path location, function description, and duplication rate of pending projects that meet the threshold conditions in a list format. Supports drill-down to view the association information with historical projects, including the associated project name, path location, function description, and duplication rate. It also supports linked viewing, displaying pending documents and comparison documents side by side, and supports bidirectional click positioning and highlighting to intuitively display duplicate content.

[0060] The report management module, based on XWPFDocument technology, transforms the data output from the content duplication review module into well-structured, professionally formatted, and uniformly formatted review reports. These reports are stored in a database for easy retrieval. The module supports rapid report generation, flexible customization, convenient viewing, and export, ensuring effective delivery and persistent storage of review conclusions. It also supports online previewing and direct export to Word format.

[0061] The processing flow of the report management module is as follows: a. Add a new report generation task: Enter the report name, select a preset report generation template, and select a project name; b. Report Generation: Click "Report Generation" to generate a report based on the preset report generation template; already generated reports can be regenerated. c. Report viewing: Supports online preview, export to Word format, and viewing of historical versions.

[0062] The above are merely specific embodiments of the present invention, but the technical features of the present invention are not limited thereto. Any simple changes, equivalent substitutions, or modifications made based on the present invention to solve essentially the same technical problems and achieve essentially the same technical effects are all covered within the protection scope of the present invention.

Claims

1. A text similarity calculation method based on multi-algorithm fusion, characterized in that Comprise the following steps: step 1, pending project document structured processing: (1.1) obtain the document: obtain the pending project document as the to-be-compared document; obtain the historical project document, and take the function description in the historical project document as the compared text; (1.2) process the document: the to-be-compared document in (1.1) is structured, and structured data is obtained; the function description is extracted from the structured data, and each function description is taken as a to-be-compared text to form a to-be-compared text set; step 2, similarity calculation: calling HanLP combined with cosine similarity algorithm and large model deep semantic understanding similarity algorithm to calculate the similarity between the to-be-compared text in (1.2) and the compared text in (1.1), and obtain the final similarity value S final , the calculation steps are as follows: (2.1) using HanLP combined with HanLP word segmentation function of cosine similarity algorithm respectively to (1.2) in the to-be-compared text and (1.1) in the compared text are processed, the preprocessing of to-be-compared text and compared text is completed; (2.2) based on TF-IDF method, the to-be-compared text and the compared text in (2.1) are converted into corresponding feature vectors, and two feature vectors are obtained; (2.3) calling HanLP combined with cosine similarity algorithm to calculate the two feature vectors in (2.2), and obtain the HanLP combined with cosine similarity value S cos , then compare S cos and the numerical value of W1, W2, and the comparison result is as follows: if S cos <W1 or S cos >W2, set the final similarity value S final =S cos ; if W1≤S cos ≤W2, calling large model deep semantic understanding similarity algorithm to calculate the similarity between the to-be-compared text in (1.2) and the compared text in (1.1), and obtain the large model deep semantic understanding similarity value S sem , set the final similarity value S final =S sem ; step 3, output results: output the final similarity value S final . 2.The method of claim 1, wherein: In (1.1) of step 1, if the to-be-reviewed project document is in PDF format, the content of the to-be-reviewed project document is parsed using Apache PDFBox, converted into a Word format, and the original layout tags in the to-be-reviewed project document in the PDF format are retained. 3.The method of claim 1, wherein: The specific steps of (1.2) in step 1 are: (1.2.1) parsing the document: structurally parsing the to-be-compared document in (1.1) to obtain the parsed to-be-compared document; (1.2.2) extracting information: extracting each item of functional description from the parsed to-be-compared document in (1.2.1), taking each item of functional description as a to-be-compared text, and forming a to-be-compared text set; (1.2.3) obtaining results: obtaining the structured data after structural processing, and storing the to-be-compared text set in (1.2.2) in a database.

4. The multi-algorithm fusion based text similarity calculation method according to claim 3, characterized in that: In (1.2.1) of step 1, the to-be-compared document in (1.1) is structurally parsed using Apache POI to obtain the directory structure and paragraph information of the to-be-compared document.

5. The method of claim 3, wherein the method is based on multi-algorithm fusion. In step (1.2.2), regular expressions are used to extract key information from the parsed to-be-compared document in (1.2.1), including project name, reporting unit, and reporting date.

6. The multi-algorithm fusion based text similarity calculation method according to claim 1, characterized in that: In (2.1) of step 2, the to-be-compared text in (1.2) and the compared text in (1.1) are subjected to word segmentation processing using HanLP combined with the HanLP word segmentation function of the cosine similarity algorithm: unified lowercase, removal of empty characters, filtering of stop words, and synonym expansion, to complete the preprocessing of the to-be-compared text and the compared text.

7. The multi-algorithm fusion based text similarity calculation method according to claim 1, characterized in that: In (2.3) of step 2, W1 < W2, W1 and W2 are both threshold values for calling a large model deep semantic understanding similarity algorithm; when the text similarity value calculated by HanLP combined with the cosine similarity algorithm falls within the range [W1, W2], the large model deep semantic understanding similarity algorithm is called to calculate the text similarity. 8.The method for text similarity calculation based on multi-algorithm fusion according to claim 1, characterized in that: In step 2 (2.3), the calculation steps of the large model deep semantic understanding similarity algorithm are as follows: ①Deploy the large model: design the Prompt prompt word; ②Synonym expansion is performed on the to-be-compared text in (1.2) and the compared text in (1.1) respectively, and then the large model is called to calculate the similarity between the synonym expanded to-be-compared text and the compared text, to obtain a large model deep semantic understanding similarity value S sem , and the final similarity value S final =S sem is set.

9. A text similarity calculation system based on multi-algorithm fusion, characterized in that, It comprises: The project library module is used for parsing the uploaded to-be-reviewed project document, extracting key information and functional description from the parsed to-be-reviewed project document, and forming structured data; the content duplication review module is used for performing text similarity calculation on the functional description of the to-be-reviewed project document input by the project library module and each document in the historical project document set, outputting the corresponding similarity values, comparing each similarity value with a preset substantial duplication threshold, and displaying the duplicate content and comparison results according to the comparison results; and the report management module is used for converting the data results output by the content duplication review module into a review report.

10. The multi-algorithm fusion based text similarity computation system according to claim 9, wherein: The text acquisition module is used for obtaining the to-be-reviewed project document in the project library and outputting the to-be-reviewed project document to the text processing module; and the text processing module is used for structurally processing the to-be-reviewed project document input by the text acquisition module to obtain structured data.

11. The multi-algorithm fusion based text similarity computation system according to claim 10, wherein: The text processing module comprises a format conversion module, a text analysis module and an information extraction module; the format conversion module is configured to perform format conversion on the input project document under review to obtain a project document under review in a standard format; the text analysis module is configured to perform structured analysis on the project document under review in the standard format to obtain an analyzed project document under review; and the information extraction module is configured to extract key information and functional description from the analyzed project document under review and output the key information and the functional description to a database for storage.

12. The multi-algorithm fusion based text similarity computation system according to claim 9, wherein: The content review module calls HanLP in combination with a cosine similarity algorithm and a large model deep semantic understanding similarity algorithm to perform text similarity calculation on the functional description of each document in the historical project document set and the project document under review input by the project library module, obtain corresponding similarity values and output the same, and then compare each similarity value with a preset substantial repetition threshold value, and display the repeated content and the comparison result according to the comparison result.

Images