A five-dimensional context semantic feature project application form duplicate checking and sorting method

Abstract

The application relates to a five-dimensional context semantic feature project application form duplicate checking and sorting method, and relates to the field of natural language processing. The application is aimed at solving the problems of low duplicate checking accuracy and efficiency and the inability to review duplicate checking results of the existing project application form document duplicate checking method. The application comprises the following steps: establishing a dimension baseline standardization module, a five-dimensional uneven rewriting detection module and a cliff truncation symmetric fine judgment module; obtaining the mean value of the historical application form library and the to-be-judged application form in the historical library in terms of the dimension semantic similarity and the standard deviation of the historical library in terms of the current dimension semantic similarity, filling in a five-dimensional definition table, taking the five-dimensional definition table as input, sequentially calling the dimension baseline standardization module and the five-dimensional uneven rewriting detection module to sort the whole library, obtaining a whole library sorting result list, taking the whole library sorting result as the input of the cliff truncation symmetric fine judgment module; and obtaining a duplicate judgment result list based on the output of the cliff truncation symmetric fine judgment module. The application is used for technical document duplicate checking.

Description

Technical Field

[0001] This invention relates to the field of natural language processing, and in particular to a method for deduplication and ranking of project application documents based on five-dimensional contextual semantic features. Background Technology

[0002] Research and engineering institutions accumulate thousands, even tens of thousands, of project applications over time. If a new application substantially overlaps with previously approved or submitted projects in the database in terms of research path, it will result in redundant investment and a waste of scientific and technological resources. Therefore, it is essential to screen out such duplicate applications before project approval. Project applications are lengthy and have a fixed structure; what truly determines whether an application is duplicated is not the wording, but rather the research path elements: the problem domain it addresses, the research objectives it sets, the chosen technical route, the research content it arranges, and the functional requirements it promises. The same set of research path elements can be used to write multiple seemingly disparate applications with completely different wording, chapter order, and terminology. Therefore, it is crucial to move beyond literal comparison and quickly compare new applications with the historical database at the level of research path elements, providing judgment criteria that can be reviewed point by point during the review process.

[0003] Currently, the technical methods commonly used in the industry for document duplication checking mainly include the following categories: 1. Literal plagiarism detection methods based on repeated character strings and word frequency vectors, using the number of consecutive repeated characters or the cosine of word frequency as the duplication criterion. The main limitation of this type of method is that the criterion is entirely based on the surface text. The author can reduce the duplication rate below the threshold simply by replacing synonyms, adjusting word order, splitting long sentences, and rearranging chapters, without changing the research path elements at all; it is completely undetectable to changes such as replacing a certain method with a closely related one, thus resulting in low accuracy. 2. Full-text single-vector semantic plagiarism detection methods, which encode the entire application into a semantic vector and then compare it. The main limitations of this type of method are: application documents can be tens of thousands of words long, with a high proportion of formatted content such as budget, institution introduction, personnel composition, and schedule. Averaging the entire text into a single vector dilutes the small amount of content that truly reflects the research path, severely weakening the comparison signal and resulting in low plagiarism detection accuracy. Furthermore, a single vector can only provide a total score, failing to pinpoint the specific part where the duplication occurs, making it impossible to verify during the review process. III. Plagiarism detection methods based on topic models and keyword sets extract the document's topic distribution or keyword set for comparison. The main limitation of this method is that the granularity of topics and keywords is too coarse. Applications from the same institution are typically concentrated in a few business areas with highly homogeneous topics. The topic similarity between any two documents in the database is generally very high, and this criterion almost completely loses its distinguishing power within a homogeneous database, resulting in low plagiarism detection accuracy. IV. Manual review and comparison methods involve review experts judging each new application to determine whether it duplicates an existing project. The main limitations of this method are: faced with thousands of historical databases, experts cannot exhaustively compare them but can only conduct random checks, resulting in a high rate of missed detections and low accuracy in plagiarism detection; moreover, manual evaluation is a large workload and requires a lot of time, resulting in low efficiency in plagiarism detection; at the same time, this method leaves no trace in the judgment process, making it impossible to verify the plagiarism detection results. Summary of the Invention

[0004] This invention addresses the problems of low accuracy and efficiency, and the inability to verify plagiarism results in existing project application document plagiarism detection methods. It proposes a five-dimensional contextual semantic feature-based method for ranking project application documents for plagiarism detection.

[0005] A method for deduplication and ranking of project application documents using five-dimensional contextual semantic features, comprising the following steps:

[0006] S1. Create a historical application database HisList, pending application NewApp, a five-dimensional definition table DimTable, and a duplicate judgment result list CFResultList, and initialize the duplicate judgment result list CFResultList to an empty list;

[0007] S2. Using the five-dimensional semantic vector corresponding to NewApp, the five-dimensional semantic vector of a historical application, and DimTable to form the baseline normalized input BXBZInput, establish the dimensional baseline normalization module BXBZModel. The input of BXBZModel is the baseline normalized input BXBZInput, and the output of BXBZModel is the baseline normalized output BXBZOutput.

[0008] BXBZOutput includes a five-dimensional standardized deviation list, BXBZZList.

[0009] S3. Construct the imbalance detection input BJHInput using the five-dimensional standardized deviation list BXBZZList, and establish the five-dimensional imbalance rewrite detection module BJHModel. The input of BJHModel is the imbalance detection input BJHInput, and the output of BJHModel is the imbalance detection output BJHOutput.

[0010] BJHOutput includes: BJHUneven (dimensional imbalance), and BJHGapList (a list of low-skewness dimension numbers).

[0011] S4. Construct the cliff truncation symmetry precision judgment module DYJDModel. The input of DYJDModel is the cliff precision judgment input DYJDInput, and the output of DYJDModel is the cliff precision judgment output DYJDOutput.

[0012] DYJDInput includes: BJHUneven (dimensional imbalance) and BJHGapList (low deviation dimension number list);

[0013] S5. Obtain the mean (DMBase) of the semantic similarity of the previous dimension in the historical database and the standard deviation (DMStd) of the semantic similarity of the current dimension in the historical database for the historical application database and the application to be judged. Fill DMBase and DMSt into the five-dimensional definition table DimTable. Then, using the five-dimensional definition table as input, sequentially call the dimension baseline standardization module BXBZModel and the five-dimensional imbalance rewriting detection module BJHModel to sort the entire database and obtain the full database sorting result list CFRankList. Finally, call the cliff truncation symmetric precision judgment module DYJDModel, using CFRankList as input to DYJDModel, and output DYJDOutput. Assemble CFResultList based on DYJDOutput, CFRankList, and HisList.

[0014] S6. Use CFResultList to output the duplicate detection results and provide suggestions.

[0015] Furthermore, in S1, the following steps are taken: establishing a historical application database HisList, pending application NewApp, a five-dimensional definition table DimTable, and a duplicate determination result list CFResultList.

[0016] Each record in the HisList historical application database corresponds to a historical project application and includes the following fields: application number (HID), project name (HTitle), application year (HYear), full text of the application (HText), and the five-dimensional contextual semantic feature (HFeatList) of the current application.

[0017] Each element in HFeatList corresponds to a dimension and contains the following fields: dimension number FDDim, extracted text for the current dimension FDText, and semantic vector for the current dimension FDVec; HFeatList is initialized as an empty list;

[0018] The pending application form (NewApp) includes the following fields: project name, application year, full text of the application form, and five-dimensional contextual semantic features of the current application form;

[0019] The five-dimensional contextual semantic features of the current application form include the following fields: dimension number, extracted text of the current dimension, and semantic vector of the current dimension;

[0020] Each record in the five-dimensional definition table DimTable contains the following fields: Dimension ID DMID, Dimension Name DMName, Extraction Prompt Template for the Current Dimension DMPrompt, Mean of Semantic Similarity of the Current Dimension in the Historical Database DMBase, Standard Deviation of Semantic Similarity of the Current Dimension in the Historical Database DMStd, and Prior Weight of the Current Dimension DMWeight.

[0021] Both DMBase and DMStd are initialized to 0;

[0022] The duplicate detection result list CFResultList includes the following fields:

[0023] The following data is included: historical application number (CRHID), project name (CRTitle), application year (CRYear), five-dimensional original semantic similarity between the pending application and the historical application (CRSimList), five-dimensional standardized deviation (CRZList), dimensional imbalance (CRUneven), list of dimensions with deviation below the median (CRGapList), positive order judgment score (CRScoreF), negative order judgment score (CRScoreR), final duplication score (CRScore), judgment instability (CRStab), manual review marker (CRCheck), five-dimensional duplication details and original evidence from both parties (CRDetail), and duplication judgment marker (CRDup).

[0024] In this context, CRCheck = 1 indicates that manual review is required, and CRCheck = 0 indicates that manual review is not required; CRDup = 1 indicates that the application is considered a duplicate, and CRDup = 0 indicates that the application is not considered a duplicate.

[0025] Furthermore, in S2, the baseline normalization input BXBZInput is composed of the five-dimensional semantic vector corresponding to NewApp, the five-dimensional semantic vector of a historical application, and DimTable. A dimensional baseline normalization module BXBZModel is then established. The input to BXBZModel is the baseline normalization input BXBZInput, and the output of BXBZModel is the baseline normalization output BXBZOutput. Specifically:

[0026] S201. Using the five-dimensional semantic vector of the application to be judged, the five-dimensional semantic vector of a historical application, and the five-dimensional definition table, a baseline standardized input BXBZInput is formed, and a dimension baseline standardized module BXBZModel is established. The input of BXBZModel is the baseline standardized input BXBZInput.

[0027] The five-dimensional semantic vectors of the pending application and the historical application both include the following fields: dimension number FDDim, extracted text of the current dimension FDText, and semantic vector of the current dimension FDVec;

[0028] When the dimension numbers in the five-dimensional definition table are 1-5, the corresponding dimension names are: problem domain, research objective, technical route, research content, and functional requirements, respectively.

[0029] S202. Initialize the baseline standardized output BXBZOutput's five-dimensional original semantic similarity list BXBZSimList = empty list, initialize the five-dimensional standardized deviation list BXBZZList = empty list, initialize the comprehensive ranking score BXBZScore = 0; initialize the deviation weighted sum BXBZSum = 0; initialize the weighted sum BXBZWSum = 0.

[0030] S203. Initialize the dimension traversal counter BXBZCounter=1;

[0031] S204, Standardized first temporary vector BXBZTemp1 = semantic vector of the BXBZCounter dimension of the application to be judged; Standardized second temporary vector BXBZTemp2 = semantic vector of the BXBZCounter dimension of the current historical application;

[0032] S205, Standardized Third Temporary Storage Add BXBZTemp3 to BXBZSimList;

[0033] in, It is the vector magnitude. It is the dot product symbol, and EPS is the offset constant;

[0034] S206, Standardize the 4th temporary storage BXBZTemp4=(BXBZTemp3-DMBase) / (DMStd+EPS), and add BXBZTemp4 to BXBZZList;

[0035] Wherein, DMBase is the mean of the semantic similarity of the current dimension in the historical database, and DMStd is the standard deviation of the semantic similarity of the current dimension in the historical database;

[0036] S207, Let the weighted sum of deviations be BXBZSum = BXBZSum + BXBZTemp4 × DMWeight; Let the weighted sum be BXBZWSum = BXBZWSum + DMWeight;

[0037] Wherein, DMWeight is the prior weight of the current dimension;

[0038] S208. Set BXBZCounter = BXBZCounter + 1; if BXBZCounter is less than or equal to DimNum, go to S204; otherwise, go to S209.

[0039] S209. Set BXBZScore = BXBZSum / (BXBZWSum + EPS);

[0040] S210. Output BXBZOutput as the result of BXBZModel.

[0041] Furthermore, in S3, the imbalance detection input BJHInput is constructed using the five-dimensional standardized deviation list BXBZZList, and a five-dimensional imbalance rewrite detection module BJHModel is established. The input of BJHModel is the imbalance detection input BJHInput, and the output of BJHModel is the imbalance detection output BJHOutput. Specifically:

[0042] S301. Construct the imbalance detection input BJHInput using the five-dimensional standardized deviation list BXBZZList, and establish the five-dimensional imbalance rewrite detection module BJHModel. The input of BJHModel is the imbalance detection input BJHInput.

[0043] S302. Initialize the imbalance detection output BJHOutput: Dimension imbalance degree BJHUneven=0, Low deviation dimension number list BJHGapList=empty set, Imbalance flag BJHFlag=0;

[0044] S303, Imbalanced First Temporary List BJHTemp1 = List obtained by sorting the five-dimensional standardized deviations in BJHInput from largest to smallest;

[0045] S304, the unbalanced second temporary storage BJHTemp2 = the average of the first GWN values ​​in BJHTemp1;

[0046] Wherein, GWN is the preset number of high-scoring dimensions;

[0047] S305, the median of the five-dimensional standardized deviation in the unbalanced third temporary storage BJHTemp3=BJHInput;

[0048] S306, Set the dimensional imbalance degree BJHUneven=BJHTemp2-BJHTemp3;

[0049] S307. If BJHUneven is greater than GXYZ and BJHTemp2 is greater than GWYZ, then go to S308; otherwise go to S309.

[0050] Wherein, GXYZ is the dimensional imbalance threshold, and GWYZ is the lower limit of high-scoring dimensional deviation;

[0051] S308. Set BJHFlag=1; Determine whether the standardized deviation value of each dimension of the five-dimensional standardized deviation list BXBZZList in BJHInput is less than BJHTemp3. If the standardized deviation value of the current dimension is less than BJHTemp3, add DMID in the DimTable corresponding to the current dimension to BJHGapList and go to S309; ​​otherwise, go directly to S309.

[0052] S309. Output BJHOutput as the result of BJHModel.

[0053] Furthermore, in S4, the construction of the cliff truncation symmetry precision judgment module DYJDModel, the input of DYJDModel is the cliff precision judgment input DYJDInput, and the output of DYJDModel is the cliff precision judgment output DYJDOutput, specifically as follows:

[0054] S401. Establish the cliff truncation symmetry precision judgment module DYJDModel. The input of DYJDModel is the cliff precision judgment input DYJDInput.

[0055] The cliff-edge precision judgment input DYJDInput includes: a list of historical applications sorted in descending order, five-dimensional extracted text of the historical application list, five-dimensional extracted text of the application to be judged, a list of dimensional imbalance of document pairs, and a list of low deviation dimension numbers of document pairs.

[0056] S402, Initialize the candidate set truncation position of the cliff fine judgment output DYJDOutput DYJDCut=JMin, fine judgment result list DYJDList=empty list; Initialize the maximum drop DYJDMaxGap=0; Initialize the drop traversal counter DYJDCounter1=JMin;

[0057] Among them, JMin is the minimum size of the candidate set;

[0058] S403. Fine-tune the first temporary storage quantity DYJDTemp1 = the DYJDCounter1 item minus the DYJDCounter1+1 item in the comprehensive sorting score in descending order; if DYJDTemp1 is greater than DYJDMaxGap, then set DYJDMaxGap=DYJDTemp1 and set DYJDCut=DYJDCounter1.

[0059] Wherein, the first temporary storage value DYJDTemp1 represents the sorting score difference at the current position;

[0060] S404. Set DYJDCounter1 = DYJDCounter1 + 1; if DYJDCounter1 is less than or equal to JMax and DYJDCounter1 is less than HisNum, then go to S403; otherwise, go to S405.

[0061] HisNum is the total number of historical applications, and JMax is the maximum size of the candidate set.

[0062] S405. Initialize the candidate traversal counter DYJDCounter2=1;

[0063] S406, Fine-grained judgment of the second temporary text DYJDTemp2 = paired judgment prompt words;

[0064] The paired judgment prompts are obtained using the five-dimensional extracted text of the application to be judged and the second candidate application (DYJDCounter2).

[0065] S407, Let the third temporary storage quantity of the refined judgment, DYJDTemp3, be the ascending score; let the eighth temporary storage text of the refined judgment, DYJDTemp8, be the ascending five-dimensional duplicate point details and the original evidence from both sides;

[0066] The orthogonal score was obtained by combining the orthogonal five-dimensional repetition details with the original evidence from both sides, using the large language model specified by the second temporary text DYJDTemp2 and LLMName for precise analysis.

[0067] S408. Obtain the fourth temporary text DYJDTemp4 for precise judgment using DYJDTemp2, and obtain the fifth temporary text DYJDTemp5 for precise judgment based on the fourth temporary text DYJDTemp4. Specifically:

[0068] The fourth temporary text for detailed judgment, DYJDTemp4, is a prompt word obtained by swapping the order of the pending application and the candidate application in DYJDTemp2.

[0069] The fifth temporary value, DYJDTemp5, is determined by reversing the order score.

[0070] The reverse score is obtained in the following way:

[0071] Keeping the sampling temperature constant, the fourth temporary text DYJDTemp4 is input into the large language model specified by LLMName for inference. The model output is parsed, and the repetition scores of each of the five dimensions are extracted. The repetition scores of each dimension are used to perform a weighted average calculation with the DMWeight of the corresponding dimension to obtain the weighted average. The weighted average is used as the reverse score.

[0072] S409, the sixth temporary storage quantity for precise judgment is DYJDTemp6=(DYJDTemp3+DYJDTemp5) / 2; the seventh temporary storage quantity for precise judgment is DYJDTemp7=ABS(DYJDTemp3-DYJDTemp5);

[0073] Where ABS represents the absolute value;

[0074] S410. If DYJDTemp7 is greater than the symmetric judgment consistency tolerance DCYZ, then set the manual review flag of the current candidate application to 1; otherwise, set it to 0. Add an element to DYJDList, which includes: the application number of the current candidate application, setting the forward score to DYJDTemp3, the reverse score to DYJDTemp5, the final score to DYJDTemp6, the stability difference to DYJDTemp7, the manual review flag, and the details of the five-dimensional duplication points given by the forward judgment and the original evidence from both parties to DYJDTemp8.

[0075] S411. Set DYJDCounter2 = DYJDCounter2 + 1; if DYJDCounter2 is less than or equal to DYJDCut, go to S406; otherwise, go to S412.

[0076] S412. Sort the data in DYJDList from highest to lowest according to the final score; output DYJDOutput as the result of DYJDModel.

[0077] Furthermore, the pairwise determination prompt words in S406 are obtained in the following way:

[0078] First, the five-dimensional extracted texts of the application to be judged and the second candidate application are arranged side by side according to their dimensions;

[0079] Then, the list of dimension imbalance and low deviation dimension numbers of the current document pair is appended into the prompt words, and an instruction is added to prompt the large model to focus on comparing the above low deviation dimensions.

[0080] Then, add model task instructions: require the model to output a repetition score of 0-100 and the corresponding judgment reasons for each of the five dimensions: problem domain, research objective, technical route, research content, and functional requirements.

[0081] Then, additional evidence output requirements are added: the model is required to extract original text statements that are repeated in the application to be judged and the second candidate application in DYJDCounter2 under each dimension, as evidence for judgment;

[0082] Then, an output length constraint is added: the total number of words in the entire output content of the model is limited to no more than LLMLen;

[0083] Finally, the prompt words, model task instructions, evidence output requirements, and output length are concatenated into paired decision prompt words.

[0084] Further, the positive order score in S407 is obtained in the following way: read the second temporary text DYJDTemp2, use the large language model specified by LLMName, set the model sampling temperature to LLMTemp, input DYJDTemp2 into the large language model specified by LLMName to perform inference, parse the model output results, extract the repetition scores corresponding to each of the five dimensions, use the repetition scores of each dimension and the DMWeight of the corresponding dimension to perform weighted average calculation, obtain the weighted average, and use the weighted average as the positive order score;

[0085] The details of the five-dimensional repetition points and the original text evidence from both parties were obtained as follows: The second temporary text DYJDTemp2 of the fine-judgment was read, the large language model specified by LLMName was used, the model sampling temperature was set to LLMTemp, DYJDTemp2 was input into the large language model specified by LLMName to perform inference, and the repetition scores corresponding to the five dimensions of problem domain, research objectives, technical route, research content and functional requirements, the judgment reasons of the current dimension, and the repetitive sentences extracted from the original text of the application to be judged and the historical applications were extracted from the model output results as the details of the five-dimensional repetition points and the original text evidence from both parties.

[0086] Further, in step S5, the mean (DMBase) of the semantic similarity of the previous dimension in the historical database and the standard deviation (DMStd) of the semantic similarity of the current dimension in the historical database are obtained from the historical application database and the application to be judged. DMBase and DMSt are filled into the five-dimensional definition table DimTable. Then, the five-dimensional definition table is used as input to sequentially call the dimension baseline standardization module BXBZModel and the five-dimensional imbalance rewriting detection module BJHModel to sort the entire database and obtain the full database sorting result list CFRankList. Finally, the cliff truncation symmetric precision judgment module DYJDModel is called, and CFRankList is used as input to DYJDModel to output DYJDOutput. Based on DYJDOutput, CFRankList and HisList, CFResultList is assembled, specifically as follows:

[0087] S501. Initialize and set the feature extraction counter CFCounter1=1;

[0088] S502, Check the first temporary variable CFTemp1 for duplicates and retrieve the CFCounter1 record from HisList; if HFeatList of CFTemp1 is not empty, go to S506; otherwise, go to S503.

[0089] S503. Initialize and set the dimension extraction counter CFCounter2=1;

[0090] S504, the second temporary text for deduplication, CFTemp2, is extracted using the large language model specified by LLMName at the sampling temperature LLMTemp, with DMPrompt of the CFCounter2 record in DimTable as the prompt word, and HText of CFTemp1 as the input. The output of the large language model specified by LLMName is the text extracted in the current dimension.

[0091] The third temporary vector for deduplication, CFTemp3, is the EmbDim-dimensional semantic vector obtained by encoding CFTemp2 using the semantic vector model specified by EmbName.

[0092] Add an element to HFeatList of CFTemp1, with FDDim=CFCounter2, FDText=CFTemp2, and FDVec=CFTemp3 for the new element;

[0093] S505, Let CFCounter2 = CFCounter2 + 1; If CFCounter2 is less than or equal to DimNum, then go to S504; otherwise, go to S506.

[0094] S506. Set CFCounter1 = CFCounter1 + 1; if CFCounter1 is less than or equal to HisNum, go to S502; otherwise, go to S507.

[0095] S507. NewApp extracts five-dimensional text from the application for judgment and encodes it into a semantic vector, which is then filled into NewApp's HFeatList.

[0096] S508. Initialize baseline statistics counter CFCounter3=1;

[0097] S509. Randomly select SampleN pairs of distinct historical applications from HisList, calculate the cosine similarity of the two semantic vectors of each pair of historical applications in the third dimension of CFCounter3, fill the mean of the similarity of SampleN into DMBase of the third record of DimTable, and fill the standard deviation of the similarity of SampleN into DMSted.

[0098] Where SampleN is the baseline sampling pair number;

[0099] S510, Set CFCounter3 = CFCounter3 + 1; If CFCounter3 is less than or equal to DimNum, go to S509; otherwise, go to S511.

[0100] S511. Initialize the full database sorting counter CFCounter4 = 1; Initialize the full database sorting result list CFRankList = an empty list;

[0101] S512, Call BXBZModel, the input of BXBZModel is BXBZInput=NewApp and the five-dimensional semantic vector of the CFCounter4 record of HisList, and the DimTable that has been filled into the baseline, the output of BXBZModel is BXBZOutput, let the fourth temporary variable for deduplication CFTemp4=BXBZOutput.

[0102] S513. Call BJHModel. The input of BJHModel is BJHInput=CFTemp4's BXBZZList. The output of BJHModel is BJHOutput. Let the 5th temporary variable for deduplication, CFTemp5, be equal to BJHOutput.

[0103] S514. Using the fifth temporary variable CFTemp5 and the fourth temporary variable CFTemp4, obtain the sixth temporary variable CFTemp6. Add an element to CFRankList. The newly added element includes: the current historical application number, BXBZSimList and BXBZZList of CFTemp4, BJHUneven and BJHGapList of CFTemp5, and CFTemp6.

[0104] S515. Let CFCounter4 = CFCounter4 + 1; if CFCounter4 is less than or equal to HisNum, go to S512; otherwise, go to S516.

[0105] S516. Sort CFRankList from highest to lowest based on its overall ranking score;

[0106] The value of the comprehensive ranking score is CFTemp6;

[0107] S517. Call DYJDModel. The input of DYJDModel is DYJDInput = the sorted CFRankList, the five-dimensional extracted text of NewApp and each candidate application, and the list of dimension imbalance and low deviation dimension numbers of each document pair. The output of DYJDModel is DYJDOutput. Let the 7th temporary variable for plagiarism detection, CFTemp7, = DYJDOutput.

[0108] A document pair consists of a pending application (NewApp) and a candidate application;

[0109] S518. Set the result summary counter CFCounter5=1;

[0110] S519. Extract the CFCounter5 element of DYJDList from CFTemp7 as the fine-judgment record; based on the historical application number HID in the fine-judgment record, find the sorted record with the number HID in CFRankList; use the HID of the sorted record to search for the CRHID, CRTitle, and CRYear of the current historical application in the HisList historical application database, and extract the five-dimensional original similarity CRSimList, five-dimensional standardized deviation CRZList, dimensional imbalance CRUneven, and low deviation dimension number list CRGapList from the sorted record; extract the positive order score CRScoreF, negative order score CRScoreR, final duplication score CRScore, judgment stability difference CRStab, manual review mark CRCheck, and five-dimensional duplication details and original evidence from both parties CRDetail from the fine-judgment record; if CRScore is greater than or equal to CFYZ, set CRDup=1, otherwise set CRDup=0; assemble all the above fields into a new result element and append the new result element to the result list CFResultList;

[0111] S520, set CFCounter5 = CFCounter5 + 1; if CFCounter5 is less than or equal to len(CFTemp7's DYJDList), then go to S519, otherwise output CFResultList.

[0112] Furthermore, the sixth temporary deduplication value CFTemp6 in S514 is obtained in the following way:

[0113] ;

[0114] in, It is the overall ranking score in CFTemp4. It is an imbalance marker in CFTemp5. It is the dimensionality imbalance in CFTemp5, and JCXS is the imbalance additive coefficient.

[0115] Furthermore, in step S6, the method of using CFResultList to output the duplicate detection result and provide suggestions is as follows:

[0116] S601. If CFResultList is empty, output that no duplicate item of this application was found in the history database and proceed to S606; otherwise, proceed to S602.

[0117] S602. Initialize the output counter OutCounter=1;

[0118] S603. Retrieve the OutCounter record from CFResultList and output the following information: current historical application number CRHID, project name CRTitle, application year CRYear, five-dimensional original semantic similarity CRSimList, five-dimensional standardized deviation CRZList, dimensional imbalance CRUneven, list of dimensions with deviations below the median CRGapList, positive order score CRScoreF, negative order score CRScoreR, final duplicate score CRScore, stability difference CRStab, manual review marker CRCheck, five-dimensional duplicate details, original evidence from both parties CRDetail, and duplicate judgment marker CRDup.

[0119] S604. Set OutCounter = OutCounter + 1; if OutCounter is less than or equal to len(CFResultList), go to S603; otherwise, go to S605.

[0120] Where len() is the list length function;

[0121] S605. Record the number of records in CFResultList where CRDup equals 1 as CFDupN, the number of records where CRCheck equals 1 as CFChkN, and the maximum value of CRScore among all records as CFMaxScore. If CFDupN is greater than 0, output: This application constitutes substantial duplication of the research path with CFDupN projects in the historical database, with the highest duplication score being CFMaxScore. It is recommended to focus on this during the project approval process. Otherwise, output: This application does not constitute substantial duplication with any project in the historical database. If CFChkN is greater than 0, output: Among them, CFChkN judgments are at the boundary, and the difference between positive and negative scores exceeds the tolerance, requiring manual review. Otherwise, no additional output is added.

[0122] The beneficial effects of this invention are as follows:

[0123] (1) This invention first calculates the mean and standard deviation of the similarity of each dimension by sampling and pairing in the historical database. Then, it standardizes the similarity of the document pairs to be judged according to the mean and standard deviation of that dimension. This invention takes into account that the applications of the same unit are concentrated in a few business directions. The problem domain dimension is highly homogeneous in the database, and the similarity of any two documents is generally very high. The high similarity on this dimension does not carry the information for judging duplication. However, the technical route dimension is scattered in the database and has a generally low similarity. Only the high similarity on this dimension has the significance of judging duplication. Therefore, the duplication judgment of this invention is based on the degree of deviation of the similarity of a certain dimension from the general level in the database rather than its absolute size. In this way, the contribution of the two to the ranking is correctly distinguished, breaking through the limitation that the topic and keyword criteria lose the ability to distinguish in the homogeneous database, and improving the accuracy of duplication detection.

[0124] (2) This invention takes into account the mutual constraints of the five dimensions of an application in the writing process. The problem domain defines the scope of the research objectives, the technical route supports the research content, and the textual description of the technical route is the most specific and the easiest to be detected by word-for-word plagiarism detection. Therefore, it is the first part to be changed when rewriting to avoid plagiarism. However, the dimensions that reflect the essence of the research, such as the problem domain, research objectives, research content and functional requirements, cannot be changed without changing the project itself. Therefore, the document rewritten to avoid plagiarism will inevitably present an unbalanced structure with most dimensions having a high degree of deviation and some dimensions being stretched. Even if two independently written applications belong to the same field, their similarity in each dimension should be at a moderate level overall, without such structural discrepancies. Therefore, the five-dimensional imbalance rewriting detection module of this invention uses the calculable statistical features left by rewriting avoidance as the detection basis. It takes the difference between the mean of the highest deviation of the five dimensions and the median of the vector to measure the degree of imbalance, and marks the dimensions below the median. This breaks through the limitations of literal plagiarism detection being easily avoided by synonym replacement, word order adjustment and close-kin method replacement, and improves the plagiarism detection accuracy.

[0125] (3) Considering that there is a significant gap between the ranking scores of historical applications that are duplicates of the application to be judged and the other applications, the cliff-cut symmetrical precision judgment module of this invention does not cut off the candidate set with a fixed number of candidates, nor does it make a decision based on a single judgment. After arranging the ranking scores in descending order, it cuts off at the point where the difference between two adjacent items is the largest. This allows the size of the candidate set to automatically expand and contract with the actual duplication situation, ensuring that no duplicate items are missed when there are many duplicate items and that the judgment cost is not wasted when there are few duplicate items. Furthermore, because the pairwise judgment of the large language model is sensitive to the presentation order of the two documents, each candidate pair is judged once in ascending order and once in descending order. The average of the two scores is used to determine the score, and the difference between the two scores is used to measure the judgment stability. Those with a difference exceeding the tolerance are marked as requiring manual review. The model is also required to extract the duplicate sentences in the original text of both parties for each dimension as evidence, which breaks through the limitations of manual comparison, which cannot exhaustively list duplicates, leave no trace of judgment, and cannot review the conclusions item by item at the review meeting. At the same time, this invention reduces the workload of manual evaluation and improves the efficiency of duplication detection. Attached Figure Description

[0126] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0127] This invention utilizes a large language model to extract the contextual semantic features of each application into five dimensions: problem domain, research objectives, technical route, research content, and functional requirements. These features are then encoded dimension-by-dimensionally using a semantic vector model. This process constructs a dimensional baseline standardization module, a five-dimensional imbalance rewriting detection module, and a cliff-trimmed symmetric precision judgment module. The dimensional baseline standardization module first uses the mean and standard deviation of semantic similarity for each dimension from a sampled pair in the historical database as a baseline. Then, it subtracts the mean of each dimension from the similarity of the document pair to be judged and divides it by the standard deviation of that dimension to convert it into a dimensionless standardized deviation. This deviation is then further analyzed using the five-dimensional imbalance rewriting detection module. The balanced rewriting detection module takes the difference between the mean of the highest-performing components of the five-dimensional deviation and the median of the vector as the dimensional imbalance degree, and marks the separated dimensions accordingly. Using a cliff-cutting symmetric precision judgment module, the candidate set is truncated at the point of maximum score drop. For each candidate pair, a large language model judgment is performed twice, once in ascending order and once in descending order, and the mean score is used to determine the value, while the difference is used to measure stability. Finally, a list of historical applications sorted by duplication score is output, along with their five-dimensional scores, details of duplication points, and original evidence from both parties. This achieves a duplication ranking system within a homogeneous application database that transcends literal comparison, resists rewriting circumvention, and allows for item-by-item verification. The invention will now be described in conjunction with specific implementation methods.

[0128] Specific implementation method one: as follows Figure 1 As shown, the specific process of the five-dimensional contextual semantic feature-based project application deduplication and ranking method in this embodiment is as follows:

[0129] S1. Establish the HisList historical application database, NewApp pending applications, and DimTable five-dimensional definition table; construct the name parameters for the large language model, the name parameters for the semantic vector model, and control parameters; establish the CFResultList list for duplicate judgment results; and complete data initialization. Specifically:

[0130] S101. Input the historical application database HisList. Each record in HisList corresponds to a historical project application and includes the following fields:

[0131] HID: Application number, a positive integer;

[0132] HTitle: Project Name;

[0133] HYear: Year of Reporting;

[0134] HText: The full text of the application form;

[0135] HFeatList: The five-dimensional contextual semantic features of the current application, initialized as an empty list;

[0136] Each element in HFeatList corresponds to a dimension and contains the following fields: dimension number FDDim, extracted text for the current dimension FDText, and semantic vector for the current dimension FDVec;

[0137] S102, Enter the pending judgment application form NewApp;

[0138] The application to be judged includes the following fields: project name, application year, full text of the application, and five-dimensional contextual semantic features of the current application;

[0139] The five-dimensional contextual semantic features of the current application are initialized as an empty list, including the following fields: dimension number, extracted text of the current dimension, and semantic vector of the current dimension;

[0140] S103. Construct a five-dimensional definition table DimTable. DimTable will always contain five records, each containing the following fields:

[0141] DMID: Dimension ID, ranging from 1 to 5;

[0142] DMName: Dimension name;

[0143] When the dimension numbers are 1-5, the corresponding dimension names are: problem domain, research objective, technical route, research content, and functional requirements, respectively.

[0144] DMPrompt: Extraction prompt template for the current dimension;

[0145] DMPrompt requires large language models to summarize a dimension based solely on the main body of the application in two sentences. The first sentence is a specific statement describing the current dimension in the application, and the second sentence is a general summary, replacing the proprietary method names and technology names appearing in the first sentence with the names of the method families or technology families to which they belong. The two sentences should not exceed 200 words in total. The reason for setting up the general summary is that replacing a method with its close relative method will change the specific statement but will not change the method family to which it belongs. Therefore, the general summary can keep the semantic vectors of such replacements close.

[0146] DMBase: The mean of the semantic similarity of the current dimension in the historical database, initialized to 0;

[0147] DMStd: The standard deviation of the semantic similarity of the current dimension in the historical database, initialized to 0;

[0148] DMWeight: The prior weight of the current dimension; the default value for all five dimensions is 1.

[0149] S104. Construct the large language model name parameter LLMName, which defaults to "Qwen2.5-72B-Instruct". Construct the sampling temperature LLMTemp, which defaults to 0. Setting the value to zero ensures that the same input always produces the same output. Construct the maximum output length in a single iteration LMLen, which defaults to 2048 (words).

[0150] In this step, Qwen2.5-72B-Instruct is an open-source Chinese instruction model that can be deployed locally;

[0151] S105. Construct the semantic vector model name parameter EmbName. The default value of EmbName is "BGE-M3". Input the semantic vector dimension EmbDim. The default value of EmbDim is 1024.

[0152] BGE-M3 is an open-source Chinese semantic vector model that can be deployed locally. Its maximum text length that can be encoded in a single pass is 8192 words, which is sufficient to cover the extracted text in a single dimension.

[0153] S106. Input the number of baseline sampling pairs, SampleN. The default value of SampleN is 20000 (pairs).

[0154] Since the standard error of the sample mean is equal to the population standard deviation divided by the arithmetic square root of the sample size, the estimation error of the baseline mean based on this default value is reduced to less than one percent of the current dimension standard deviation, which is much smaller than the magnitude of deviation that standardization is required to distinguish. Therefore, it is not necessary to exhaustively enumerate all pairs in the historical database. In implementation, a different value can be selected according to the database capacity and computing power.

[0155] S107. Input candidate set minimum size JMin, the default value of JMin is 5; input candidate set maximum size JMax, the default value of JMax is 50;

[0156] S108. Input the number of high-scoring dimensions GWN, the default value of GWN is 2; input the lower limit of high-scoring dimension deviation GWYZ, the default value of GWYZ is 2.0; input the dimension imbalance threshold GXYZ, the default value of GXYZ is 1.5; input the imbalance addition coefficient JCXS, the default value of JCXS is 1.0.

[0157] S109. Input the duplicate detection threshold CFYZ, the default value of CFYZ is 70 (out of 100); input the symmetry detection consistency tolerance DCYZ, the default value of DCYZ is 10 (out of 10).

[0158] S110. Establish an offset constant EPS = 0.000001 to prevent the denominator from being zero; obtain the total number of historical applications HisNum = len(HisList); obtain the total number of dimensions DimNum = len(DimTable);

[0159] Here, len() is the number of elements in the set;

[0160] S111. Create a duplicate detection result list CFResultList = an empty list; CFResultList contains the following fields:

[0161] CRHID: Historical application number;

[0162] CRTitle: Project name from previous application documents;

[0163] CRYear: The year in which the historical application was submitted;

[0164] CRSimList: The five-dimensional original semantic similarity between the pending application and the historical application;

[0165] CRZList: Five-dimensional standardized deviation;

[0166] CRUneven: Dimensional Imbalance;

[0167] CRGapList: A list of dimension numbers with deviations below the median;

[0168] CRScoreF: Ascending order score;

[0169] CRScoreR: Reverse order determination score;

[0170] CRScore: Final repeated score, which is the average of the scores in ascending and descending order;

[0171] CRStab: Determines the stability difference by taking the absolute value of the difference between the positive and negative scores;

[0172] CRCheck: Manual review flag, 1 indicates manual review is required, 0 indicates no manual review is required;

[0173] CRDetail: A detailed list of five points of overlap and original evidence from both parties;

[0174] CRDup: Duplicate flag, 1 indicates that the application is considered a duplicate, and 0 indicates that it is not considered a duplicate.

[0175] This step initializes the data and parameters. Input includes the historical application database and applications to be judged, the five-dimensional definition table, model specifications, and various thresholds. Extraction and judgment are performed using the locally deployable open-source model Qwen2.5-72B-Instruct, with the sampling temperature fixed at zero. Semantic encoding uses the open-source model BGE-M3, outputting a 1024-dimensional vector. The explicit specification of the model type and temperature ensures that all results can be reproduced by others.

[0176] S2. Using the five-dimensional semantic vector corresponding to the application to be judged, the five-dimensional semantic vector of a historical application, and the five-dimensional definition table, a baseline-standardized input BXBZInput is constructed. A dimensional baseline-standardized module BXBZModel is then established. The input to BXBZModel is the baseline-standardized input BXBZInput, and the output of BXBZModel is the baseline-standardized output BXBZOutput. Specifically:

[0177] S201. Using the five-dimensional semantic vector of the application to be judged, the five-dimensional semantic vector of a historical application, and the five-dimensional definition table, a baseline standardized input BXBZInput is formed, and a dimension baseline standardized module BXBZModel is established. The input of BXBZModel is the baseline standardized input BXBZInput.

[0178] The baseline standardized input BXBZInput includes: the five-dimensional semantic vector corresponding to the application to be judged, the five-dimensional semantic vector of a historical application, and a five-dimensional definition table;

[0179] The five-dimensional semantic vectors of the pending application and the historical application both include the following fields: dimension number FDDim, extracted text of the current dimension FDText, and semantic vector of the current dimension FDVec;

[0180] The five-dimensional definition table includes the following fields: Dimension ID DMID, Dimension Name DMName, Extraction Prompt Template DMPrompt for the current dimension, Mean of Semantic Similarity of the current dimension in the historical database DMBase, Standard Deviation of Semantic Similarity of the current dimension in the historical database DMStd, and Prior Weight DMWeight for the current dimension.

[0181] When the dimension numbers are 1-5, the corresponding dimension names are: problem domain, research objective, technical route, research content, and functional requirements, respectively.

[0182] S202. Initialize the baseline standardized output BXBZOutput's five-dimensional original semantic similarity list BXBZSimList = empty list, initialize the five-dimensional standardized deviation list BXBZZList = empty list, initialize the comprehensive ranking score BXBZScore = 0; initialize the deviation weighted sum BXBZSum = 0; initialize the weighted sum BXBZWSum = 0.

[0183] S203. Initialize the dimension traversal counter BXBZCounter=1;

[0184] S204, Standardized first temporary vector BXBZTemp1 = semantic vector of the BXBZCounter dimension of the application to be judged; Standardized second temporary vector BXBZTemp2 = semantic vector of the BXBZCounter dimension of the current historical application;

[0185] S205, Standardized Third Temporary Storage Add BXBZTemp3 to BXBZSimList;

[0186] in, It is the vector magnitude. It is the dot product symbol, and EPS is the offset constant;

[0187] S206, Standardize the 4th temporary storage BXBZTemp4=(BXBZTemp3-DMBase) / (DMStd+EPS), and add BXBZTemp4 to BXBZZList;

[0188] Wherein, DMBase is the mean of the semantic similarity of the current dimension in the historical database, and DMStd is the standard deviation of the semantic similarity of the current dimension in the historical database;

[0189] BXBZTemp4 represents the standardized deviation of the current dimension, meaning how many standard deviations the similarity of this document to the current dimension exceeds the general level in the library. According to typical values, a similarity of 0.92 in the problem domain is equivalent to only 0.67 when the mean is 0.90 and the standard deviation is 0.03 in the library, while a similarity of 0.85 in the technical route is equivalent to 0.3 when the mean is 0.40 and the standard deviation is 0.15 in the library.

[0190] S207, Let the weighted sum of deviations be BXBZSum = BXBZSum + BXBZTemp4 × DMWeight; Let the weighted sum be BXBZWSum = BXBZWSum + DMWeight;

[0191] Wherein, DMWeight is the prior weight of the current dimension;

[0192] S208. Set BXBZCounter = BXBZCounter + 1; if BXBZCounter is less than or equal to DimNum, go to S204; otherwise, go to S209.

[0193] S209. Set BXBZScore = BXBZSum / (BXBZWSum + EPS);

[0194] In this step, the weighted average of the five-dimensional standardized deviations is used as the comprehensive ranking score for the historical application.

[0195] S210. Output BXBZOutput as the result of BXBZModel.

[0196] The principle of this module is as follows: Applications from the same unit are concentrated in a few business directions, and the distribution of semantic similarity in each dimension varies greatly within the database. The problem domain dimension is highly homogeneous, and the similarity between any two documents in the database is generally very high in this dimension. However, high similarity in this dimension does not carry deduplication information. On the other hand, the technical route dimension is scattered within the database and generally has low similarity. Only high similarity in this dimension has deduplication significance. Since deduplication is truly based on the degree of deviation of the similarity in a certain dimension from the general level in the database, rather than its absolute size, the mean and standard deviation of the similarity in each dimension should be calculated by sampling and pairing documents in the database as a baseline. Then, the mean of the dimension is subtracted from the similarity of the document pair to be judged, and the result is divided by the standard deviation of the dimension to convert it into a dimensionless standardized deviation.

[0197] This step standardizes the dimensional baseline. First, the mean and standard deviation of semantic similarity for each dimension are calculated from sampled pairs in the historical database and used as the baseline. Then, the similarity of each dimension of the document pair to be judged is standardized to the deviation based on this baseline. High similarity in homogeneous dimensions such as the problem domain is automatically suppressed, while high similarity in dispersed dimensions such as technical routes is automatically amplified.

[0198] S3. Construct the imbalance detection input BJHInput using the five-dimensional standardized deviation list BXBZZList, and establish the five-dimensional imbalance rewrite detection module BJHModel. The input of BJHModel is the imbalance detection input BJHInput, and the output of BJHModel is the imbalance detection output BJHOutput. Specifically:

[0199] S301. Construct the imbalance detection input BJHInput using the five-dimensional standardized deviation list BXBZZList, and establish the five-dimensional imbalance rewrite detection module BJHModel. The input of BJHModel is the imbalance detection input BJHInput.

[0200] S302. Initialize the imbalance detection output BJHOutput: Dimension imbalance degree BJHUneven=0, Low deviation dimension number list BJHGapList=empty set, Imbalance flag BJHFlag=0;

[0201] S303, Imbalanced First Temporary List BJHTemp1 = List obtained by sorting the five-dimensional standardized deviations in BJHInput from largest to smallest;

[0202] S304, The unbalanced second temporary quantity BJHTemp2 = the mean of the first GWN values ​​in BJHTemp1, that is, the level of concentration of the document pair on the most similar dimensions.

[0203] Where GWN is the number of high-scoring dimensions; a document pair refers to a combination of a pending application and a previous application.

[0204] S305, the median of the five-dimensional standardized deviation in the unbalanced third temporary storage BJHTemp3=BJHInput;

[0205] The median is the value in the middle of a set of numbers sorted from smallest to largest. The median is used instead of the mean because the median is not affected by a few extremely low dimensions that are stretched, and thus can represent the general level of the document for the unmodified parts.

[0206] S306, Set the dimensional imbalance degree BJHUneven=BJHTemp2-BJHTemp3;

[0207] In this step, the larger the difference in dimensional imbalance, the more obvious the document's structure is, where it is highly skewed in a few dimensions but pulled apart in the other dimensions.

[0208] S307. If BJHUneven is greater than GXYZ and BJHTemp2 is greater than GWYZ, then go to S308; otherwise, go to S309.

[0209] The condition that BJHTemp2 is greater than GWYZ is added in this step because the difference between dimensions can only be attributed to rewriting avoidance when the high-scoring dimension itself has reached a level of significant deviation. Otherwise, two unrelated documents may also show a difference due to accidental fluctuations.

[0210] S308. Set BJHFlag=1; Determine whether the standardized deviation value of each dimension in the five-dimensional standardized deviation list BXBZZList of BJHInput is less than BJHTemp3. If the standardized deviation value of the current dimension is less than BJHTemp3, add DMID from the DimTable corresponding to the current dimension to BJHGapList as the separated dimension for comparison during subsequent fine-tuning; Proceed to S309; ​​Otherwise, do not perform any operation and proceed directly to S309.

[0211] S309. Output BJHOutput as the result of BJHModel.

[0212] The principle of this module is as follows: the five dimensions of an application are mutually constrained during the writing process. The problem domain defines the scope of the research objectives, the technical route supports the research content, and the five dimensions cannot be valued independently. Because the textual description of the technical route is the most specific and easiest to be detected by word-for-word plagiarism checks, it is the first part to be modified during rewriting and avoidance. However, the dimensions that reflect the essence of the research, such as the problem domain, research objectives, research content, and functional requirements, cannot be changed without altering the project itself. Therefore, the rewritten and avoided documents will inevitably present an unbalanced structure with most dimensions having extremely high deviations and some dimensions being significantly different. Furthermore, even if two independently written applications belong to the same field, the similarity of their dimensions should be at a moderate level overall, without such a structural gap. Therefore, the degree of imbalance of the five-dimensional deviation vector itself constitutes a calculable feature of rewriting and avoidance. The mean of the highest components of the vector minus the median of the vector is its measure.

[0213] This step enables the detection of five-dimensional imbalance in the rewriting process. The difference between the mean of the highest-performing components of the five-dimensional deviation and the median of the vector is taken as the dimensional imbalance degree, and dimensions below the median are marked. This is based on the principle that rewriting avoids altering only the most specific dimensions, such as technical routes, which are merely textual descriptions and cannot change the other dimensions reflecting the essence of the research, thus inevitably leaving structural discrepancies.

[0214] S4. Construct the cliff-cutting precision judgment input DYJDInput using the descending-ordered list of historical applications, the five-dimensional extracted text of the historical application list, the five-dimensional extracted text of the applications to be judged, the dimensional imbalance list of document pairs, and the low-deviation dimension number list of document pairs. Construct the cliff-cutting symmetric precision judgment module DYJDModel. The input of DYJDModel is the cliff-cutting precision judgment input DYJDInput, and the output of DYJDModel is the cliff-cutting precision judgment output DYJDOutput. Specifically:

[0215] S401. Establish the cliff truncation symmetry precision judgment module DYJDModel. The input of DYJDModel is the cliff precision judgment input DYJDInput.

[0216] The cliff-edge precision judgment input DYJDInput includes: a list of historical applications sorted in descending order, five-dimensional extracted text of the historical application list, five-dimensional extracted text of the application to be judged, a list of dimensional imbalance of document pairs, and a list of low deviation dimension numbers of document pairs.

[0217] S402, Initialize the candidate set truncation position of the cliff fine judgment output DYJDOutput DYJDCut=JMin, fine judgment result list DYJDList=empty list; Initialize the maximum drop DYJDMaxGap=0; Initialize the drop traversal counter DYJDCounter1=JMin;

[0218] Among them, JMin is the minimum size of the candidate set;

[0219] S403. Fine-tune the first temporary storage quantity DYJDTemp1 = the DYJDCounter1 item minus the DYJDCounter1+1 item in the comprehensive sorting score in descending order; if DYJDTemp1 is greater than DYJDMaxGap, then set DYJDMaxGap=DYJDTemp1 and set DYJDCut=DYJDCounter1.

[0220] Wherein, the first temporary storage value DYJDTemp1 represents the sorting score difference at the current position;

[0221] S404. Set DYJDCounter1 = DYJDCounter1 + 1; if DYJDCounter1 is less than or equal to JMax and DYJDCounter1 is less than HisNum, then go to S403; otherwise, go to S405.

[0222] HisNum represents the total number of historical applications;

[0223] S405. Initialize the candidate traversal counter DYJDCounter2=1;

[0224] S406, Fine-grained judgment of the second temporary text DYJDTemp2 = paired judgment prompt words;

[0225] The paired decision prompts are obtained through the following methods:

[0226] First, the five-dimensional extracted texts of the application to be judged and the second candidate application are arranged side by side according to their dimensions;

[0227] Then, the list of dimension imbalance and low deviation dimension numbers of the current document pair is appended into the prompt words, and an instruction is added to prompt the large model to focus on comparing the above low deviation dimensions.

[0228] Then, add model task instructions: require the model to output a repetition score of 0-100 and the corresponding judgment reasons for each of the five dimensions: problem domain, research objective, technical route, research content, and functional requirements.

[0229] Then, additional evidence output requirements are added: the model is required to extract original text statements that are repeated in the application to be judged and the second candidate application in DYJDCounter2 under each dimension, as evidence for judgment;

[0230] Then, an output length constraint is added: the total number of words in the entire output content of the model is limited to no more than LLMLen;

[0231] Finally, the prompt words, model task instructions, evidence output requirements, and output length are concatenated into paired decision prompt words;

[0232] S407, Let the third temporary storage quantity of the refined judgment, DYJDTemp3, be the ascending score; let the eighth temporary storage text of the refined judgment, DYJDTemp8, be the ascending five-dimensional duplicate point details and the original evidence from both sides;

[0233] The positive order score is obtained as follows: Read the second temporary text DYJDTemp2, use the large language model specified by LLMName, set the model sampling temperature to LLMTemp, input DYJDTemp2 into the large language model specified by LLMName to perform inference, parse the model output results, extract the repetition scores corresponding to each of the five dimensions, use the repetition scores of each dimension and the DMWeight of the corresponding dimension to perform weighted average calculation, obtain the weighted average, and use the weighted average as the positive order score;

[0234] The weighted mean is obtained by multiplying the scores of each dimension by their corresponding weights, summing the results, and then dividing by the sum of the weights.

[0235] The details of the five-dimensional repetition points and the original text evidence from both parties were obtained as follows: The second temporary text DYJDTemp2 of the fine judgment was read, the large language model specified by LLMName was used, the model sampling temperature was set to LLMTemp, DYJDTemp2 was input into the large language model specified by LLMName to perform inference, and the repetition scores corresponding to the five dimensions of problem domain, research objectives, technical route, research content and functional requirements, the judgment reasons of the current dimension, and the repetitive sentences extracted from the original text of the application to be judged and the historical applications were extracted from the model output results as the details of the five-dimensional repetition points and the original text evidence from both parties.

[0236] S408. Obtain the fourth temporary text DYJDTemp4 for precise judgment using DYJDTemp2, and obtain the fifth temporary text DYJDTemp5 for precise judgment based on the fourth temporary text DYJDTemp4. Specifically:

[0237] The fourth temporary text for detailed judgment, DYJDTemp4, is a prompt word obtained by swapping the order of the pending application and the candidate application in DYJDTemp2.

[0238] The fifth temporary value, DYJDTemp5, is determined by reversing the order score.

[0239] The reverse score is obtained in the following way:

[0240] Keeping the sampling temperature constant, the fourth temporary text DYJDTemp4 is input into the large language model specified by LLMName for inference. The model output is parsed, and the repetition scores of each of the five dimensions are extracted. The repetition scores of each dimension are used to perform a weighted average calculation with the DMWeight of the corresponding dimension to obtain the weighted average. The weighted average is used as the reverse score.

[0241] The reverse order determination is used to measure the sensitivity of the presentation order. Its five-dimensional details are not stored separately. The duplicate details and the original evidence from both parties are based on the forward output DYJDTemp8.

[0242] S409, the sixth temporary storage quantity for precise judgment is DYJDTemp6=(DYJDTemp3+DYJDTemp5) / 2; the seventh temporary storage quantity for precise judgment is DYJDTemp7=ABS(DYJDTemp3-DYJDTemp5);

[0243] Where ABS represents the absolute value;

[0244] In this step, the 7th temporary value DYJDTemp7 is used to determine the stability difference; the 6th temporary value DYJDTemp6 is the final repetition score.

[0245] S410. If DYJDTemp7 is greater than the symmetric judgment consistency tolerance DCYZ, then set the manual review flag of the current candidate application to 1; otherwise, set it to 0. Add an element to DYJDList, which includes: the application number of the current candidate application, setting the forward score to DYJDTemp3, the reverse score to DYJDTemp5, the final score to DYJDTemp6, the stability difference to DYJDTemp7, the manual review flag, and the details of the five-dimensional duplication points given by the forward judgment and the original evidence from both parties to DYJDTemp8.

[0246] S411. Set DYJDCounter2 = DYJDCounter2 + 1; if DYJDCounter2 is less than or equal to DYJDCut, go to S406; otherwise, go to S412.

[0247] S412. Sort the data in DYJDList from highest to lowest according to the final score; output DYJDOutput as the result of DYJDModel.

[0248] The principle of this module is as follows: For historical applications that are duplicates of the application to be judged, there is a significant gap between their comprehensive ranking scores and those of the other applications. Because the ranking scores are arranged in descending order and truncated at the point where the difference between the two adjacent items is the largest, the size of the candidate set can be automatically scaled according to the actual duplication situation. This ensures that no duplicates are missed when there are many duplicates and that the judgment cost is not wasted when there are few duplicates. Furthermore, because the pairwise judgment of the large language model is sensitive to the presentation order of the two documents in the prompt words, different conclusions may be obtained when the same pair of documents is input in different orders. Therefore, each candidate pair is judged once in ascending order and once in descending order. The average of the two scores is taken as the final score, and the difference between the two scores is taken as a measure of the judgment stability. If the difference exceeds the tolerance, it means that the document pair is on the judgment boundary and should be submitted for manual review.

[0249] This step achieves cliff-cutting and symmetrical precision judgment. After sorting the comprehensive ranking scores in descending order, the candidate set is truncated at the point where the difference between adjacent items is the largest, so that its size can be scaled up or down according to the actual repetition situation. For each candidate pair, a large language model judgment is performed once in ascending order and once in descending order. The mean is used to determine the score, and the difference is used to measure the stability. Those exceeding the tolerance are submitted for manual review, and the model is required to extract the original text evidence from both sides dimension by dimension.

[0250] S5. Extract five-dimensional contextual semantic features from each historical application and pending application, and encode these features into semantic vectors. Using these semantic vectors, calculate the mean (DMBase) and standard deviation (DMStd) of the current dimension's semantic similarity in the historical database through sampling pairing. Fill DMBase and DMSt into the five-dimensional definition table DimTable. Then, using the five-dimensional definition table as input, sequentially call the dimensional baseline standardization module BXBZModel and the five-dimensional imbalance rewriting detection module BJHModel to sort the entire database, obtaining the full database ranking result list CFRankList. Finally, call the cliff truncation symmetric precision judgment module DYJDModel, using CFRankList as input to DYJDModel, and use DYJDModel to determine the top candidates pairwise, obtaining CFResultList. Specifically:

[0251] S501. Initialize and set the feature extraction counter CFCounter1=1;

[0252] S502, Check the first temporary variable CFTemp1 for duplicates and retrieve the CFCounter1 record from HisList; if HFeatList of CFTemp1 is not empty, go to S506; otherwise, go to S503.

[0253] The conditions in this step ensure that applications that have already been entered into the database and whose semantic features have been extracted will not be extracted again, thereby supporting incremental maintenance of the historical database.

[0254] S503. Initialize and set the dimension extraction counter CFCounter2=1;

[0255] S504, the second temporary text for deduplication, CFTemp2, is extracted using the large language model specified by LLMName at the sampling temperature LLMTemp, with DMPrompt of the CFCounter2 record in DimTable as the prompt word, and HText of CFTemp1 as the input. The output of the large language model specified by LLMName is the text extracted in the current dimension.

[0256] The third temporary vector for deduplication, CFTemp3, is the EmbDim-dimensional semantic vector obtained by encoding CFTemp2 using the semantic vector model specified by EmbName.

[0257] Add an element to HFeatList of CFTemp1 with FDDim=CFCounter2, FDText=CFTemp2, and FDVec=CFTemp3;

[0258] S505, Let CFCounter2 = CFCounter2 + 1; If CFCounter2 is less than or equal to DimNum, then go to S504; otherwise, go to S506.

[0259] S506. Set CFCounter1 = CFCounter1 + 1; if CFCounter1 is less than or equal to HisNum, go to S502; otherwise, go to S507.

[0260] S507. Extract the five-dimensional text of the application to be judged from the NewApp and encode it into a semantic vector in the same way as S503 to S505, and fill it into the HFeatList of NewApp;

[0261] S508. Initialize baseline statistics counter CFCounter3=1;

[0262] S509. Randomly select SampleN pairs of distinct historical applications from HisList, calculate the cosine similarity of the two semantic vectors of each pair of historical applications in the third dimension of CFCounter3, fill the mean of the similarity of SampleN into DMBase of the third record of DimTable, and fill the standard deviation of the similarity of SampleN into DMStd.

[0263] Wherein, the standard deviation is the arithmetic square root of the mean of the squares of the differences between each similarity and its mean; because the baseline is obtained by the historical database itself, this method can automatically re-evaluate the baseline as the business direction within the database changes without the need for manual adjustment of the weights of each dimension.

[0264] S510, Set CFCounter3 = CFCounter3 + 1; If CFCounter3 is less than or equal to DimNum, go to S509; otherwise, go to S511.

[0265] S511. Initialize the full database sorting counter CFCounter4 = 1; Initialize the full database sorting result list CFRankList = an empty list;

[0266] S512, Call BXBZModel, the input of BXBZModel is BXBZInput=NewApp and the five-dimensional semantic vector of the CFCounter4 record of HisList, and the DimTable that has been filled into the baseline, the output of BXBZModel is BXBZOutput, let the fourth temporary variable for deduplication CFTemp4=BXBZOutput.

[0267] S513. Call BJHModel. The input of BJHModel is BJHInput=CFTemp4's BXBZZList. The output of BJHModel is BJHOutput. Let the 5th temporary variable for deduplication, CFTemp5, be equal to BJHOutput.

[0268] S514. Using the fifth temporary variable CFTemp5 and the fourth temporary variable CFTemp4, obtain the sixth temporary variable CFTemp6. Add an element to CFRankList. The newly added element includes: the current historical application number, BXBZSimList and BXBZZList of CFTemp4, BJHUneven and BJHGapList of CFTemp5, and CFTemp6.

[0269] The 6th temporary buffer for plagiarism detection, CFTemp6, is obtained through the following method:

[0270]

[0271] in, It is the overall ranking score in CFTemp4. It is an imbalance marker in CFTemp5. It is the dimensional imbalance in CFTemp5, and JCXS is the imbalance additive coefficient;

[0272] In this step, the sixth temporary storage value CFTemp6 represents the comprehensive ranking score of the historical application. This formula gives a bonus to documents judged to have an imbalanced structure in the ranking, because the weighted mean of the five-dimensional deviation of duplicate applications that have been rewritten to avoid the problem is often lowered. If sorted only by the mean, they will be ranked after irrelevant documents and missed. Adding the imbalance bonus can push such documents into the candidate set.

[0273] S515. Let CFCounter4 = CFCounter4 + 1; if CFCounter4 is less than or equal to HisNum, go to S512; otherwise, go to S516.

[0274] S516. Sort CFRankList from highest to lowest based on the overall ranking score;

[0275] The value of the comprehensive ranking score is CFTemp6;

[0276] S517. Call DYJDModel. The input of DYJDModel is DYJDInput = the sorted CFRankList, the five-dimensional extracted text of NewApp and each candidate application, and the list of dimension imbalance and low deviation dimension numbers of each document pair. The output of DYJDModel is DYJDOutput. Let the 7th temporary variable for plagiarism detection, CFTemp7, = DYJDOutput.

[0277] A document pair consists of a pending application (NewApp) and a candidate application;

[0278] In this step, the descending sorted list of historical applications in DYJDInput equals the sorted CFRankList;

[0279] S518. Set the result summary counter CFCounter5=1;

[0280] S519. Extract the CFCounter5 element of DYJDList from CFTemp7 as the fine-judgment record; based on the historical application number HID in the fine-judgment record, sort the records with the number HID in CFRankList; use the HID of the sorted record to search for the CRHID, CRTitle, and CRYear of the current historical application in the HisList historical application database, and extract the five-dimensional original similarity CRSimList, five-dimensional standardized deviation CRZList, dimensional imbalance CRUneven, and low deviation dimension number list CRGapList from the sorted records; extract the positive order score CRScoreF, negative order score CRScoreR, final duplication score CRScore, judgment stability difference CRStab, manual review mark CRCheck, and five-dimensional duplication details and original evidence from both parties CRDetail from the fine-judgment record; if CRScore is greater than or equal to CFYZ, set CRDup=1, otherwise set CRDup=0; assemble all the above fields into a new result element and append the new result element to the result list CFResultList;

[0281] S520, set CFCounter5 = CFCounter5 + 1; if CFCounter5 is less than or equal to len(CFTemp7's DYJDList), then go to S519, otherwise output CFResultList.

[0282] This step involves extracting semantic features one by one and supporting incremental data entry. A baseline is established, and the deviation and dimensional imbalance are calculated for each data entry in the entire database. The imbalance is used to add to the ranking score so that duplicate applications avoided by rewriting do not sink to the bottom. After ranking, the data is submitted to the precision judgment module for final decision.

[0283] S6. Use CFResultList to output the duplicate detection results and provide project proposal suggestions, specifically:

[0284] S601. If CFResultList is empty, output "No item that is a duplicate of this application was found in the history database" and go to S606; otherwise, go to S602.

[0285] S602. Initialize the output counter OutCounter=1;

[0286] S603. Retrieve the OutCounter record from CFResultList and output the current historical application number CRHID, project name CRTitle, application year CRYear, five-dimensional original semantic similarity CRSimList, five-dimensional standardized deviation CRZList, dimensional imbalance CRUneven, list of deviations below the median CRGapList, positive order score CRScoreF, negative order score CRScoreR, final duplication score CRScore, judgment stability difference CRStab, manual review mark CRCheck, five-dimensional duplication details and original evidence from both parties CRDetail and duplication judgment mark CRDup;

[0287] S604. Set OutCounter = OutCounter + 1; if OutCounter is less than or equal to len(CFResultList), go to S603; otherwise, go to S605.

[0288] S605. Record the number of records in CFResultList where CRDup equals 1 as CFDupN, the number of records where CRCheck equals 1 as CFChkN, and the maximum value of CRScore among all records as CFMaxScore. If CFDupN is greater than 0, output "This application constitutes substantial duplication of the research path with CFDupN projects in the historical database, with the highest duplication score being CFMaxScore. It is recommended to focus on review during the project approval process." Otherwise, output "This application does not constitute substantial duplication of any project in the historical database." If CFChkN is greater than 0, also output "CFChkN judgments are at the boundary, and the difference between the forward and reverse scores exceeds the tolerance. They must be manually reviewed." Otherwise, no additional output is added.

[0289] This step outputs the five-dimensional scores, deviation, imbalance, separated dimensions, positive and negative scores, stability, original evidence, and duplicate judgment markers for each item, and summarizes whether it is recommended to focus on this aspect during the project approval process.

[0290] This invention converts absolute similarity into standardized deviation by using the baseline of dimensions statistically derived from the historical database. This ensures that plagiarism detection retains its distinguishing ability even in a highly homogeneous application database with concentrated business focus. Furthermore, the baseline is automatically re-evaluated with the database without manual weighting. The imbalanced structure of the five-dimensional deviation vector serves as a calculable feature for rewrite avoidance, transforming rewrite avoidance from an interference that needs to be countered into detectable evidence. The imbalance is further used to add to the ranking score, preventing duplicate applications that have undergone rewrite avoidance from being missed due to a lower mean. The maximum difference in ranking score automatically determines the candidate set boundary, and the mean score is determined by the bidirectional judgment of positive and negative order, while the difference determines the stability. This controls the cost of calling a large language model while ensuring that each judgment is accompanied by a five-dimensional score and original evidence from both parties, allowing for review on a case-by-case basis during the review meeting.

Claims

1. A method for deduplication and ranking of project application documents using five-dimensional contextual semantic features, characterized in that: The specific process of the method is as follows: S1. Create a historical application database HisList, pending application NewApp, a five-dimensional definition table DimTable, and a duplicate judgment result list CFResultList, and initialize the duplicate judgment result list CFResultList to an empty list; S2. Using the five-dimensional semantic vector corresponding to NewApp, the five-dimensional semantic vector of a historical application, and DimTable to form the baseline normalized input BXBZInput, establish the dimensional baseline normalization module BXBZModel. The input of BXBZModel is the baseline normalized input BXBZInput, and the output of BXBZModel is the baseline normalized output BXBZOutput. BXBZOutput includes a five-dimensional standardized deviation list, BXBZZList. S3. Construct the imbalance detection input BJHInput using the five-dimensional standardized deviation list BXBZZList, and establish the five-dimensional imbalance rewrite detection module BJHModel. The input of BJHModel is the imbalance detection input BJHInput, and the output of BJHModel is the imbalance detection output BJHOutput. BJHOutput includes: BJHUneven (dimensional imbalance), and BJHGapList (a list of low-skewness dimension numbers). S4. Construct the cliff truncation symmetry precision judgment module DYJDModel. The input of DYJDModel is the cliff precision judgment input DYJDInput, and the output of DYJDModel is the cliff precision judgment output DYJDOutput. DYJDInput includes: BJHUneven (dimensional imbalance) and BJHGapList (low deviation dimension number list); S5. Obtain the mean (DMBase) of the semantic similarity of the previous dimension in the historical database and the standard deviation (DMStd) of the semantic similarity of the current dimension in the historical database for the historical application database and the application to be judged. Fill DMBase and DMSt into the five-dimensional definition table DimTable. Then, using the five-dimensional definition table as input, sequentially call the dimension baseline standardization module BXBZModel and the five-dimensional imbalance rewriting detection module BJHModel to sort the entire database and obtain the full database sorting result list CFRankList. Finally, call the cliff truncation symmetric precision judgment module DYJDModel, using CFRankList as input to DYJDModel, and output DYJDOutput. Assemble CFResultList based on DYJDOutput, CFRankList, and HisList. S6. Use CFResultList to output the duplicate detection results and provide suggestions.

2. The method for deduplication and ranking of project application documents based on five-dimensional contextual semantic features according to claim 1, characterized in that: The S1 establishes a historical application database HisList, pending application NewApp, a five-dimensional definition table DimTable, and a duplicate determination result list CFResultList, specifically as follows: Each record in the HisList historical application database corresponds to a historical project application and includes the following fields: application number (HID), project name (HTitle), application year (HYear), full text of the application (HText), and the five-dimensional contextual semantic feature (HFeatList) of the current application. Each element in HFeatList corresponds to a dimension and contains the following fields: dimension number FDDim, extracted text for the current dimension FDText, and semantic vector for the current dimension FDVec; HFeatList is initialized as an empty list; The pending application form (NewApp) includes the following fields: project name, application year, full text of the application form, and five-dimensional contextual semantic features of the current application form; The five-dimensional contextual semantic features of the current application form include the following fields: dimension number, extracted text of the current dimension, and semantic vector of the current dimension; Each record in the five-dimensional definition table DimTable contains the following fields: Dimension ID DMID, Dimension Name DMName, Extraction Prompt Template for the Current Dimension DMPrompt, Mean of Semantic Similarity of the Current Dimension in the Historical Database DMBase, Standard Deviation of Semantic Similarity of the Current Dimension in the Historical Database DMStd, and Prior Weight of the Current Dimension DMWeight. Both DMBase and DMStd are initialized to 0; The duplicate detection result list CFResultList includes the following fields: The following data is included: historical application number (CRHID), project name (CRTitle), application year (CRYear), five-dimensional original semantic similarity between the pending application and the historical application (CRSimList), five-dimensional standardized deviation (CRZList), dimensional imbalance (CRUneven), list of dimensions with deviation below the median (CRGapList), positive order judgment score (CRScoreF), negative order judgment score (CRScoreR), final duplication score (CRScore), judgment instability (CRStab), manual review marker (CRCheck), five-dimensional duplication details and original evidence from both parties (CRDetail), and duplication judgment marker (CRDup). In this context, CRCheck = 1 indicates that manual review is required, and CRCheck = 0 indicates that manual review is not required; CRDup = 1 indicates that the application is considered a duplicate, and CRDup = 0 indicates that the application is not considered a duplicate.

3. The method for deduplication and ranking of project application documents based on five-dimensional contextual semantic features according to claim 2, characterized in that: In S2, the baseline-normalized input BXBZInput is composed of the five-dimensional semantic vector corresponding to NewApp, the five-dimensional semantic vector of a historical application, and DimTable. A dimensional baseline-normalized module BXBZModel is then established. The input to BXBZModel is the baseline-normalized input BXBZInput, and the output of BXBZModel is the baseline-normalized output BXBZOutput. Specifically: S201. Using the five-dimensional semantic vector of the application to be judged, the five-dimensional semantic vector of a historical application, and the five-dimensional definition table, a baseline standardized input BXBZInput is formed, and a dimension baseline standardized module BXBZModel is established. The input of BXBZModel is the baseline standardized input BXBZInput. The five-dimensional semantic vectors of the pending application and the historical application both include the following fields: dimension number FDDim, extracted text of the current dimension FDText, and semantic vector of the current dimension FDVec; When the dimension numbers in the five-dimensional definition table are 1-5, the corresponding dimension names are: problem domain, research objective, technical route, research content, and functional requirements, respectively. S202. Initialize the baseline standardized output BXBZOutput's five-dimensional original semantic similarity list BXBZSimList = empty list, initialize the five-dimensional standardized deviation list BXBZZList = empty list, initialize the comprehensive ranking score BXBZScore = 0; initialize the deviation weighted sum BXBZSum = 0; initialize the weighted sum BXBZWSum = 0. S203. Initialize the dimension traversal counter BXBZCounter=1; S204, Standardized first temporary vector BXBZTemp1 = semantic vector of the BXBZCounter dimension of the application to be judged; Standardized second temporary vector BXBZTemp2 = semantic vector of the BXBZCounter dimension of the current historical application; S205, Standardized Third Temporary Storage Add BXBZTemp3 to BXBZSimList; in, It is the vector magnitude. It is the dot product symbol, and EPS is the offset constant; S206, Standardize the 4th temporary storage BXBZTemp4=(BXBZTemp3-DMBase) / (DMStd+EPS), and add BXBZTemp4 to BXBZZList; Wherein, DMBase is the mean of the semantic similarity of the current dimension in the historical database, and DMStd is the standard deviation of the semantic similarity of the current dimension in the historical database; S207, Let the weighted sum of deviations be BXBZSum = BXBZSum + BXBZTemp4 × DMWeight; Let the weighted sum be BXBZWSum = BXBZWSum + DMWeight; Wherein, DMWeight is the prior weight of the current dimension; S208. Set BXBZCounter = BXBZCounter + 1; if BXBZCounter is less than or equal to DimNum, go to S204; otherwise, go to S209. S209. Set BXBZScore = BXBZSum / (BXBZWSum + EPS); S210. Output BXBZOutput as the result of BXBZModel.

4. The method for deduplication and ranking of project application documents based on five-dimensional contextual semantic features according to claim 3, characterized in that: In step S3, the imbalance detection input BJHInput is constructed using the five-dimensional standardized deviation list BXBZZList, and a five-dimensional imbalance rewrite detection module BJHModel is established. The input of BJHModel is the imbalance detection input BJHInput, and the output of BJHModel is the imbalance detection output BJHOutput. Specifically: S301. Construct the imbalance detection input BJHInput using the five-dimensional standardized deviation list BXBZZList, and establish the five-dimensional imbalance rewrite detection module BJHModel. The input of BJHModel is the imbalance detection input BJHInput. S302. Initialize the imbalance detection output BJHOutput: Dimension imbalance degree BJHUneven=0, Low deviation dimension number list BJHGapList=empty set, Imbalance flag BJHFlag=0; S303, Imbalanced First Temporary List BJHTemp1 = List obtained by sorting the five-dimensional standardized deviations in BJHInput from largest to smallest; S304, the unbalanced second temporary storage BJHTemp2 = the average of the first GWN values ​​in BJHTemp1; Wherein, GWN is the preset number of high-scoring dimensions; S305, the median of the five-dimensional standardized deviation in the unbalanced third temporary storage BJHTemp3=BJHInput; S306, Set the dimensional imbalance degree BJHUneven=BJHTemp2-BJHTemp3; S307. If BJHUneven is greater than GXYZ and BJHTemp2 is greater than GWYZ, then go to S308; otherwise go to S309. Wherein, GXYZ is the dimensional imbalance threshold, and GWYZ is the lower limit of high-scoring dimensional deviation; S308. Set BJHFlag=1; Determine whether the standardized deviation value of each dimension of the five-dimensional standardized deviation list BXBZZList in BJHInput is less than BJHTemp3. If the standardized deviation value of the current dimension is less than BJHTemp3, add DMID in the DimTable corresponding to the current dimension to BJHGapList and go to S309; ​​otherwise, go directly to S309. S309. Output BJHOutput as the result of BJHModel.

5. The method for deduplication and ranking of project application documents based on five-dimensional contextual semantic features according to claim 4, characterized in that: The S4 module DYJDModel, which constructs a cliff-cut symmetry precision judgment module, has the following inputs: DYJDInput and DYJDOutput. S401. Establish the cliff truncation symmetry precision judgment module DYJDModel. The input of DYJDModel is the cliff precision judgment input DYJDInput. The cliff-edge precision judgment input DYJDInput includes: a list of historical applications sorted in descending order, five-dimensional extracted text of the historical application list, five-dimensional extracted text of the application to be judged, a list of dimensional imbalance of document pairs, and a list of low deviation dimension numbers of document pairs. S402, Initialize the candidate set truncation position of the cliff fine judgment output DYJDOutput DYJDCut=JMin, fine judgment result list DYJDList=empty list; Initialize the maximum drop DYJDMaxGap=0; Initialize the drop traversal counter DYJDCounter1=JMin; Among them, JMin is the minimum size of the candidate set; S403. Fine-tune the first temporary storage quantity DYJDTemp1 = the DYJDCounter1 item minus the DYJDCounter1+1 item in the comprehensive sorting score in descending order; if DYJDTemp1 is greater than DYJDMaxGap, then set DYJDMaxGap=DYJDTemp1 and set DYJDCut=DYJDCounter1. Wherein, the first temporary storage value DYJDTemp1 represents the sorting score difference at the current position; S404. Set DYJDCounter1 = DYJDCounter1 + 1; if DYJDCounter1 is less than or equal to JMax and DYJDCounter1 is less than HisNum, then go to S403; otherwise, go to S405. HisNum is the total number of historical applications, and JMax is the maximum size of the candidate set. S405. Initialize the candidate traversal counter DYJDCounter2=1; S406, Fine-grained judgment of the second temporary text DYJDTemp2 = paired judgment prompt words; The paired judgment prompts are obtained using the five-dimensional extracted text of the application to be judged and the second candidate application (DYJDCounter2). S407, Let the third temporary storage quantity of the refined judgment, DYJDTemp3, be the ascending score; let the eighth temporary storage text of the refined judgment, DYJDTemp8, be the ascending five-dimensional duplicate point details and the original evidence from both sides; The orthogonal score was obtained by combining the orthogonal five-dimensional repetition details with the original evidence from both sides, using the large language model specified by the second temporary text DYJDTemp2 and LLMName for precise analysis. S408. Obtain the fourth temporary text DYJDTemp4 for precise judgment using DYJDTemp2, and obtain the fifth temporary text DYJDTemp5 for precise judgment based on the fourth temporary text DYJDTemp4. Specifically: The fourth temporary text for detailed judgment, DYJDTemp4, is a prompt word obtained by swapping the order of the pending application and the candidate application in DYJDTemp2. The fifth temporary value, DYJDTemp5, is determined by reversing the order score. The reverse score is obtained in the following way: Keeping the sampling temperature constant, the fourth temporary text DYJDTemp4 is input into the large language model specified by LLMName for inference. The model output is parsed, and the repetition scores of each of the five dimensions are extracted. The repetition scores of each dimension are used to perform a weighted average calculation with the DMWeight of the corresponding dimension to obtain the weighted average. The weighted average is used as the reverse score. S409, the sixth temporary storage quantity for precise judgment is DYJDTemp6=(DYJDTemp3+DYJDTemp5) / 2; the seventh temporary storage quantity for precise judgment is DYJDTemp7=ABS(DYJDTemp3-DYJDTemp5); Where ABS represents the absolute value; S410. If DYJDTemp7 is greater than the symmetric judgment consistency tolerance DCYZ, then set the manual review flag of the current candidate application to 1; otherwise, set it to 0. Add an element to DYJDList, which includes: the application number of the current candidate application, setting the forward score to DYJDTemp3, the reverse score to DYJDTemp5, the final score to DYJDTemp6, the stability difference to DYJDTemp7, the manual review flag, and the details of the five-dimensional duplication points given by the forward judgment and the original evidence from both parties to DYJDTemp8. S411. Set DYJDCounter2 = DYJDCounter2 + 1; if DYJDCounter2 is less than or equal to DYJDCut, go to S406; otherwise, go to S412. S412. Sort the data in DYJDList from highest to lowest according to the final score; output DYJDOutput as the result of DYJDModel.

6. The method for deduplication and ranking of project application documents based on five-dimensional contextual semantic features according to claim 5, characterized in that: The paired determination prompt words in S406 are obtained through the following method: First, the five-dimensional extracted texts of the application to be judged and the second candidate application are arranged side by side according to their dimensions; Then, the list of dimension imbalance and low deviation dimension numbers of the current document pair is appended into the prompt words, and an instruction is added to prompt the large model to focus on comparing the above low deviation dimensions. Then, add model task instructions: require the model to output a repetition score of 0-100 and the corresponding judgment reasons for each of the five dimensions: problem domain, research objective, technical route, research content, and functional requirements. Then, additional evidence output requirements are added: the model is required to extract original text statements that are repeated in the application to be judged and the second candidate application in DYJDCounter2 under each dimension, as evidence for judgment; Then, an output length constraint is added: the total number of words in the entire output content of the model is limited to no more than LLMLen; Finally, the prompt words, model task instructions, evidence output requirements, and output length are concatenated into paired decision prompt words.

7. The method for deduplication and ranking of project application documents based on five-dimensional contextual semantic features according to claim 6, characterized in that: The positive order score in S407 is obtained as follows: Read the second temporary text DYJDTemp2, use the large language model specified by LLMName, set the model sampling temperature to LLMTemp, input DYJDTemp2 into the large language model specified by LLMName to perform inference, parse the model output results, extract the repetition scores corresponding to each of the five dimensions, use the repetition scores of each dimension and the corresponding DMWeight to perform weighted average calculation to obtain the weighted average, and use the weighted average as the positive order score; The details of the five-dimensional repetition points and the original text evidence from both parties were obtained as follows: The second temporary text DYJDTemp2 of the fine-judgment was read, the large language model specified by LLMName was used, the model sampling temperature was set to LLMTemp, DYJDTemp2 was input into the large language model specified by LLMName to perform inference, and the repetition scores corresponding to the five dimensions of problem domain, research objectives, technical route, research content and functional requirements, the judgment reasons of the current dimension, and the repetitive sentences extracted from the original text of the application to be judged and the historical applications were extracted from the model output results as the details of the five-dimensional repetition points and the original text evidence from both parties.

8. The method for deduplication and ranking of project application documents based on five-dimensional contextual semantic features according to claim 7, characterized in that: In step S5, the mean (DMBase) of the semantic similarity of the previous dimension in the historical database and the standard deviation (DMStd) of the semantic similarity of the current dimension in the historical database are obtained from the historical application database and the application to be judged. DMBase and DMSt are filled into the five-dimensional definition table DimTable. Then, the five-dimensional definition table is used as input to sequentially call the dimensional baseline standardization module BXBZModel and the five-dimensional imbalance rewriting detection module BJHModel to sort the entire database and obtain the full database sorting result list CFRankList. Finally, the cliff truncation symmetric precision judgment module DYJDModel is called, and CFRankList is used as input to DYJDModel to output DYJDOutput. Based on DYJDOutput, CFRankList and HisList, CFResultList is assembled, specifically as follows: S501. Initialize and set the feature extraction counter CFCounter1=1; S502, Check the first temporary variable CFTemp1 for duplicates and retrieve the CFCounter1 record from HisList; if HFeatList of CFTemp1 is not empty, go to S506; otherwise, go to S503. S503. Initialize and set the dimension extraction counter CFCounter2=1; S504, the second temporary text for deduplication, CFTemp2, is extracted using the large language model specified by LLMName at the sampling temperature LLMTemp, with DMPrompt of the CFCounter2 record in DimTable as the prompt word, and HText of CFTemp1 as the input. The output of the large language model specified by LLMName is the text extracted in the current dimension. The third temporary vector for deduplication, CFTemp3, is the EmbDim-dimensional semantic vector obtained by encoding CFTemp2 using the semantic vector model specified by EmbName. Add an element to HFeatList of CFTemp1, with FDDim=CFCounter2, FDText=CFTemp2, and FDVec=CFTemp3 for the new element; S505, Let CFCounter2 = CFCounter2 + 1; If CFCounter2 is less than or equal to DimNum, then go to S504; otherwise, go to S506. S506. Set CFCounter1 = CFCounter1 + 1; if CFCounter1 is less than or equal to HisNum, go to S502; otherwise, go to S507. S507. NewApp extracts five-dimensional text from the application for judgment and encodes it into a semantic vector, which is then filled into NewApp's HFeatList. S508. Initialize baseline statistics counter CFCounter3=1; S509. Randomly select SampleN pairs of distinct historical applications from HisList, calculate the cosine similarity of the two semantic vectors of each pair of historical applications in the third dimension of CFCounter3, fill the mean of the similarity of SampleN into DMBase of the third record of DimTable, and fill the standard deviation of the similarity of SampleN into DMSted. Where SampleN is the baseline sampling pair number; S510, Set CFCounter3 = CFCounter3 + 1; If CFCounter3 is less than or equal to DimNum, go to S509; otherwise, go to S511. S511. Initialize the full database sorting counter CFCounter4 = 1; Initialize the full database sorting result list CFRankList = an empty list; S512, Call BXBZModel, the input of BXBZModel is BXBZInput=NewApp and the five-dimensional semantic vector of the CFCounter4 record of HisList, and the DimTable that has been filled into the baseline, the output of BXBZModel is BXBZOutput, let the fourth temporary variable for deduplication CFTemp4=BXBZOutput. S513. Call BJHModel. The input of BJHModel is BJHInput=CFTemp4's BXBZZList. The output of BJHModel is BJHOutput. Let the 5th temporary variable for deduplication, CFTemp5, be equal to BJHOutput. S514. Using the fifth temporary variable CFTemp5 and the fourth temporary variable CFTemp4, obtain the sixth temporary variable CFTemp6. Add an element to CFRankList. The newly added element includes: the current historical application number, BXBZSimList and BXBZZList of CFTemp4, BJHUneven and BJHGapList of CFTemp5, and CFTemp6. S515. Let CFCounter4 = CFCounter4 + 1; if CFCounter4 is less than or equal to HisNum, go to S512; otherwise, go to S516. S516. Sort CFRankList from highest to lowest based on its overall ranking score; The value of the comprehensive ranking score is CFTemp6; S517. Call DYJDModel. The input of DYJDModel is DYJDInput = the sorted CFRankList, the five-dimensional extracted text of NewApp and each candidate application, and the list of dimension imbalance and low deviation dimension numbers of each document pair. The output of DYJDModel is DYJDOutput. Let the 7th temporary variable for plagiarism detection, CFTemp7, = DYJDOutput. A document pair consists of a pending application (NewApp) and a candidate application; S518. Set the result summary counter CFCounter5=1; S519. Extract the CFCounter5 element of DYJDList from CFTemp7 as the fine-judgment record; based on the historical application number HID in the fine-judgment record, find the sorted record with the number HID in CFRankList; use the HID of the sorted record to search for the CRHID, CRTitle, and CRYear of the current historical application in the HisList historical application database, and extract the five-dimensional original similarity CRSimList, five-dimensional standardized deviation CRZList, dimensional imbalance CRUneven, and low deviation dimension number list CRGapList from the sorted record; extract the positive order score CRScoreF, negative order score CRScoreR, final duplication score CRScore, judgment stability difference CRStab, manual review mark CRCheck, and five-dimensional duplication details and original evidence from both parties CRDetail from the fine-judgment record; if CRScore is greater than or equal to CFYZ, set CRDup=1, otherwise set CRDup=0; assemble all the above fields into a new result element and append the new result element to the result list CFResultList; S520, set CFCounter5 = CFCounter5 + 1; if CFCounter5 is less than or equal to len(CFTemp7's DYJDList), then go to S519, otherwise output CFResultList.

9. The method for deduplication and ranking of project application documents based on five-dimensional contextual semantic features according to claim 8, characterized in that: The sixth temporary deduplication item CFTemp6 in S514 is obtained through the following method: ； in, It is the overall ranking score in CFTemp4. It is an imbalance marker in CFTemp5. It is the dimensionality imbalance in CFTemp5, and JCXS is the imbalance additive coefficient.

10. The method for deduplication and ranking of project application documents based on five-dimensional contextual semantic features according to claim 9, characterized in that: The specific steps in S6, which involve using CFResultList to output the duplicate detection results and provide suggestions, are as follows: S601. If CFResultList is empty, output that no duplicate item of this application was found in the history database and proceed to S606; otherwise, proceed to S602. S602. Initialize the output counter OutCounter=1; S603. Retrieve the OutCounter record from CFResultList and output the following information: current historical application number CRHID, project name CRTitle, application year CRYear, five-dimensional original semantic similarity CRSimList, five-dimensional standardized deviation CRZList, dimensional imbalance CRUneven, list of dimensions with deviations below the median CRGapList, positive order score CRScoreF, negative order score CRScoreR, final duplicate score CRScore, stability difference CRStab, manual review marker CRCheck, five-dimensional duplicate details, original evidence from both parties CRDetail, and duplicate judgment marker CRDup. S604. Set OutCounter = OutCounter + 1; if OutCounter is less than or equal to len(CFResultList), go to S603; otherwise, go to S605. Where len() is the list length function; S605. Record the number of records in CFResultList where CRDup equals 1 as CFDupN, the number of records where CRCheck equals 1 as CFChkN, and the maximum value of CRScore among all records as CFMaxScore. If CFDupN is greater than 0, output: This application constitutes substantial duplication of the research path with CFDupN projects in the historical database, with the highest duplication score being CFMaxScore. It is recommended to focus on this during the project approval process. Otherwise, output: This application does not constitute substantial duplication with any project in the historical database. If CFChkN is greater than 0, output: Among them, CFChkN judgments are at the boundary, and the difference between positive and negative scores exceeds the tolerance, requiring manual review. Otherwise, no additional output is added.

Images