Configurable multi-dimensional thesis intelligent evaluation method and system based on large language model
By performing structured analysis and multi-dimensional evaluation of academic papers, the problem of fixed evaluation dimensions and difficulty in positioning in existing technologies has been solved, realizing configurable and quantifiable intelligent evaluation of papers and improving the adaptability and accuracy of the evaluation.
Patent Information
- Application Number
- CN202611243636.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-17
- Publication Date
- 2026-11-06
AI Technical Summary
Existing technologies for paper evaluation suffer from problems such as fixed evaluation dimensions, inability to locate evaluation opinions line by line, and difficulty in parsing model outputs, resulting in incomplete evaluation results and a lack of specific positioning of each part of the paper.
By performing structured parsing of the paper documents, a unified structured model with paragraph identifiers and chapter levels is generated. A set of configurable review dimensions is dynamically obtained, and a large language model is used for multi-dimensional evaluation. The scores of each dimension are independently called and weighted and normalized. Problems are identified paragraph by paragraph and located in the original text to generate a report containing the total score, grade and annotations.
It enables multi-dimensional intelligent evaluation of papers that is configurable, quantifiable, and traceable, improving the adaptability and accuracy of the evaluation and ensuring that the evaluation opinions are operable and traceable under general office software.
Smart Images
Figure CN123049207A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of document processing technology, and in particular to a configurable multi-dimensional intelligent evaluation method and system for papers based on a large language model. Background Technology
[0002] With the maturation of large language models in the field of text understanding, the demand for intelligent evaluation of dissertations and course papers is increasing. Currently, dissertation evaluation schemes mainly fall into three categories: The first category is rule-based and statistical text detection schemes, such as plagiarism detection, grammatical error detection, and formatting compliance checks. These schemes can only detect surface-level formatting issues in a paper and cannot evaluate deeper dimensions such as the paper's academic quality, logical structure, and rigor of argumentation.
[0003] The second approach involves submitting the entire paper to a large language model at once and obtaining an overall evaluation from the model. This approach is limited by the context window size of the large language model. When the paper is long, the entire text needs to be truncated, resulting in incomplete evaluation information. At the same time, the evaluation opinions output by the model are only general descriptions and lack specific positioning of each part of the paper, so users cannot know which part of the original text the evaluation opinion corresponds to.
[0004] The third category consists of auxiliary tools designed for specific writing tasks, such as abstract generation tools and grammar editing tools. These tools have limited functionality, providing assistance only for a single stage of the writing process and failing to form a complete closed loop for paper evaluation.
[0005] Furthermore, existing solutions often suffer from unstable data structures output by large language models during model invocation, which can easily lead to parsing failures such as pseudo-JSON, rendering the entire evaluation results invalid. At the same time, suggestive text can be easily embedded in the main text of the paper, causing the model to suffer from "cue word injection" problems, which affect the objectivity of the evaluation results. Summary of the Invention
[0006] The purpose of this application is to provide a configurable, multi-dimensional intelligent evaluation method and system for papers based on a large language model. This method can solve the technical problems in the prior art, such as fixed evaluation dimensions, inability to locate evaluation opinions line by line, and difficulty in parsing model output. It can achieve configurable, quantifiable, and traceable intelligent evaluation of papers.
[0007] To achieve the above objectives, this application provides the following solution: Firstly, this application provides a configurable, multi-dimensional intelligent evaluation method for papers based on a large language model, comprising the following steps: The paper document and review request are obtained, and a structured model is obtained by parsing the paper document; the structured model includes paragraph identifiers and paragraph texts corresponding to each paragraph of the paper document.
[0008] Based on the configuration identifier in the review request, the set of review dimensions corresponding to the configuration identifier is obtained from the configuration storage; the set of review dimensions contains at least one review dimension, and each review dimension corresponds to a weight and a prompt word template.
[0009] For any review dimension, based on all paragraph text of the structured model and the prompt word template corresponding to the review dimension, a model call request corresponding to the review dimension is constructed, and the large language model is called to evaluate the paper document for the review dimension, thereby obtaining the dimension score under the review dimension.
[0010] Based on the weights and scores corresponding to each of the evaluation dimensions, the total score of the paper is determined, and the total score is mapped to the evaluation level of the paper according to a preset score threshold.
[0011] The text of each paragraph of the structured model is input into the large language model in batches to obtain a list of questions returned by the large language model; the list of questions contains at least one question entry; the question entry contains the original text of the question fragment and the annotation content.
[0012] For each of the aforementioned question entries, locate the start and end offsets of the characters in the original text of the question fragment within the paragraph text corresponding to the question entry, and associate and save the character start and end offsets with the annotation content.
[0013] A report document is generated based on the dimension scores corresponding to each of the aforementioned review dimensions, the total score of the paper, and the evaluation level.
[0014] In the report document, each annotation is anchored to the corresponding text position in the report document according to the character start and end offsets corresponding to each of the problem entries.
[0015] Optionally, the paper document is parsed to obtain a structured model, including: Based on the file format of the paper document, the target parsing engine is determined from multiple parsing engines.
[0016] The target parsing engine parses the paper document to obtain the paragraph text of each paragraph and the corresponding formatting information.
[0017] Based on the layout information described in each paragraph, determine the chapter level of each paragraph.
[0018] Assign a globally consecutive paragraph identifier to each paragraph according to their order of arrangement in the paper document.
[0019] The structured model is generated based on the paragraph text, chapter level, and paragraph identifier of each paragraph; the structured model includes the paragraph identifier, chapter level, and paragraph text corresponding to each paragraph.
[0020] Optionally, based on the file format of the paper document, a target parsing engine is determined from multiple parsing engines, specifically including: When the paper document is a modern document with style identifiers, the style-driven parsing engine is determined to be the target parsing engine.
[0021] The paper document is parsed using the target parsing engine, including: The paper document is read paragraph by paragraph, and the heading level of each paragraph is determined based on the style tags used in each paragraph.
[0022] Optionally, based on the file format of the paper document, a target parsing engine is determined from multiple parsing engines, specifically including: When the paper document is an old-style document without style identifiers, the number-driven parsing engine is determined to be the target parsing engine.
[0023] The paper document is parsed using the target parsing engine, including: The chapter number regular expression is used to match the beginning of each paragraph, and the chapter level of each paragraph is inferred based on the matched chapter number pattern.
[0024] Optionally, based on the configuration identifier in the review request, the set of review dimensions corresponding to the configuration identifier is obtained from the configuration storage, specifically including: Extract the configuration identifier from the review request.
[0025] Retrieve all review dimension records corresponding to the configuration identifier from the configuration storage.
[0026] Filter out the review dimension records that are in the enabled state from all review dimension records.
[0027] The evaluation dimension set is obtained by sorting the records of each evaluation dimension in ascending order according to their sorting numbers; each evaluation dimension record contains a dimension name, dimension identifier, dimension category, weight, highest score, and custom prompt word template.
[0028] Optionally, the model invocation request corresponding to any of the review dimensions includes: The system message carries security rules and output specifications, and injects the server's runtime system date as the base time.
[0029] Pre-set confirmation message for the assistant role.
[0030] The paper text message for the user role includes all paragraph text of the structured model; wherein, the prompt word template corresponding to the review dimension is embedded in the system message or the paper text message for the user role.
[0031] Optionally, the paper can be evaluated across various review dimensions using a large language model in a multi-threaded, parallel manner. Each of the aforementioned review dimensions independently returns a 100-point scale score, dimension level, review comments, and modification suggestions.
[0032] The token consumption corresponding to each of the aforementioned review dimensions is accumulated atomically across threads for metering and billing.
[0033] The review results for each of the aforementioned review dimensions are stored in the database separately. The review dimensions do not affect each other, and a failure to call a single dimension does not block other dimensions.
[0034] Optionally, the total score of the paper document is determined based on the weight and dimensional score corresponding to each of the aforementioned review dimensions, including: Construct a mapping relationship between dimension identifiers and weights.
[0035] For each of the aforementioned review dimensions that has been reviewed and whose dimension scores are not empty, the scores of each dimension are multiplied by their corresponding weights and then summed to obtain a weighted sum. At the same time, the weights corresponding to the scores of each dimension are accumulated to obtain the total weight.
[0036] The total score of the paper is obtained by dividing the weighted sum by the total weight; when the total weight is zero, the total score of the paper is zero.
[0037] Mapping the total score of the paper to an evaluation level for the paper document based on a preset score threshold includes: When the total score of the paper is not lower than the first threshold, the evaluation level is determined to be A.
[0038] When the total score of the paper is lower than the first threshold and not lower than the second threshold, the evaluation level is determined to be B.
[0039] When the total score of the paper is lower than the second threshold but not lower than the third threshold, the evaluation level is determined to be C.
[0040] When the total score of the paper is lower than the third threshold, the evaluation level will be determined as D.
[0041] Optionally, in the report document, based on the character start and end offsets corresponding to each of the problem entries, each annotation is anchored to the corresponding text position in the report document, specifically including: Before writing comments into the report document, the paragraph text corresponding to the issue entry is matched with a text prefix in the report document to locate the position of the paragraph text in the report document.
[0042] Filter out non-content layout units within the paragraph text and establish character offset mappings for each content layout unit.
[0043] Based on the character start and end offsets, determine the target layout unit into which the original text of the problem fragment falls, split the target layout unit at the position corresponding to the character start and end offsets, and divide the original text of the problem fragment into independent layout units.
[0044] Insert a comment interval start mark before the segmented independent typesetting unit, and insert a comment interval end mark and a comment reference mark after the independent typesetting unit, and write the comment content into the independent comment component.
[0045] When there are multiple problem entries in the same paragraph, annotation marks are inserted sequentially from back to front in descending order of the starting offset of each problem entry.
[0046] Secondly, this application provides a configurable, multi-dimensional intelligent paper evaluation system based on a large language model, including the following functional modules: The document parsing module is used to parse the thesis document and obtain a structured model; the structured model includes the paragraph identifier and paragraph text corresponding to each paragraph of the thesis document.
[0047] The dimension configuration module is used to obtain the set of review dimensions corresponding to the configuration identifier from the configuration storage according to the configuration identifier in the review request; the set of review dimensions contains at least one review dimension, and each review dimension corresponds to a weight and a prompt word template.
[0048] The dimension review module is used to construct a model call request for any review dimension based on all paragraph text of the structured model and the prompt word template corresponding to the review dimension. It then calls the large language model to evaluate the paper document for the review dimension and obtains the dimension score under the review dimension.
[0049] The aggregation and grading module is used to determine the total score of the paper document based on the weight and dimension score corresponding to each of the review dimensions, and to map the total score of the paper document to the evaluation level of the paper document according to a preset score threshold.
[0050] The paragraph review module is used to input the paragraph text of the structured model into the large language model in batches and obtain the question list returned by the large language model; the question list contains at least one question entry; the question entry contains the original text of the question fragment and the annotation content.
[0051] The offset positioning module is used to locate the start and end offsets of characters in the original text of the question segment in the paragraph text corresponding to each question entry, and to associate and save the start and end offsets of characters with the annotation content.
[0052] The report generation module generates a report document based on the dimension scores corresponding to each of the review dimensions, the total score of the paper document, and the evaluation level. In the report document, according to the character start and end offsets corresponding to each of the question items, the annotation content is anchored to the corresponding text position in the report document.
[0053] According to the specific embodiments provided in this application, the following technical effects are disclosed: This application provides a configurable multi-dimensional intelligent evaluation method and system for academic papers based on a large language model. The method first performs structured parsing of the paper document to obtain a unified structured model carrying paragraph identifiers and chapter levels, laying the foundation for subsequent precise positioning. Then, based on the configuration identifiers in the review request, it dynamically obtains a set of review dimensions that can be started / stopped, sorted, and have built-in weights and custom prompt word templates. This allows the evaluation criteria to be flexibly adjusted according to discipline, level, or institution without modifying the program, significantly improving the system's adaptability and scalability. On this basis, independent call requests are constructed for each dimension, and then the large language model is called in parallel for scoring. This shortens the overall processing time and achieves fine-grained multi-dimensional evaluation. Simultaneously, each dimension is independently stored in the database without blocking each other, ensuring high availability. Finally, the effective scores are weighted, normalized, and aggregated using the weights of each dimension to obtain the total score of the paper. By mapping configurable thresholds to evaluation levels, interpretable and comparable quantitative ratings can still be provided even when dimensions change dynamically. Then, all paragraphs are fed into the model in fixed batches for segment-by-segment problem identification. The start and end character offsets of the problem segments are immediately located in the original text and pre-saved. These offsets are recorded before report generation, thus completely unaffected by paragraph drift caused by subsequent report content insertion, significantly improving the robustness and hit rate of annotation location. Finally, when generating a report document containing the total score, level, and review comments for each dimension, each annotation is precisely anchored to its corresponding text position in the original text based on the pre-saved character offsets through text relocation and cross-typesetting unit character-level splitting. This achieves "line-by-line review" style highlighted annotations in general office software, making evaluation comments traceable and operable, effectively solving the pain points of existing solutions where evaluations are detached from the original text and difficult to locate precisely. Attached Figure Description
[0054] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0055] Figure 1 This is an application environment diagram of a configurable multi-dimensional intelligent paper evaluation method based on a large language model, provided as an embodiment of this application.
[0056] Figure 2 This is a flowchart illustrating a configurable, multi-dimensional intelligent paper evaluation method based on a large language model, as provided in one embodiment of this application.
[0057] Figure 3 This is a detailed flowchart of step S101 in a configurable multi-dimensional intelligent paper evaluation method based on a large language model, provided as an embodiment of this application.
[0058] Figure 4 This is a schematic diagram of the message structure of a large language model call request in a configurable multi-dimensional intelligent paper evaluation method based on a large language model, provided as an embodiment of this application.
[0059] Figure 5 This is a detailed flowchart of step S108 in a configurable multi-dimensional intelligent paper evaluation method based on a large language model, provided as an embodiment of this application.
[0060] Figure 6 This is a schematic diagram of a cross-typesetting unit character-level precise anchoring mechanism in a configurable multi-dimensional intelligent paper evaluation method based on a large language model, provided as an embodiment of this application.
[0061] Figure 7 This is a schematic diagram illustrating the weighted aggregation of results at different granularities in a configurable multi-dimensional intelligent paper evaluation method based on a large language model, as provided in an embodiment of this application.
[0062] Figure 8 This is a schematic diagram of the functional modules of a configurable multi-dimensional intelligent paper evaluation system based on a large language model, provided as an embodiment of this application.
[0063] Figure 9 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation
[0064] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0065] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, this application will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0066] The configurable multi-dimensional intelligent paper evaluation method based on a large language model provided in this application embodiment can be applied to, for example... Figure 1 The application environment shown includes a user terminal 100 and a server 200. The user terminal 100 communicates with the server 200 via a network. The user uploads the paper to be reviewed and the review request to the server 200 through the user terminal 100. After receiving the paper, the server 200 executes the intelligent paper evaluation method provided in this embodiment, generates an annotated review report document, and returns the review report document to the user terminal 100. The user terminal 100 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, etc. The server 200 can be implemented using a standalone server or a server cluster consisting of multiple servers.
[0067] In one exemplary embodiment, such as Figure 2 As shown, a configurable, multi-dimensional intelligent evaluation method for papers based on a large language model is presented. This method includes the following steps: S101. Parse the paper document to obtain a structured model; the structured model includes paragraph identifiers and paragraph text corresponding to each paragraph of the paper document.
[0068] Specifically, after a user uploads their paper document to server 200 via user terminal 100, server 200 first parses the paper document. Server 200 generates a globally unique review number and encapsulates the paper document as a message, which is then delivered to a message queue. Multiple consumer threads concurrently retrieve the paper message, and the state machine-driven orchestration engine sequentially executes four stages: "document parsing → dimensional review → paragraph (line-by-line) review → report generation," ultimately outputting a review report with line-by-line annotations of the original text and storing it in the database.
[0069] like Figure 3 As shown, in an exemplary embodiment, after obtaining the paper document to be reviewed, the state machine is in the first stage "document parsing". Step S101 can be specifically implemented by the following steps S1011 to S1015: S1011. Based on the file format of the paper document, determine the target parsing engine from multiple parsing engines.
[0070] Specifically, server 200 reads the paper document from the message, reads the file extension or header information of the paper document, and identifies the format type of the paper document. Server 200 is pre-configured with multiple parsing engines, including a style-driven parsing engine and a number-driven parsing engine.
[0071] When the paper document is a modern document with style tags (e.g., docx format), the server 200 determines the style-driven parsing engine as the target parsing engine. A modern document is a document format that contains style tags, and paragraphs define their format attributes (such as heading levels, body text styles, etc.) by referencing styles.
[0072] When the paper document is an old-style document without style tags (such as doc or plain text format), the server 200 determines that the number-driven parsing engine is the target parsing engine. An old-style document refers to a document format that does not contain style tags or whose style tags are not readable, and whose paragraph hierarchy can only be inferred from the chapter numbering pattern.
[0073] S1012. The paper document is parsed using the target parsing engine to obtain the paragraph text of each paragraph of the paper document and the corresponding formatting information of the paragraph text.
[0074] When the target parsing engine is a style-driven parsing engine, the parsing process is as follows: Server 200 reads each paragraph of the thesis document segment by segment, determining the heading level (e.g., first-level heading, second-level heading, third-level heading, body text, etc.) based on the styles cited in each paragraph. Simultaneously, Server 200 extracts text blocks from the thesis document unit by unit, preserving font and size information along with the text blocks. Server 200 also scans each paragraph sequentially using a state machine, marking the start and end positions of the abstract and table of contents sections based on preset section features (e.g., keywords such as "abstract," "keywords," "table of contents," "references," etc.).
[0075] When the target parsing engine is a number-driven parsing engine, the parsing process is as follows: Server 200 uses regular expressions to match the beginning of each paragraph with chapter numbers, and infers the chapter level of each paragraph based on the matched chapter number patterns. Chapter number regular expressions include, but are not limited to, patterns such as "Chapter X", "1.", "1.1", and "1.1.1". For example, when a paragraph begins with "Chapter 1", server 200 identifies the paragraph as a first-level heading; when it begins with "1.1", server 200 identifies it as a second-level heading, and so on. Even in the absence of style information, the number-driven parsing engine can still reconstruct the chapter hierarchy of the thesis document.
[0076] S1013. Determine the chapter level of each paragraph based on the layout information described in each paragraph.
[0077] Specifically, when parsing using a style-driven parsing engine, server 200 directly reads the chapter level from the styles referenced in each paragraph. For example, paragraphs referencing the "Heading 1" style are identified as first-level headings, paragraphs referencing the "Heading 2" style are identified as second-level headings, and paragraphs referencing the "Body" style are identified as body text. When parsing using a number-driven parsing engine, server 200 determines the chapter level based on the chapter numbering pattern matched by the regular expression in step S1012.
[0078] S1014. Assign a globally consecutive paragraph identifier to each paragraph according to the order in which the paragraphs are arranged in the paper document.
[0079] Specifically, server 200 assigns a globally unique paragraph number to each paragraph, starting from the first paragraph of the paper document and incrementing from 1 throughout the entire document. For example, if the paper document contains 100 paragraphs, each paragraph will be assigned a paragraph number from 1 to 100.
[0080] S1015. Generate the structured model based on the paragraph text, chapter level, and paragraph identifier of each paragraph; the structured model includes the paragraph identifier, chapter level, and paragraph text corresponding to each paragraph.
[0081] Specifically, server 200 associates the paragraph text, chapter level, and paragraph identifiers extracted in the above steps to form a unified structured model. This structured model also includes an ordered sequence of typesetting units that constitute each paragraph, with each typesetting unit carrying font and font size information. The paragraph identifiers serve as positioning anchors throughout subsequent stages, used to transmit paragraph positioning information between stages and avoid repeated parsing of the paper document in later stages.
[0082] After completing the document parsing, the state machine moves to the second stage, "Dimension Review", which includes steps S102 to S104 below.
[0083] S102. Based on the configuration identifier in the review request, obtain the review dimension set corresponding to the configuration identifier from the configuration storage; the review dimension set contains at least one review dimension, and each review dimension corresponds to a weight and a prompt word template.
[0084] Specifically, server 200 extracts configuration identifiers from the review request. The configuration identifier is used to uniquely identify a combination of review dimensions, and different configuration identifiers correspond to different subject types, training levels, or review institutions.
[0085] In an exemplary embodiment, step S102 specifically includes: After retrieving the configuration identifier from the review request, server 200 queries the configuration storage for all review dimension records corresponding to that configuration identifier. The configuration storage can be a relational database or a non-relational database. Each review dimension record includes a dimension name (e.g., "Topic Value"), a dimension identifier (e.g., "topic_value"), a dimension category (e.g., "Academic"), a weight (e.g., 0.15), a maximum score (e.g., 100), a custom prompt template (e.g., a dedicated review instruction for this dimension), a sort number (used to determine the display order of each dimension in the report), and an enable / disable status (used to enable or disable the dimension without deleting the record).
[0086] Server 200 filters out the review dimension records that are in the enabled state from all review dimension records, and sorts them in ascending order according to the sort number in each review dimension record to obtain the set of review dimensions for this review.
[0087] S103. For any review dimension, based on all paragraph texts of the structured model and the prompt word templates corresponding to the review dimension, call the large language model to evaluate the paper document for the review dimension and obtain the dimension score under the review dimension.
[0088] Specifically, server 200 constructs an independent large language model call request for each review dimension in the review dimension set. If a review dimension is configured with a custom prompt word template, server 200 uses that custom prompt word template; otherwise, server 200 uses a pre-stored standard review prompt word template. The standard review prompt word template is a general paper review instruction that includes basic requirements for paper quality evaluation.
[0089] In an exemplary embodiment, the construction model invocation request in step S103 specifically includes: Server 200 constructs a multi-turn message structure as model input. For example... Figure 4 As shown, this multi-round message structure consists of three parts: The first round consists of a system message. This message carries security rules and output specifications, and injects the runtime system date of server 200 as the base time. The base time is used to eliminate temporal misjudgments caused by training deadlines in large language models. For example, it enables the model to use the current date as a reference to determine whether the timeliness of statements in a paper is accurate.
[0090] The second round is a pre-set confirmation message for the Assistant Role; this message is a fixed phrase (such as "Okay, I will review the paper as required"), the purpose of which is to make the subsequent paper content understood by the model as data objects to be evaluated, rather than new instructions, thereby downgrading any instructional text appearing in the paper to data to be evaluated, improving the objectivity of the scoring and the ability to resist prompt injection.
[0091] The third round consists of the paper's main text message for the user role. This message contains the entire paragraph text of the structured model and can be truncated to a preset token limit to avoid exceeding the model's context window limit. The prompt word templates corresponding to the review dimensions are embedded in the system message or the user role's paper's main text message.
[0092] In an exemplary embodiment, the invocation of the large language model in step S103 specifically includes: Server 200 uses a multi-threaded, parallel approach to evaluate the paper document across various review dimensions using a large language model. The calls to each review dimension are independent and concurrent, minimizing overall review time. Each review dimension independently returns a percentage score (e.g., 85 points), a dimension grade (e.g., A), review comments (a descriptive paragraph), and several suggested revisions (listed individually). The token consumption for each review dimension is atomically accumulated across threads for metering and billing. Review results for each dimension are stored separately in a database, ensuring that each dimension is independent and that a failure in one review dimension does not block the execution of other review dimensions.
[0093] In an exemplary embodiment, step S103 is followed by a step of structured recovery of the text returned by the large language model.
[0094] Specifically, the returned text for each evaluation dimension from the large language model is often in pseudo-JSON format, containing code fences, unescaped quotes, control characters, or trailing commas. Server 200 first removes the code fences from the returned text, extracts the outermost structure using regular expressions, and then attempts to parse it as standard JSON.
[0095] When standard parsing fails, server 200 extracts data field by field according to a predefined dimensional field structure. The dimensional field structure predefines the field names (such as "score", "level", "comment", "suggestions") and their data types that should be included in the results of each review dimension. Server 200 uses the position of the next known field name in the dimensional field structure as the right boundary of the current field value, and a combination of right quotes, commas, and left quotes as array element separators to escape and reconstruct the current field value into a valid JSON structure. This structured recovery process can stably recover evaluation data without retrying the large language model, avoiding the loss of single-dimensional review results.
[0096] S104. Determine the total score of the paper document based on the weight and dimension score corresponding to each of the review dimensions, and map the total score of the paper document to the evaluation level of the paper document according to the preset score threshold.
[0097] Specifically, server 200 reads the weights of each review dimension from the review dimension set and constructs a mapping relationship between dimension identifiers and weights.
[0098] For each of the aforementioned review dimensions that has been reviewed and whose scores are not empty, multiply each dimension score by its corresponding weight and sum the results to obtain a weighted sum; simultaneously, accumulate the weights corresponding to each dimension score to obtain the total weight; finally, divide the weighted sum by the total weight to obtain the paper's total score. This process can be expressed by the following formula: in, This represents the total score of the paper, and i represents the index of the review dimension that has been reviewed and whose score is not empty. This represents the score for the i-th review dimension. This represents the weight corresponding to the i-th review dimension. When the total weight is zero (e.g., all dimensions have zero weights or no dimensions have been reviewed), the total score of the paper is zero. The total score of the paper is rounded to two decimal places.
[0099] With dimensions that can be dynamically added or removed, the calculation method for the total score of the paper can be kept consistent under different dimension configurations by normalizing the calculation based on the weights of the dimensions participating in the review.
[0100] Server 200 maps the total score of the paper to the evaluation level of the paper document according to a preset score threshold. Specifically: When the total score of the paper is not lower than the first threshold, the server 200 will determine the evaluation level as A (excellent); when the total score of the paper is lower than the first threshold but not lower than the second threshold, the evaluation level will be determined as B (good); when the total score of the paper is lower than the second threshold but not lower than the third threshold, the evaluation level will be determined as C (medium); when the total score of the paper is lower than the third threshold, the evaluation level will be determined as D (failing).
[0101] The first, second, and third thresholds are configurable; for example, the first threshold can be configured to 90 points, the second to 75 points, and the third to 60 points. The dimensional scores for each review dimension are also mapped to dimensional levels according to the above rules, ensuring that each dimension has both quantitative and qualitative scores.
[0102] Server 200 will include the paper's total score, evaluation level, scores for each dimension, and dimension levels in the review results, serving as an interpretable basis for quantitative evaluation.
[0103] After completing the dimension review, the state machine proceeds to the third stage, "paragraph (line-by-line) review", which includes the following steps S105 to S106.
[0104] S105. Input the text of each paragraph of the structured model into the large language model in batches, and obtain the question list returned by the large language model; the question list contains at least one question entry; the question entry contains the original text of the question fragment and the annotation content.
[0105] Specifically, server 200 divides all paragraphs of the structured model into multiple batches according to a preset batch size (e.g., 5 paragraphs per batch). Each paragraph is marked with its paragraph identifier and then input into the large language model. For each batch, server 200 independently calls the large language model. If a single batch call fails, that batch is skipped without affecting the processing of other batches or the continuation of the entire review process.
[0106] The large language model returns a list of issues with paragraph identifiers for problematic paragraphs. Each issue entry includes the corresponding dimension (the review dimension corresponding to the issue, such as "logical structure"), severity (such as "serious", "moderate", "minor"), issue description (a textual explanation of the issue), suggested revisions (suggestions on how to revise the issue), and the original text of the issue fragment (a textual fragment containing the issue that is precisely copied from the original paper).
[0107] S106. For each of the aforementioned question entries, locate the start and end offsets of the characters in the original text of the question fragment in the paragraph text corresponding to the question entry, and associate and save the character start and end offsets with the annotation content.
[0108] Specifically, upon receiving each issue entry, server 200 immediately performs a string matching search within the paragraph text to which the issue fragment belongs, using the original text of the issue fragment as the key. If a complete match is found, server 200 records the start and end character indices of the issue fragment within the paragraph text, serving as character start and end offsets. These character start and end offsets are recorded prior to the generation of the report document and are therefore unaffected by paragraph position shifts caused by subsequent insertion of report content. These character start and end offsets are saved to the database along with the issue results, serving as the primary relocation basis when subsequently writing annotations to the report document.
[0109] After the paragraph-level problem list and the pre-stored character start and end offsets are written into the database, the state machine advances the task state to the next stage, "report generation".
[0110] S107. Generate a report document based on the dimension scores corresponding to each of the aforementioned review dimensions, the total score of the paper document, and the evaluation level.
[0111] Specifically, server 200 generates a review report document based on the structured model generated in step S101, and writes the total paper score, evaluation level, scores and levels of each dimension, review comments and comprehensive evaluation obtained in step S104 into the review report document. Inserting report content into the review report document here will cause the overall position of the original paragraphs to shift, so repositioning and correction are required in the subsequent step S108.
[0112] S108. In the report document, according to the start and end offsets of the characters corresponding to each of the problem entries, each of the annotation contents is anchored to the corresponding text position in the report document.
[0113] like Figure 5 As shown, in an exemplary embodiment, step S108 specifically includes steps S1081 to S1085: S1081. Before writing annotations into the report document, perform text prefix matching on the paragraph text corresponding to the issue entry in the report document to locate the position of the paragraph text in the report document.
[0114] Specifically, before writing each comment to the report document, server 200 uses the original text of the paragraph to which the issue entry belongs as the matching key value, performs an exact match in the generated report document by text prefix, and determines the matched position as the current position of the paragraph in the report document. This step is used to correct the overall paragraph count drift caused by the insertion of report content in step S107.
[0115] S1082. Filter out non-content layout units within the paragraph text and establish character offset mappings for each content layout unit.
[0116] Specifically, paragraph text is distributed across multiple run units in the report document. A run unit is a continuous text segment in the document that has uniform formatting attributes (such as font, font size, color, etc.). Server 200 filters out non-content run units within the paragraph (such as run units that only contain spaces or punctuation marks and do not carry actual text content), and establishes a character start and end offset mapping for each content run unit within the paragraph.
[0117] S1083. Based on the character start and end offsets, determine the target typesetting unit into which the original text of the problem fragment falls, split the target typesetting unit at the position corresponding to the character start and end offsets, and divide the original text of the problem fragment into independent typesetting units.
[0118] Specifically, server 200 locates the target layout unit where the character start and end offsets of the problem fragment pre-stored in step S106 fall. Within this target layout unit, the original problem fragment is split at the start and end positions of the character start and end offsets, thus dividing it into an independent layout unit. During the split, the layout attributes of the original layout unit are copied, and the leading and trailing spaces of the original problem fragment are preserved.
[0119] S1084. Insert a note interval start mark before the segmented independent typesetting unit, insert a note interval end mark and a note reference mark after the independent typesetting unit, and write the note content into the independent note component.
[0120] Specifically, server 200 inserts a comment range start marker (e.g., a commentRangeStart marker in DOCX format) before the independent typesetting unit segmented in step S1083, and inserts a comment range end marker (e.g., a commentRangeEnd marker) and a comment reference marker (e.g., a commentReference marker) after the independent typesetting unit. The comment content is then written to a separate comment component (e.g., a comments.xml component in DOCX format). Through these markers, the comment content is precisely anchored to the corresponding text fragment in the original document, which is then highlighted in the document, and the comment content is displayed in the comment pane on the right side of the document.
[0121] When there are multiple problem entries in the same paragraph, the server 200 inserts annotation marks in descending order of the starting offset of each problem entry (i.e., from back to front) to avoid the earlier inserted annotations changing the position offset of subsequent unprocessed segments.
[0122] S1085. A progressively downgraded strategy is adopted for matching the original text of the problem fragment in the paragraph text.
[0123] Specifically, the matching of the original text of the question fragment within the paragraph text is progressively downgraded according to the following four levels: The first level involves precise matching of the original text of the problem segment, that is, matching every character exactly.
[0124] The second level involves normalizing the quotation marks in the original text of the problem segment before attempting a match when an exact match fails. Quotation mark normalization involves converting Chinese quotation marks (“” and '’) to English quotation marks (" " and ' ') before attempting a match to resolve inconsistencies in quotation mark characters caused by different input methods or typesetting environments.
[0125] The third level involves normalizing the spaces in the original text of the problematic segment before attempting to match it, especially when quotation mark normalization fails. Space normalization involves converting full-width spaces to half-width spaces and merging multiple consecutive spaces into a single space before attempting to match it, in order to resolve inconsistencies in space formatting during typesetting.
[0126] In the fourth stage, when the space normalization match fails, the system falls back to the character start and end offsets pre-stored in step S106 for positioning.
[0127] When all four levels fail, the annotation content will be anchored to the entire paragraph, meaning the entire paragraph will be used as the annotation scope.
[0128] The following combination Figure 6 The precise character-level anchoring mechanism across typesetting units in this application is further explained. Figure 6 This example illustrates the complete process of precise anchoring of Word annotations across the Run character level.
[0129] like Figure 6 As shown, in the original paragraph, the text is scattered across multiple run units. Run A, Run B, Run C, and Run D are illustrated in the figure. The target segment to be anchored (in orange) spans two run units, Run B and Run C. Because the start and end positions of the target segment are located within the run units rather than at their boundaries, annotations cannot be directly located using the run unit boundaries; therefore, character-level splitting of the run units is necessary.
[0130] Server 200 first filters non-content layout units within a paragraph (e.g., layout units containing only spaces or punctuation marks and not actual text content), establishing a character offset mapping for each content layout unit, i.e., establishing the start and end offsets of each content layout unit within the paragraph. Then, based on the pre-stored startOffset and endOffset of the problem fragment, the server locates which layout unit each start and end offset falls into, determining the range of layout units spanned by the target fragment. During this process, the server employs a strategy combining text prefix matching (findParagraphByText) with non-empty paragraph count matching to locate the current position of the paragraph text within the generated report document, resolving the paragraph drift issue caused by report content insertion.
[0131] Server 200 splits the typesetting unit at character offsets (splitCtrAt). Specifically, at the start offset of the target segment, the server splits the typesetting unit into the first half and an independent target segment typesetting unit; at the end offset of the target segment, it splits the typesetting unit into an independent target segment typesetting unit and the second half. After splitting, the target segment is precisely divided into an independent typesetting unit.
[0132] Server 200 uses a document programming interface (such as XmlCursor.moveXml) to insert a comment range start marker (e.g., w:commentRangeStart in DOCX format) before the independent typesetting unit, and a comment range end marker (e.g., w:commentRangeEnd) and a comment reference marker (e.g., w:commentReference) after the independent typesetting unit. Through the combination of these three markers, the comment content is precisely anchored to the corresponding text fragment in the original text.
[0133] When multiple annotations exist within the same paragraph, server 200 inserts annotation markers sequentially in descending order of the starting offset of each annotation entry (i.e., from back to front). Inserting from back to front prevents previously inserted markers from altering the positional offset of subsequently unprocessed segments, thus ensuring that each annotation is precisely positioned to its corresponding text segment. After each insertion operation, server 200 re-traverses and retrieves the typesetting unit references to prevent reference invalidation due to changes in document structure. If any step in the precise anchoring process fails, the server degenerates into adding annotations to the entire paragraph, i.e., inserting a start marker for the annotation interval at the beginning of the paragraph and an end marker for the annotation interval at the end of the paragraph.
[0134] Server 200 also writes the text of each comment into a separate comment component (e.g., comments.xml in DOCX format). Each comment entry (CTComment) contains metadata such as author information (using the dimension name as the author), initials, and timestamps. The comment text is organized into sections according to "problem / solution," and the level tags are presented in bold. The comment reference tag (commentReference) associates the target segment within a paragraph with the comment entry in comments.xml.
[0135] like Figure 6 As shown in the effect diagram, after the above-mentioned character-level precise anchoring across typesetting units, when users open the review report document using common office software (such as Microsoft Word), the target segment is precisely underlined or highlighted in the text, and the corresponding annotation content of the segment is displayed in the annotation bar on the right side of the document. This achieves character-level precise anchoring between "AI evaluation opinions" and "specific text of the original paper", making the review opinions traceable and operable.
[0136] In one exemplary embodiment, a "comprehensive review" stage may also be included before the "report generation" stage: In the comprehensive review phase, server 200 uses only the abstract text from the structured model and the corresponding dimension scores and review comments for each review dimension as input to construct a comprehensive review model call request, without inputting the full text of the paper into the model. Server 200 calls the large language model to generate comprehensive evaluation opinions, which are an overall evaluation of the paper. Server 200 also performs structured recovery processing on the comprehensive evaluation opinions returned by the large language model and stores them in the database, forming complete review data together with the dimension scores and each issue item. This comprehensive review step obtains a holistic review under a fixed context window and controllable cost, avoiding repeated reasoning on the entire text.
[0137] At this point, the same paper has undergone differentiated review at three granularities: the dimensional level uses the entire text as input for multi-dimensional parallel scoring; the paragraph level identifies problems paragraph by paragraph; and the comprehensive level generates an overall evaluation using only the abstract and dimensional level results as input. Figure 7 As shown, after weighting, aggregating, and assembling the results at different granularities, the process proceeds to the "report generation" stage.
[0138] Throughout the entire process, Server 200 also incorporates multi-level fault tolerance mechanisms. At the message queue layer, a message resubmission mechanism allows messages to be resubmitted if processing fails. At the model invocation layer, a retry mechanism allows for retries a predetermined number of times if a single invocation fails. At the result parsing layer, the aforementioned three-level degradation parsing strategy is implemented. At the batch processing layer, a segment-by-batch skipping mechanism ensures that a single batch failure does not affect the overall process. At the report generation layer, a report generation degradation mechanism allows for regression to a compatible or simplified report version if generating a higher version report fails. Simultaneously, Server 200 provides real-time feedback on the review progress through both cached progress keys and server push notifications.
[0139] Multiple consumer threads in the message queue concurrently retrieve paper messages, with messages for different papers processed by different consumer threads, naturally allowing for parallel processing between papers. The above stages can be performed simultaneously on different papers on server 200, ensuring both line-by-line review accuracy and batch processing throughput. After the report document is generated, server 200 stores the review report document with original line-by-line annotations and all review data in the database, and the state machine sets the paper task status to "complete".
[0140] By implementing steps S101 to S108, this application uses a configurable set of review dimensions to drive a large language model for multi-dimensional evaluation of papers. This ensures that the evaluation criteria are entirely determined by the configured data, and adding or adjusting dimensions requires no modification to the program code. Through independent evaluation of each dimension followed by weighted aggregation and threshold mapping, interpretable quantification from fine-grained dimensional scoring to overall evaluation level is achieved. By identifying problems segment by segment and pre-storing character offsets, annotations are precisely anchored to the original text during report generation using text relocation combined with cross-typesetting unit character-level splitting, achieving a traceable "line-by-line" review style. Furthermore, this application further enhances the stability and objectivity of the review process by performing structured recovery on the model-returned text and using a multi-round message structure with instruction-data separation to defend against prompt word injection.
[0141] Based on the same inventive concept, this application also provides a system for implementing the configurable multi-dimensional intelligent paper evaluation method based on a large language model as described above. The solution provided by this system is similar to the implementation scheme described in the above method. In an exemplary embodiment, such as... Figure 8 As shown, a configurable, multi-dimensional intelligent paper evaluation system based on a large language model is provided, including the following functional modules: The document parsing module is used to parse the thesis document and obtain a structured model; the structured model includes the paragraph identifier and paragraph text corresponding to each paragraph of the thesis document.
[0142] The dimension configuration module is used to obtain the set of review dimensions corresponding to the configuration identifier from the configuration storage according to the configuration identifier in the review request; the set of review dimensions contains at least one review dimension, and each review dimension corresponds to a weight and a prompt word template.
[0143] The dimension review module is used to construct a model call request for any review dimension based on all paragraph text of the structured model and the prompt word template corresponding to the review dimension. It then calls the large language model to evaluate the paper document for the review dimension and obtains the dimension score under the review dimension.
[0144] The aggregation and grading module is used to determine the total score of the paper document based on the weight and dimension score corresponding to each of the review dimensions, and to map the total score of the paper document to the evaluation level of the paper document according to a preset score threshold.
[0145] The paragraph review module is used to input the paragraph text of the structured model into the large language model in batches and obtain the question list returned by the large language model; the question list contains at least one question entry; the question entry contains the original text of the question fragment and the annotation content.
[0146] The offset positioning module is used to locate the start and end offsets of characters in the original text of the question segment in the paragraph text corresponding to each question entry, and to associate and save the start and end offsets of characters with the annotation content.
[0147] The report generation module generates a report document based on the dimension scores corresponding to each of the review dimensions, the total score of the paper document, and the evaluation level. In the report document, according to the character start and end offsets corresponding to each of the question items, the annotation content is anchored to the corresponding text position in the report document.
[0148] certainly, Figure 8 The architecture shown is merely exemplary; it can be omitted as needed when implementing different functionalities. Figure 8 One or at least two components of the system shown.
[0149] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 9As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores intermediate and result data during the intelligent paper evaluation process. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it can implement the configurable multi-dimensional intelligent paper evaluation method based on a large language model provided in the previous embodiment.
[0150] Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0151] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.
[0152] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.
[0153] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.
[0154] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with relevant regulations and be authorized by the owner of the corresponding device.
[0155] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).
[0156] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.
[0157] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0158] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A configurable, multi-dimensional intelligent evaluation method for papers based on a large language model, characterized in that, include: Obtain the paper documents and review requests, and obtain a structured model by parsing the paper documents; The structured model includes paragraph identifiers and paragraph text corresponding to each paragraph of the paper document; Based on the configuration identifier in the review request, the set of review dimensions corresponding to the configuration identifier is obtained from the configuration storage; the set of review dimensions contains at least one review dimension, and each review dimension corresponds to a weight and a prompt word template; For any review dimension, based on all paragraph text of the structured model and the prompt word template corresponding to the review dimension, construct the model call request corresponding to the review dimension, call the large language model to evaluate the paper document for the review dimension, and obtain the dimension score under the review dimension. Based on the weights and scores of each of the review dimensions, the total score of the paper is determined, and the total score is mapped to the evaluation level of the paper according to a preset score threshold. The text of each paragraph of the structured model is input into the large language model in batches to obtain a list of questions returned by the large language model; the list of questions contains at least one question entry; the question entry contains the original text of the question fragment and annotation content; For each of the aforementioned question entries, locate the start and end character offsets of the original text of the question fragment in the paragraph text corresponding to the question entry, and associate and save the character start and end offsets with the annotation content; A report document is generated based on the dimension scores corresponding to each of the aforementioned review dimensions, the total score of the paper, and the evaluation level. In the report document, each annotation is anchored to the corresponding text position in the report document according to the character start and end offsets corresponding to each of the problem entries.
2. The configurable multi-dimensional intelligent paper evaluation method based on a large language model according to claim 1, characterized in that, The paper document was parsed to obtain a structured model, including: Based on the file format of the paper document, the target parsing engine is determined from multiple parsing engines; The paper document is parsed by the target parsing engine to obtain the paragraph text of each paragraph and the corresponding layout information of the paragraph text; Based on the layout information described in each paragraph, determine the chapter level of each paragraph; Assign a globally consecutive paragraph identifier to each paragraph according to the order in which the paragraphs are arranged in the paper document; The structured model is generated based on the paragraph text, chapter level, and paragraph identifier of each paragraph; the structured model includes the paragraph identifier, chapter level, and paragraph text corresponding to each paragraph.
3. The configurable multi-dimensional intelligent paper evaluation method based on a large language model according to claim 2, characterized in that, Based on the file format of the paper document, the target parsing engine is determined from multiple parsing engines, specifically including: When the paper document is a new type of document with style identifiers, the style-driven parsing engine is determined to be the target parsing engine. The paper document is parsed using the target parsing engine, including: The paper document is read paragraph by paragraph, and the heading level of each paragraph is determined based on the style tags used in each paragraph.
4. The configurable multi-dimensional intelligent paper evaluation method based on a large language model according to claim 2, characterized in that, Based on the file format of the paper document, the target parsing engine is determined from multiple parsing engines, specifically including: When the paper document is an old-style document without style identifiers, the number-driven parsing engine is determined to be the target parsing engine; The paper document is parsed using the target parsing engine, including: The chapter number regular expression is used to match the beginning of each paragraph, and the chapter level of each paragraph is inferred based on the matched chapter number pattern.
5. The configurable multi-dimensional intelligent paper evaluation method based on a large language model according to claim 1, characterized in that, Based on the configuration identifier in the review request, retrieve the set of review dimensions corresponding to the configuration identifier from the configuration storage, specifically including: Extract the configuration identifier from the review request; Retrieve all review dimension records corresponding to the configuration identifier from the configuration storage; Filter out the review dimension records that are in the enabled state from all review dimension records; The evaluation dimension set is obtained by sorting the records of each evaluation dimension in ascending order according to their sorting numbers; each evaluation dimension record contains a dimension name, dimension identifier, dimension category, weight, highest score, and custom prompt word template.
6. The configurable multi-dimensional intelligent paper evaluation method based on a large language model according to claim 1, characterized in that, The model invocation request corresponding to any of the aforementioned review dimensions includes: System messages, carrying security rules and output specifications, and injecting the server's runtime system date as the base time; Pre-set confirmation messages for the assistant role; The paper text message for the user role includes all paragraph text of the structured model; wherein, the prompt word template corresponding to the review dimension is embedded in the system message or the paper text message for the user role.
7. The configurable multi-dimensional intelligent paper evaluation method based on a large language model according to claim 1, characterized in that, The paper is evaluated across various review dimensions using a large language model in a multi-threaded, parallel manner. Each of the aforementioned review dimensions independently returns a 100-point scale score, dimension level, review comments, and modification suggestions; The token consumption corresponding to each of the aforementioned review dimensions is accumulated atomically across threads for metering and billing purposes. The review results for each of the aforementioned review dimensions are stored in the database separately. The review dimensions do not affect each other, and a failure to call a single dimension does not block other dimensions.
8. The configurable multi-dimensional intelligent paper evaluation method based on a large language model according to claim 5, characterized in that, Based on the weights and scores corresponding to each of the aforementioned review dimensions, the total score for the paper document is determined, including: Construct a mapping relationship between dimension identifiers and weights; For each of the review dimensions that has been reviewed and whose dimension scores are not empty, multiply each dimension score by its corresponding weight and sum them to obtain a weighted sum. At the same time, accumulate the weights corresponding to each dimension score to obtain the total weight. The total score of the paper is obtained by dividing the weighted sum by the total weight; when the total weight is zero, the total score of the paper is zero. Mapping the total score of the paper to an evaluation level for the paper document based on a preset score threshold includes: When the total score of the paper is not lower than the first threshold, the evaluation level is determined to be A. When the total score of the paper is lower than the first threshold and not lower than the second threshold, the evaluation level is determined to be B. When the total score of the paper is lower than the second threshold but not lower than the third threshold, the evaluation level is determined to be C. When the total score of the paper is lower than the third threshold, the evaluation level will be determined as D.
9. The configurable multi-dimensional intelligent evaluation method for papers based on a large language model according to claim 1, characterized in that, In the report document, based on the start and end offsets of the characters corresponding to each of the aforementioned issue entries, each annotation is anchored to its corresponding text position in the report document, specifically including: Before writing comments into the report document, the paragraph text corresponding to the problem entry is used to perform text prefix matching in the report document to locate the position of the paragraph text in the report document. Filter out non-content layout units within the paragraph text and establish character offset mappings for each content layout unit; Based on the character start and end offsets, determine the target layout unit into which the original text of the question fragment falls, split the target layout unit at the position corresponding to the character start and end offsets, and divide the original text of the question fragment into independent layout units; Insert a comment interval start mark before the segmented independent typesetting unit, and insert a comment interval end mark and a comment reference mark after the independent typesetting unit, and write the comment content into the independent comment component; When there are multiple problem entries in the same paragraph, annotation marks are inserted sequentially from back to front in descending order of the starting offset of each problem entry.
10. A configurable, multi-dimensional intelligent paper evaluation system based on a large language model, characterized in that, include: The document parsing module is used to parse academic papers and obtain a structured model. The structured model includes paragraph identifiers and paragraph text corresponding to each paragraph of the paper document; The dimension configuration module is used to retrieve the set of review dimensions corresponding to the configuration identifier from the configuration storage according to the configuration identifier in the review request; the set of review dimensions contains at least one review dimension, and each review dimension corresponds to a weight and a prompt word template. The dimension review module is used to construct a model call request corresponding to any review dimension based on all paragraph text of the structured model and the prompt word template corresponding to the review dimension, call the large language model to evaluate the paper document for the review dimension, and obtain the dimension score under the review dimension. The aggregation and grading module is used to determine the total score of the paper document based on the weight and dimension score corresponding to each of the review dimensions, and to map the total score of the paper document to the evaluation level of the paper document according to a preset score threshold. The paragraph review module is used to input the paragraph text of the structured model into the large language model in batches and obtain the question list returned by the large language model; the question list contains at least one question entry; the question entry contains the original text of the question fragment and the annotation content; The offset positioning module is used to locate the start and end offsets of characters in the original text of the question fragment in the paragraph text corresponding to each question entry, and to associate and save the start and end offsets of characters with the annotation content. The report generation module generates a report document based on the dimension scores corresponding to each of the aforementioned review dimensions, the total score of the paper document, and the evaluation level. In the report document, each annotation is anchored to the corresponding text position in the report document according to the character start and end offsets corresponding to each of the problem entries.