Intelligent review scoring model optimization method

Abstract

The invention relates to the technical field of data review, and discloses an intelligent review scoring model optimization method, which comprises the following steps: acquiring review elements, and constructing a review knowledge graph according to the review elements; extracting historical bidding files and expert comment data, and associating the historical bidding files with the expert comment data; generating a training data set according to the review knowledge graph and a historical bidding file; according to the training data set, training the basic model to obtain a first model; obtaining expert experience data, and correcting the first model to obtain a second model; a simulation bidding document is constructed, and a simulation review result is obtained through review of the second model; and correcting the model according to a difference value between the simulation review result and a standard review result to obtain a third model. According to the method, the historical data is efficiently utilized, comprehensive and deepened understanding of the review rules is realized, one-to-one correspondence of the bid invitation files and the review standards can be realized, and the reliability and comprehensiveness of the bid evaluation result are improved.

Description

Technical Field

[0001] This application relates to the field of data review technology, and in particular to an optimization method for an intelligent review and scoring model. Background Technology

[0002] In today's deepening digital transformation, traditional manual review models are increasingly revealing their pain points such as inefficiency and strong subjectivity. AI-powered intelligent review systems, based on deep learning and natural language processing technologies, have built a data-driven, quantifiable, and traceable automated evaluation system, which is reshaping review processes across multiple industries.

[0003] In existing technologies, the utilization of historical bidding data often remains at the level of simple archiving and retrieval, lacking in-depth structured analysis and correlation mining. This results in the failure to fully release the value of a large amount of data containing expert experience and review logic, and the inability to efficiently transform it into high-quality material for training intelligent models. Simultaneously, traditional methods rely heavily on manual interpretation and static entries to understand review rules, making it difficult to systematically and structurally depict the complex logical relationships and multi-layered elements. This leads to subjective differences and misunderstandings in the execution of review standards. Furthermore, during the review process, accurately mapping specific bidding document content to abstract review standards relies heavily on the personal experience and instantaneous judgment of review experts. This is not only inefficient but also makes it difficult to guarantee the completeness and consistency of the mapping, potentially leading to omissions or misjudgments. These factors collectively result in significant bottlenecks in the reliability, objectivity, and comprehensiveness of the output results of traditional or early-generation automated bidding methods, restricting the in-depth application of intelligent review technology to highly reliable and highly available critical business scenarios. Summary of the Invention

[0004] To address the aforementioned technical issues, this application provides an intelligent evaluation and scoring model optimization method, which aims to achieve efficient utilization of historical data, a comprehensive and in-depth understanding of evaluation rules, and a one-to-one correspondence between bidding documents and evaluation standards, thereby improving the reliability and comprehensiveness of the evaluation results.

[0005] In some embodiments of this application, an intelligent review and scoring model optimization method is provided, characterized by comprising:

[0006] Obtain the review elements and construct a review knowledge graph based on the review elements;

[0007] Extract historical bid documents and expert comments data, and correlate the historical bid documents with the expert comments data;

[0008] A training dataset is generated based on the aforementioned review knowledge graph and historical tender documents;

[0009] Based on the training dataset, the base model is trained to obtain the first model;

[0010] By acquiring expert experience data, the first model is corrected to obtain the second model;

[0011] A mock tender document is constructed, and the mock review results are obtained through the second model review.

[0012] The model is corrected based on the difference between the simulated review results and the standard review results to obtain the third model.

[0013] In some embodiments of this application, obtaining review elements and constructing a review knowledge graph based on the review elements includes:

[0014] Obtain the review criteria documents;

[0015] The review criteria document is processed to extract review elements, resulting in first-class elements and second-class elements;

[0016] Establish a corresponding vocabulary and logical relationship data for each type of element;

[0017] The first and second types of elements are set as the review knowledge graph.

[0018] In some embodiments of this application, the association of expert comment data with the historical tender documents includes:

[0019] Extracting historical bid documents in a structured manner;

[0020] The expert comments data are divided into various review elements according to the review knowledge graph, and the scores of each review element are obtained.

[0021] Identify each score data in the historical bid documents and mark the corresponding review elements to the corresponding score data;

[0022] The expert comments data are extracted and labeled with the corresponding score data.

[0023] In some embodiments of this application, expert comment data is extracted and labeled with corresponding score data, including:

[0024] The expert comments data are structured and extracted to obtain evaluation corpus data and logical corpus data;

[0025] The evaluation corpus data is compared with the vocabulary database;

[0026] Identify the review elements corresponding to each evaluation corpus data;

[0027] The evaluation corpus data is labeled with the corresponding score data;

[0028] The logical corpus data is compared with the logical relation data;

[0029] Based on logical relationship data, the score data corresponding to each logical corpus data is correlated.

[0030] In some embodiments of this application, the division of various review elements includes:

[0031] Obtain the vocabulary database of expert comments and the review knowledge graph;

[0032] Extract key information from expert comments data;

[0033] Compare the similarity of each key piece of information with each of the aforementioned vocabulary databases;

[0034] The key information is set as the review element corresponding to the highest similarity.

[0035] In some embodiments of this application, modifications are made to obtain a second model, including:

[0036] Obtain signal data for expert review;

[0037] Divide the tender documents into various modules;

[0038] Extract the dwell time and scrolling speed of experts in each module from the signal data, and extract the review order of each module;

[0039] The review importance score for each module is calculated based on the dwell time and scrolling speed.

[0040] The review priority of each module is determined based on its importance and review order.

[0041] In some embodiments of this application, the tender document is divided into various modules, including:

[0042] The tender documents are first divided according to the document structure to obtain multiple initial modules;

[0043] Identify the core review elements corresponding to each initial module, and set the initial modules with the same core elements as logical review modules.

[0044] The logic review module is divided according to the review order during the review process.

[0045] In some embodiments of this application, the review importance score of each module is calculated, including:

[0046] Set a time threshold to perform primary filtering on dwell time;

[0047] Obtain scrolling data and analyze the scrolling behavior corresponding to each scrolling data point based on behavioral patterns.

[0048] The dwell time is filtered in two stages based on the scrolling behavior to obtain the effective dwell time;

[0049] Different weights are assigned to the effective dwell time and scrolling behavior to generate review importance scores for each module.

[0050] In some embodiments of this application, the review priority of each module is determined, including:

[0051] The order score for each module is set according to the review order of each module;

[0052] A comprehensive score is generated based on the order score and importance score of each module;

[0053] The review priorities are generated by sorting the scores based on the overall scores.

[0054] In some embodiments of this application, a third model is obtained by correcting the model based on the difference between the simulated review result and the standard review result, including:

[0055] Set the corresponding standard review results for the simulated tender document;

[0056] The standard review results include scores and comments;

[0057] A reward signal is set based on the difference between the standard review result score and the simulated review result score.

[0058] A reward value is generated based on the reward signal, and the parameters of the second model are corrected based on the reward value to obtain the third model.

[0059] Compared with existing technologies, the intelligent review and scoring model optimization method in this application has the following advantages:

[0060] A standardized process, encompassing data cleaning, knowledge injection, and model training, was established, enabling efficient and in-depth utilization of multi-source, heterogeneous historical data. Furthermore, the systematic construction and fusion of the review knowledge graph laid the foundation for the model's scientific validity. Specifically, this invention not only automatically parses and associates massive amounts of historical tender documents and expert comments, but also, by constructing a structured review knowledge graph, makes scattered review rules, keyword libraries, and logical relationships explicit and systematic. This process transforms model training from a simple "data fitting" into "supervised learning" based on clear domain knowledge. This method ensures high quality and consistency of the training dataset, enabling the trained basic model to deeply understand review elements and achieve a precise one-to-one correspondence between the content of the tender documents and the review standard clauses, thereby improving the standardization of the bidding process and the reliability of the model's initial decisions from the outset.

[0061] Secondly, by innovatively introducing expert behavioral signal analysis and combining it with knowledge graphs, this invention achieves a deep fusion of implicit human expert experience and explicit rule knowledge, significantly improving the model's comprehensiveness and intelligence. This invention breaks through the traditional limitation of using only the final score as a training label. By capturing and analyzing behavioral signals such as dwell time and reading order during the expert review process, it quantifies the expert's focus and review habits, and injects these implicit experiences into the model through attention guidance and priority weighting. This enables the optimized model not only to apply written rules but also to mimic efficient expert review strategies and risk insight patterns, achieving a more comprehensive and human-expert-like understanding and evaluation of tender documents, enhancing the comprehensiveness and deep credibility of the review results.

[0062] Finally, by establishing a dynamic simulation evolution mechanism based on reinforcement learning and designing a multi-dimensional reward function, the model is endowed with continuous self-optimization and adaptability, fundamentally ensuring the long-term reliability and stability of the evaluation results. Attached Figure Description

[0063] Figure 1 This is a flowchart illustrating an intelligent review and scoring model optimization method in an embodiment of this application. Detailed Implementation

[0064] The specific embodiments of this application will be described in further detail below with reference to the accompanying drawings and examples. The following examples are used to illustrate this application, but are not intended to limit the scope of this application.

[0065] In the description of this application, it should be understood that the terms "center", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0066] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0067] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0068] like Figure 1 As shown in the figure, an intelligent review and scoring model optimization method according to an embodiment of this application includes:

[0069] Obtain the review elements and construct a review knowledge graph based on the review elements;

[0070] Extract historical bid documents and expert comments data, and correlate the historical bid documents with the expert comments data;

[0071] A training dataset was generated based on the review knowledge graph and historical tender documents.

[0072] The first model is obtained by training the base model based on the training dataset.

[0073] By acquiring expert experience data, the first model is revised to obtain the second model;

[0074] A mock tender document is constructed, and the mock review results are obtained through a second model review.

[0075] The model is corrected based on the difference between the simulated review results and the standard review results to obtain the third model.

[0076] In some embodiments of this application, obtaining review elements and constructing a review knowledge graph based on the review elements includes:

[0077] Obtain the review criteria documents;

[0078] The review criteria documents are analyzed to extract review elements, resulting in Category I and Category II elements.

[0079] Establish a corresponding vocabulary and logical relationship data for each type of element;

[0080] The first and second categories of elements are set as the review knowledge graph.

[0081] Specifically, "obtaining the evaluation criteria document" refers to extracting the "Evaluation Method" section of the "Tender Documents" from the bidding management system. This section specifies the dimensions, scores, specific requirements, and scoring criteria of the evaluation in natural language, and serves as the authoritative source of knowledge for this method.

[0082] Specifically, mock tenders can be generated using a generative model (such as GPT) based on a corpus of historical tender documents, or written by domain experts based on typical scenarios. Standard review results can be generated by a pre-trained high-precision benchmark model, or determined by a review panel composed of multiple senior experts.

[0083] Specifically, the extraction of review elements from the review criteria documents is achieved through natural language processing technology, which is used to perform hierarchical analysis of the evaluation methods.

[0084] Specifically, one type of element corresponds to the main categories of the review, such as "Technical Review", "Business Review", and "Price Review".

[0085] Specifically, the second category of elements consists of specific scoring items under the first category of elements. For example, under "Technical Part Review", sub-items such as "Technical Solution", "Project Team", "Implementation Plan" and "Quality Assurance" are further extracted.

[0086] Specifically, a corresponding vocabulary and logical relationship data are set for each type of element. For each type of element, domain experts or by analyzing historical comments, a vocabulary strongly related to that element is summarized. For example, the vocabulary for "project team" could include: "project manager," "technical lead," and "senior engineer." The logical relationship data defines the constraints or association rules in the review process, such as: "If the 'technical solution' mentions the use of 'BIM technology,' then the 'project team' should include personnel with 'BIM engineer' qualifications." This is a cross-element verification logic.

[0087] Specifically, setting the first-class and second-class elements as a review knowledge graph involves storing and representing the aforementioned structured information in graph form. In the knowledge graph, each "first-class element" and "second-class element" is a node. Nodes are connected by edges; for example, an edge representing the "containment" relationship connects the "Technical Part Review" node and the "Technical Solution" node. The attributes of each second-class element node store its corresponding "score weight," "vocabulary," and "logical relationship data." This graph provides the machine with a structured framework for understanding review criteria.

[0088] In some embodiments of this application, the association of expert comment data with historical tender documents includes:

[0089] Extracting historical bid documents in a structured manner;

[0090] The expert comments data are divided into various review elements according to the review knowledge graph, and the scores of each review element are obtained.

[0091] Identify each score data point in historical bid documents and mark the corresponding review elements onto the corresponding score data points;

[0092] The expert comments data are extracted and labeled with the corresponding score data.

[0093] Specifically, "structured extraction of historical bid documents" refers to using document parsing technologies (such as OCR, PDF, and Word) to convert unstructured bid documents into semi-structured or structured data. The extracted content includes not only plain text but also document metadata, such as chapter titles, paragraphs, tables, and lists, forming an internal representation that preserves the original document's logical hierarchy (such as chapters, sections, clauses, and paragraphs), facilitating accurate information location later.

[0094] Specifically, parsing the expert rating table involves dividing the expert comments data into various review elements and obtaining the scores for each element. The rating table is typically a table where rows correspond to review elements and columns correspond to expert scores. The system uses pattern matching or rules to map the text descriptions in the rating table to nodes in the review knowledge graph, determining the specific "secondary element" corresponding to each row and reading its corresponding score.

[0095] Specifically, the key step in achieving the "content-score" association is "identifying each score data point in historical bid documents and tagging the corresponding review elements to the corresponding score data." The system needs to locate the specific text area in the structured, extracted bid documents that supports or leads to a particular score. For example, for "project team score of 8 points," the system needs to find the paragraph describing the project manager's qualifications in relevant sections such as the "Key Personnel Resumes" section of the bid document through information retrieval or rule matching. Then, it tags this text area with the review element "project team" and links it to the score value "8 points," forming a triple <text fragment, review element, score>.

[0096] Specifically, the process of "extracting expert comments and labeling them with corresponding score data" involves associating the experts' written opinions with the aforementioned triples. The experts' free-text comments (such as "The project manager is experienced, but the team configuration is somewhat weak") are extracted and bound to the corresponding "review elements" and "scores," thus forming a complete training unit: <text fragment, review element, score, expert comments>.

[0097] In some embodiments of this application, expert comment data is extracted and labeled with corresponding score data, including:

[0098] The expert comments data were structured and extracted to obtain evaluation corpus data and logical corpus data;

[0099] The evaluation corpus data was compared with the vocabulary database;

[0100] Identify the review elements corresponding to each evaluation corpus data;

[0101] Label the evaluation corpus data with the corresponding score data;

[0102] Compare logical corpus data with logical relation data;

[0103] Based on logical relationship data, the score data corresponding to each logical corpus data is correlated.

[0104] Specifically, the phrase "structural extraction of expert comments to obtain evaluation corpus data and logical corpus data" refers to in-depth analysis of the comments using natural language processing technology. "Evaluation corpus data" consists of descriptive phrases directly expressing praise or criticism, such as "experienced," "detailed plan," and "flaws exist." "Logical corpus data" comprises phrases or sentence structures expressing causal, conditional, and adversative relationships, such as "because…therefore…," "although…but…," and "if…then…," revealing the reasoning chains between different rating items.

[0105] Specifically, the processes of "comparing the evaluation corpus data with the vocabulary" and "identifying the corresponding review elements" constitute a semantic matching process. The system calculates the semantic similarity between each piece of "evaluation corpus data" and the vocabulary of each type II element in the review knowledge graph. For example, the comment "the team structure is reasonable" is highly correlated with words such as "team" and "structure" in the vocabulary of the "project team" element, and is therefore identified as an evaluation belonging to the "project team" element.

[0106] Specifically, "annotating the evaluation corpus data to the corresponding score data" means that after determining the review elements corresponding to the evaluation corpus, it is added to the previously established <text fragment, review element, score> triplet to form a richer <text fragment, review element, score, evaluation corpus> record.

[0107] Specifically, the "comparison of logical corpus data and logical relationship data" and "association of scoring data" aim to uncover the deep logic within the comments. The system analyzes the "logical corpus data" and matches or infers it against predefined "logical relationship data" in the knowledge graph. This process can discover and record the dependencies or influence relationships between different scoring items; for example, a low score for "technical solution innovation" might be the reason for a "deduction" action.

[0108] In some embodiments of this application, the division of various review elements includes:

[0109] Obtain the vocabulary database of expert comments and the review knowledge graph;

[0110] Extract key information from expert comments data;

[0111] Compare the similarity of each key piece of information with each vocabulary database;

[0112] Set the key information as the review element corresponding to the highest similarity.

[0113] Specifically, natural language processing technologies such as named entity recognition and keyword extraction are used to automatically identify core words or phrases that carry the main information from long passages of free text comments. For example, the words "project manager", "ten years" and "bridge construction experience" are extracted from "the project manager has more than ten years of bridge construction experience".

[0114] Specifically, semantic attribution is determined by comparing the similarity of each key piece of information with various vocabularies. The system compares each extracted "key piece of information" with the "vocabulary" of all two types of elements in the review knowledge graph. Similarity calculation can employ word vector cosine similarity, edit distance, or semantic similarity methods based on pre-trained models. For example, the similarity between "bridge construction experience" and the "project team" vocabulary might be higher than its similarity with the "technical solution" vocabulary.

[0115] Specifically, setting key information as the review element with the highest similarity is based on the similarity comparison results, categorizing each key information under the review element with the closest semantics. In this way, even if the expert comments do not explicitly specify the element name, the element classification can be automatically completed through content analysis, providing a foundation for subsequent fine-grained association.

[0116] In some embodiments of this application, modifications are made to obtain a second model, including:

[0117] Obtain signal data for expert review;

[0118] Divide the tender documents into various modules;

[0119] Extract the dwell time and scrolling speed of experts in each module from the signal data, and extract the review order of each module;

[0120] The review importance score for each module is calculated based on dwell time and scrolling speed;

[0121] The review priority of each module is determined based on its importance and review order.

[0122] Specifically, acquiring signal data from expert reviews involves using front-end tracking technology within an electronic review platform that integrates behavior recording capabilities. This allows for the real-time, anonymous collection of human-computer interaction data from experts during the review of electronic tender documents. This data includes, but is not limited to: mouse dwell time in specific page areas, mouse wheel scrolling speed and acceleration, switching sequences and timestamps between different chapter tabs, and operation records of scoring controls (such as sliders and input boxes).

[0123] Specifically, dividing the tender document into modules establishes a reference system for analyzing behavioral data. The tender document is divided into multiple consecutive or logically independent "modules" based on its physical structure and logical content. Each module corresponds to a meaningful part of the document, such as "Chapter 1: Tender Letter and Commitment" or "Chapter 5: Construction Organization Design - Technical Solution".

[0124] Specifically, extracting the expert's dwell time and scrolling speed in each module from the signal data involves spatiotemporal alignment and aggregation of the original behavior logs. The system groups the expert's mouse hover events and scrolling events to the corresponding file modules based on timestamps and page / region coordinates. For each module, it calculates the expert's total dwell time, average dwell time per instance, and characteristics of their scrolling behavior.

[0125] In some embodiments of this application, the tender document is divided into various modules, including:

[0126] The tender documents are first divided according to the document structure to obtain multiple initial modules;

[0127] Identify the core review elements corresponding to each initial module, and set the initial modules with the same core elements as logical review modules.

[0128] The logic review module is divided according to the review order during the review process.

[0129] Specifically, identifying the core review elements corresponding to each initial module involves mapping document fragments to business knowledge. By analyzing the titles and content of the initial modules and using a review knowledge graph for matching, it is determined which "secondary element" the module primarily discusses. For example, the core review element of the module titled "V. Project Management Personnel Staffing" is identified as "Project Team".

[0130] Specifically, designating initial modules with identical core elements as logical review modules addresses situations where the content of the same review element may be scattered across different parts of the document. For example, information about the "project team" might appear in both the "personnel resumes" and "organizational chart" sections. The system aggregates these physically dispersed "initial modules" describing the same element into a logically unified "logical review module," ensuring that the review of that element covers all relevant information.

[0131] Specifically, "dividing the logical review modules according to the review order during the review process" takes into account that experts still have a reading flow within a logical module when reviewing it. For example, when reviewing the "Technical Solution" logical module, experts may first look at the "Overall Overview," then the "Key Technologies," and finally the "Implementation Plan." The system can further divide the sub-contents within the logical module according to expert behavior data or a preset general reading order (such as from top to bottom) to support more refined behavioral analysis.

[0132] In some embodiments of this application, the review importance score of each module is calculated, including:

[0133] Set a time threshold to perform primary filtering on dwell time;

[0134] Obtain scrolling data and analyze the scrolling behavior corresponding to each scrolling data point based on behavioral patterns.

[0135] The dwell time is filtered in two stages based on the scrolling behavior to obtain the effective dwell time;

[0136] Different weights are assigned to effective dwell time and scrolling behavior to generate review importance scores for each module.

[0137] Specifically, "setting a time threshold to filter dwell time" is to eliminate obviously invalid dwell times. For example, setting a minimum dwell time threshold means that dwell events shorter than this time are considered accidental hovering or quick passing and are not counted as valid review time.

[0138] Specifically, "deriving scrolling behavior corresponding to each scrolling data based on behavioral pattern analysis" involves classifying scrolling events into patterns. By analyzing scrolling speed sequences, acceleration, and their correlation with other events (such as clicks and selections), scrolling behavior is classified into several typical patterns, including "intensive reading" (extremely slow and uniform speed), "general browsing" (medium speed), and "rapid page turning / skipping" (very fast speed with large-span displacement).

[0139] Specifically, the system performs a two-stage filtering of dwell time based on scrolling behavior to obtain the effective dwell time, which is achieved by adjusting the scrolling pattern to correct for its effectiveness. For example, a relatively long dwell time, if preceded or followed by "rapid page-turning" behavior, may indicate that the expert was not reading during that period but was interrupted by other things; conversely, if the dwell time is accompanied by slow scrolling in a "careful reading" mode, then this dwell time is highly effective. The system can be designed to design rules to retain only those dwell times surrounded or interspersed with "careful reading" or "general browsing" behaviors, accumulating them into the "effective dwell time."

[0140] In some embodiments of this application, the review priority of each module is determined, including:

[0141] The order score for each module is set according to the review order of each module;

[0142] A comprehensive score is generated based on the order score and importance score of each module;

[0143] The review priority is generated by sorting the overall scores.

[0144] Specifically, setting sequential scores for each module based on their review order reflects a "first-come, first-served" or "strategic review" approach. The first module reviewed is assigned the highest base score (e.g., 100 points), with subsequent modules receiving progressively lower scores, or scores decreasing based on time intervals. Another method is to use the reciprocal of the review order as the score, i.e., 1 point for the first module, 1 / 2 point for the second, and so on.

[0145] Specifically, the comprehensive score is generated by combining the order score (representing strategy) and the importance score (representing input) into a single evaluation indicator.

[0146] Specifically, "generating review priorities by sorting according to overall scores" yields the final sequence of module importance. All modules are sorted from highest to lowest based on their overall scores; modules ranked higher are perceived as having higher relative importance and review priority by experts (as revealed by their behavior). This sequence can be used to guide the optimization of a second model, for example, during model training, allowing the model to allocate more attention or learning weights to the content of high-priority modules.

[0147] In some embodiments of this application, a third model is obtained by correcting the model based on the difference between the simulated review results and the standard review results, including:

[0148] Set the corresponding standard review results for the simulated tender documents;

[0149] The standard review results include scores and comments;

[0150] A reward signal is set based on the difference between the standard review score and the simulated review score.

[0151] A reward value is generated based on the reward signal, and the parameters of the second model are corrected based on the reward value to obtain the third model.

[0152] Specifically, the standard review result for simulated proposals is set as a "reference answer" in the reinforcement learning environment. This standard review result is not a single correct answer, but rather generated by a virtual review committee. This committee consists of multiple agent models with certain differences, trained on a large amount of historical high-performing review data. Each simulated proposal is submitted to this committee, and each agent model reviews it independently. Their output scores are aggregated (e.g., by taking the median) to form a "standard score," and their output comments are formed by commonality extraction or voting to form "standard comments." This approach simulates the consensus of real-world group decision-making and is more robust than a single standard.

[0153] Specifically, the reward signal is set based on the difference between the standard review score and the simulated review score, which forms the reward function for reinforcement learning. The reward signal (R) is usually multi-dimensional and fine-grained, not just the difference in total scores. It includes: 1) Accuracy reward (R_a): encouraging the model score to approach the standard score.

Claims

1. A method for optimizing an intelligent review and scoring model, characterized in that, include: Obtain the review elements and construct a review knowledge graph based on the review elements; Extract historical bid documents and expert comments data, and correlate the historical bid documents with the expert comments data; A training dataset is generated based on the aforementioned review knowledge graph and historical tender documents; Based on the training dataset, the base model is trained to obtain the first model; By acquiring expert experience data, the first model is corrected to obtain the second model; A mock tender document is constructed, and the mock review results are obtained through the second model review. The model is corrected based on the difference between the simulated review results and the standard review results to obtain the third model.

2. The intelligent review and scoring model optimization method as described in claim 1, characterized in that, The process of obtaining review elements and constructing a review knowledge graph based on the review elements includes: Obtain the review criteria documents; The review criteria document is processed to extract review elements, resulting in first-class elements and second-class elements; Establish a corresponding vocabulary and logical relationship data for each type of element; The first and second types of elements are set as the review knowledge graph.

3. The intelligent review and scoring model optimization method as described in claim 2, characterized in that, The association of expert comment data with the historical tender documents includes: Extracting historical bid documents in a structured manner; The expert comments data are divided into various review elements according to the review knowledge graph, and the scores of each review element are obtained. Identify each score data in the historical bid documents and mark the corresponding review elements to the corresponding score data; The expert comments data are extracted and labeled with the corresponding score data.

4. The intelligent review and scoring model optimization method as described in claim 3, characterized in that, The extraction and annotation of expert comment data to corresponding score data includes: The expert comments data are structured and extracted to obtain evaluation corpus data and logical corpus data; The evaluation corpus data is compared with the vocabulary database; Identify the review elements corresponding to each evaluation corpus data; The evaluation corpus data is labeled with the corresponding score data; The logical corpus data is compared with the logical relation data; Based on logical relationship data, the score data corresponding to each logical corpus data is correlated.

5. The intelligent review and scoring model optimization method as described in claim 3, characterized in that, The division of each review element includes: Obtain the vocabulary database of expert comments and the review knowledge graph; Extract key information from expert comments data; Compare the similarity of each key piece of information with each of the aforementioned vocabulary databases; The key information is set as the review element corresponding to the highest similarity.

6. The intelligent review and scoring model optimization method as described in claim 1, characterized in that, The process of modifying the model to obtain the second model includes: Obtain signal data for expert review; Divide the tender documents into various modules; Extract the dwell time and scrolling speed of experts in each module from the signal data, and extract the review order of each module; The review importance score for each module is calculated based on the dwell time and scrolling speed. The review priority of each module is determined based on its importance and review order.

7. The intelligent review and scoring model optimization method as described in claim 6, characterized in that, The process of dividing the tender documents into various modules includes: The tender documents are first divided according to the document structure to obtain multiple initial modules; Identify the core review elements corresponding to each initial module, and set the initial modules with the same core elements as logical review modules. The logic review module is divided according to the review order during the review process.

8. The intelligent review and scoring model optimization method as described in claim 7, characterized in that, The calculation of the review importance score for each module includes: Set a time threshold to perform primary filtering on dwell time; Obtain scrolling data and analyze the scrolling behavior corresponding to each scrolling data point based on behavioral patterns. The dwell time is filtered in two stages based on the scrolling behavior to obtain the effective dwell time; Different weights are assigned to the effective dwell time and scrolling behavior to generate review importance scores for each module.

9. The intelligent review and scoring model optimization method as described in claim 8, characterized in that, Determining the review priority of each module includes: The order score for each module is set according to the review order of each module; A comprehensive score is generated based on the order score and importance score of each module; The review priorities are generated by sorting the scores based on the overall scores.

10. The intelligent review and scoring model optimization method as described in claim 1, characterized in that, The step of correcting the model based on the difference between the simulated review results and the standard review results to obtain the third model includes: Set the corresponding standard review results for the simulated tender document; The standard review results include scores and comments; A reward signal is set based on the difference between the standard review result score and the simulated review result score. A reward value is generated based on the reward signal, and the parameters of the second model are corrected based on the reward value to obtain the third model.

Images