Evaluation method and device, electronic equipment and storage medium

Abstract

The invention provides a review method and device, electronic equipment and a storage medium, and relates to the technical field of artificial intelligence, and the method comprises the steps: receiving a first review question of a to-be-reviewed file; splicing preset review parameters, the text content of the to-be-reviewed file and the first review question to obtain a second review question; inputting the second review question into a pre-trained language processing model to obtain a first score table of the to-be-reviewed file output by the language processing model; the language processing model is obtained by training a review question sample. According to the invention, by adopting an intelligent review mode, the accuracy and efficiency of review are improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an evaluation method, device, electronic device and storage medium. Background Art

[0002] At present, technical reviews for complex domain knowledge and professional requirements involve workloads such as data collection, preliminary analysis, and review preparation stages. All of the above review work is done manually, resulting in low review efficiency. Summary of the invention

[0003] The present invention provides an evaluation method, an apparatus, an electronic device and a storage medium, which are used to solve the defect of low evaluation efficiency in the prior art and improve the accuracy and efficiency of the evaluation.

[0004] The present invention provides a review method, comprising the following steps: Receive the first review questions of the documents to be reviewed; The preset review parameters, the text content of the document to be reviewed and the first review question are combined to obtain a second review question; The second review question is input into a pre-trained language processing model to obtain a first score sheet of the to-be-reviewed document output by the language processing model; the language processing model is obtained by training review question samples.

[0005] According to a review method provided by the present invention, the step of inputting the second review question into a pre-trained language processing model to obtain a first score sheet of the to-be-reviewed document output by the language processing model comprises: According to the preset scoring indicators, the second review question is disassembled to obtain multiple review points; The plurality of review points are input into the language processing model to obtain a first scoring table of the document to be reviewed output by the language processing model.

[0006] According to a review method provided by the present invention, after inputting the second review question into a pre-trained language processing model to obtain a first score sheet of the to-be-reviewed document output by the language processing model, the method further includes: According to the first scoring table, obtaining a first target scoring indicator for reviewing the document to be reviewed; In the case that the first target scoring indicator does not cover all the scoring indicators, obtaining the uncovered second target scoring indicator; Determining the third review question for the document to be reviewed according to the second target scoring indicator; The third review question is input into the language processing model to obtain a second scoring table for the document to be reviewed output by the language processing model.

[0007] According to a review method provided by the present invention, after inputting the second review question into a pre-trained language processing model to obtain a first score sheet of the to-be-reviewed document output by the language processing model, the method further includes: When the first target scoring indicator covers all the scoring indicators and the first target scoring indicator includes a higher-level planning indicator, obtaining a list of higher-level planning documents from the document to be reviewed; Determine the fourth review question of the document to be reviewed according to the list of upper-level planning documents; The fourth review question is input into the language processing model to obtain a third scoring table for the document to be reviewed output by the language processing model.

[0008] According to a review method provided by the present invention, determining the fourth review question of the to-be-reviewed document according to the upper-level planning document list includes: Acquire the upper-level planning materials from the upper-level planning file list; When the superior planning material is stored in the warehouse, the fourth review question of the document to be reviewed is determined according to the superior planning material.

[0009] According to a review method provided by the present invention, after inputting the fourth review question into the language processing model to obtain the third score sheet of the to-be-reviewed document output by the language processing model, the method further includes: According to the third scoring table, obtaining first target higher-level planning materials for reviewing the document to be reviewed; In the case that the first target higher-level planning material does not cover all the higher-level planning materials, acquiring the uncovered second target higher-level planning material; Determine the fifth review question of the document to be reviewed based on the second target higher-level planning materials; The fifth review question is input into the language processing model to obtain a fourth scoring table for the document to be reviewed output by the language processing model.

[0010] According to a review method provided by the present invention, the step of receiving a first review question for a document to be reviewed includes: Sending review prompt information of the to-be-reviewed document to the user through a human-computer interaction interface; A first review question about the to-be-reviewed document input by the user based on voice is received.

[0011] The present invention also provides an evaluation device, comprising the following modules: A receiving module, used for receiving a first review question of a document to be reviewed; A splicing module, used for splicing the preset review parameters, the text content of the document to be reviewed and the first review question to obtain a second review question; The review module is used to input the second review question into a pre-trained language processing model to obtain a first scoring table of the document to be reviewed output by the language processing model; the language processing model is obtained by training the review question samples.

[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the above-mentioned review methods is implemented.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the review method described in any one of the above is implemented.

[0014] The review method, device, electronic device and storage medium provided by the present invention receive a first review question of a document to be reviewed; splice preset review parameters, text content of the document to be reviewed and the first review question to obtain a second review question; input the second review question into a pre-trained language processing model to obtain a first score sheet of the document to be reviewed output by the language processing model; the language processing model is obtained by training the review question samples. The present invention improves the accuracy and efficiency of the review by adopting an intelligent review method. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0016] Figure 1 It is a flow chart of the review method provided by the present invention.

[0017] Figure 2 It is a flow chart of the large model-based intelligent review method provided by the present invention.

[0018] Figure 3 It is a structural schematic diagram of the review device provided by the present invention.

[0019] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] In related technologies, large language processing models (LLM), such as GPT (Generative Pre-Trained, deep learning model), BERT (Bidirectional Encoder Representations from Transformers, language representation model) and various variants of models have achieved remarkable results in the field of natural language processing (NLP). The high performance of these models in language understanding and text generation makes them an ideal cornerstone for building intelligent interactive systems. However, simple text processing is not enough to meet the requirements of the comprehensive development of artificial intelligence, especially for intelligent agents used in actual scenarios, which require more complex functions, such as memory ability, environmental adaptability, tool use ability and social interaction ability.

[0022] Traditional AI agents mainly rely on rule engines, knowledge bases, and limited learning mechanisms, while LLM provides a new mechanism to enrich the functions of agents. LLM can generate complex word order structures, understand semantics, perform reasoning, and even create a general framework that can cope with different types of tasks. However, current LLMs still have some limitations, such as lack of long-term memory, inability to directly interact with other tools in the environment, and difficulty in processing multimodal input and output, which limit their performance in specific application scenarios.

[0023] In the field of embodied intelligence, agents not only need to process information, but also interact with the physical environment, perform actions and receive sensor input, which requires agents to be able to plan and evaluate their own behavior, and require multimodal models to process non-textual information. Therefore, it is urgent to develop an agent system that can combine the characteristics of LLM and embodied intelligence, which should be able to learn and act autonomously in more complex, dynamic and diverse environments.

[0024] On the other hand, technical reviews of complex domain knowledge and professional requirements are conducted manually, which leads to low review efficiency. The experience, knowledge background and subjective judgment of different reviewers may lead to inconsistent review results.

[0025] Based on the above problems, the present invention proposes a review method, specifically a review method implemented by an autonomous intelligent agent system based on a large language model, which integrates internal memory mechanism, external tool call and social interaction capabilities, as well as in-depth application of key technologies of embodied intelligence. Through these technologies, the intelligent agent can add functions such as real-time interaction with the environment, information integration, multi-tasking and self-optimization on the basis of processing natural language, which greatly expands the applicability and effectiveness of LLM in the real world.

[0026] Combine the following Figure 1-Figure 4 The review method, device, electronic device and storage medium of the present invention are described.

[0027] Figure 1 It is a flow chart of the review method provided by the present invention, such as Figure 1 As shown, the method includes the following: Step 101: Receive a first review question for a document to be reviewed.

[0028] In the embodiment of the present invention, the documents to be reviewed can be understood as documents related to planning schemes and project technical reviews, such as special planning documents, top-level design (including project list) documents, etc. The documents to be reviewed need to comply with the format and size requirements specified by the system, such as file type (such as PDF, Word document), document structure (such as title, paragraph), file size limit, etc., to ensure that the system can effectively process and parse the document content.

[0029] If the files uploaded by users meet the requirements, the system will store them in the internal database or file system for subsequent processing and access. At the same time, the system will call the document parsing plug-in, which automatically extracts key text information in the document, such as text content, title, paragraph, punctuation, etc. This process is to convert the document content into structured data that the system can understand and process. For example, the document parsing plug-in performs initial automated processing to convert the document content into a format that can be processed internally by the system, including storing the text content as a text string or other data structure for subsequent human-computer interaction and large model processing.

[0030] In one embodiment, the system sends review prompt information of the document to be reviewed to the user through the human-computer interaction interface, and then receives the review questions of the document to be reviewed based on the user's voice input. It can be understood that the system allows users to submit different types of project review requests through the human-computer interaction interface. At the same time, the system predefines question formats or templates for different types of reviews to guide users to conduct detailed evaluation and analysis. Users can ask scoring questions to the system according to the preset question format.

[0031] For example, after a user uploads a document to be reviewed in the system, the system will display review prompt information related to the document to be reviewed to the user through the human-computer interaction interface. The review prompt information may include specific questions for the document, review criteria, required analysis content, etc. This method can ensure that the user clearly understands the issues that need to be paid attention to and answered, so as to conduct effective review work.

[0032] Assuming that a user uploads a software design document, the review prompt information displayed by the system through the human-computer interaction interface may include: Question 1: Do you think the performance optimization solution proposed in the document is feasible? Question 2: Please analyze whether the database architecture in the document can support the expected load and data requirements. Question 3: Score the uploaded document. These review prompt information help users clarify the key points that need to be paid attention to, so that they can conduct targeted reviews.

[0033] The system supports voice input, and users can submit review questions or requirements about the documents to be reviewed to the system through speech. This method provides a more flexible and natural interaction method, which is suitable for scenarios that require quick questions and answers.

[0034] Step 102: Combine the preset review parameters, the text content of the document to be reviewed, and the first review question to obtain a second review question.

[0035] In the embodiment of the present invention, the preset review parameters include review rules and review indicators. Among them, the scoring rules are a set of standards and rules used to measure the quality of documents or specific requirements. These rules may vary according to the purpose and field of the review, including accuracy, completeness, consistency, etc. The scoring indicators are specific indicators or quantitative standards used to measure the scoring rules, including error rate and information volume, etc.

[0036] The system uses the prompt splicing plug-in to splice the preset review parameters, the text content of the document to be reviewed, and the first review question to obtain the second review question. Among them, the prompt splicing plug-in can be understood as a technical tool or program that is used to combine the user's question with the text content parsed by the system, relevant scoring rules and indicators according to the preset template or rules to form a detailed question. For example, the system first parses the document or data provided by the user, and then generates a specific and detailed question, namely the second review temperature, based on the preset review template or rules and combined with the relevant scoring indicators.

[0037] Step 103: input the second review question into a pre-trained language processing model to obtain a first scoring table for the document to be reviewed output by the language processing model.

[0038] According to the preset scoring indicators, the second review question is disassembled to obtain multiple review points, and then the multiple review points are input into the language processing model to obtain the first score sheet of the document to be reviewed output by the language processing model. Among them, the preset scoring indicators may include three-level indicators of the scoring standard, that is, a hierarchical evaluation standard or indicator system. For example, the analysis of the indicators at each level is as follows: First-level indicators: The highest level of evaluation criteria, which are the major categories or main aspects of the overall evaluation, for example, the score of the entire document.

[0039] Secondary indicators: Based on the primary indicators, they are further subdivided into specific evaluation requirements or indicators. For example, the score of the first part of the review document.

[0040] Level 3 indicators: Based on the level 2 indicators, they are further refined into specific scoring points or inspection items. For example, the score of the third section of the first part of the review document.

[0041] According to the set scoring criteria, the system will break down the entire review task or the content of the document to be evaluated according to the pre-defined scoring indicators. The breakdown process will generate multiple specific review points or review requirements. Each review point is a detailed review requirement for a specific aspect of the document, such as scoring the content of a section in the document. Each review point after the breakdown is passed as input to the pre-trained language processing model. After the model processes each review point, it will output the corresponding scoring table. Among them, the scoring table includes the scoring value and scoring description: Rating value: A specific score or grade that indicates the evaluation result of a document or solution.

[0042] Scoring instructions: Detailed description of the basis, reasons, scoring indicators and improvement suggestions for scoring. These instructions can be generated based on the scoring rules and indicators preset by the system to help users understand the logic behind the scoring results.

[0043] In the embodiment of the present invention, the language processing model refers to a large model based on natural language processing or deep learning, such as the GPT series, which is used to understand and process input text data. Among them, the language processing model is trained on the review question samples. For example, the language processing model is trained based on the following method: 1) Data collection and preparation: First, a large number of review question samples need to be collected. These samples can come from real review scenarios or constructed simulation scenarios, covering review requirements of different types and complexities.

[0044] 2) Data preprocessing: The collected review question samples are preprocessed, including word segmentation, stop word removal, tokenization, etc., so that the model can understand and process these text data.

[0045] 3) Model selection and configuration: Choose a large model suitable for the task, such as a language model based on the Transformer architecture, such as the GPT series. Configure the model's parameters, such as the number of layers and hidden units, to adapt to the complexity and requirements of the review task.

[0046] 4) Training setting: Design the setting of the training task, using unsupervised learning methods, such as language modeling tasks. During the training process, the model learns the structure and semantics of the text by predicting the next word or mask word.

[0047] 5) Model training: Use the preprocessed review question samples as training data to train the model. During the training process, the model learns to generate corresponding review results based on the input review task description and related document content.

[0048] 6) Optimization and evaluation: Periodic optimization and evaluation are required during the training process to ensure the accuracy and adaptability of the model when generating review results. For example, the optimization involved includes adjusting the learning rate, increasing the diversity of training data, adjusting the model architecture, etc. to improve the performance of the model.

[0049] 7) Verification and adjustment: After training, the model is verified and adjusted to ensure that its performance on the review task meets the expected requirements. For example, the quality and consistency of the model output can be verified through manual or automated evaluation.

[0050] The review method provided by the embodiment of the present invention receives a first review question of a document to be reviewed; splices the preset review parameters, the text content of the document to be reviewed and the first review question to obtain a second review question; inputs the second review question into a pre-trained language processing model to obtain a first score sheet of the document to be reviewed output by the language processing model; the language processing model is obtained by training the review question samples. The present invention improves the accuracy and efficiency of the review by adopting an intelligent review method.

[0051] Based on the above embodiment, after inputting the second review question into a pre-trained language processing model to obtain a first score sheet of the to-be-reviewed document output by the language processing model, the method further includes: Step 110: Obtain a first target scoring index for evaluating the document to be evaluated according to the first scoring table; Step 111: when the first target scoring indicator does not cover all the scoring indicators, obtaining the uncovered second target scoring indicator; Step 112: Determine the third review question for the document to be reviewed according to the second target scoring indicator; Step 113: input the third review question into the language processing model to obtain a second scoring table for the document to be reviewed output by the language processing model.

[0052] After the language processing model outputs the first scoring table, the scoring value and scoring description are obtained from the first scoring table, and then the first target scoring indicator for reviewing the document to be reviewed is obtained according to the scoring description. By comparing the first target scoring indicator with all preset scoring indicators, it is determined whether all preset scoring indicators are covered in the review. If the first target scoring indicator covers all preset scoring indicators, it means that the model's review of the document to be reviewed is considered comprehensive because it has covered all preset scoring criteria. If the first target scoring indicator does not cover all preset scoring indicators, it means that the model's review of the document to be reviewed is not enough to cover all scoring indicators, and further evaluation and adjustment are required to ensure that all important evaluation criteria are taken into account. At this time, it is necessary to obtain the uncovered second target scoring indicator, and then determine the third review question of the document to be reviewed based on the second target scoring indicator, and finally input the third review question into the language processing model to obtain the second scoring table of the document to be reviewed output by the language processing model.

[0053] The embodiment of the present invention can ensure that the review process is comprehensive and accurate by selecting the first target scoring indicator and confirming whether it covers all preset scoring indicators. This helps to avoid omissions or deviations in the review process and improve the reliability and effectiveness of the evaluation.

[0054] Based on the above embodiment, after inputting the second review question into a pre-trained language processing model to obtain a first score sheet of the to-be-reviewed document output by the language processing model, the method further includes: Step 120: When the first target scoring indicator covers all the scoring indicators and the first target scoring indicator includes a higher-level planning indicator, a list of higher-level planning documents is obtained from the document to be reviewed; Step 121: Determine the fourth review question of the document to be reviewed according to the upper-level planning document list; Step 122: input the fourth review question into the language processing model to obtain a third scoring table for the document to be reviewed output by the language processing model.

[0055] It is understandable that the superordinate plan is a high-level overview of the project plan. It does not involve specific technical details, but focuses on the overall goals, major stages, timetables, and resource requirements of the project. The goal of the superordinate plan is to lay the foundation for the detailed plan of the project and ensure the rationality of the overall direction and resource allocation. The superordinate planning document is a formal document or document that records the content of the superordinate plan. It can include information such as the overall goals of the project, the main stages or stage divisions, key time nodes, rough estimates of resource requirements, and risk management strategies. These documents provide a common understanding and reference framework for all parties in the project, which helps to coordinate and communicate the overall plan of the project. Superordinate planning materials refer to all relevant information and resources used to develop and support superordinate planning. These materials can include market research reports, preliminary demand analysis, preliminary technical assessments, resource budget estimates, project organizational structure design, etc. These materials provide the necessary data and background information for the development of superordinate planning to ensure the integrity and feasibility of the plan.

[0056] When the first target scoring indicator covers all scoring indicators, determine whether the first target scoring indicator includes the upper-level planning indicator. If it does, it is necessary to obtain the upper-level planning file list from the files to be reviewed, and then determine the fourth review question of the files to be reviewed based on the upper-level planning file list. Specifically, obtain the upper-level planning materials from the upper-level planning file list; when the upper-level planning materials are stored in the library, determine the fourth review question of the files to be reviewed based on the upper-level planning materials. Finally, input the fourth review question into the language processing model to obtain the third score sheet of the files to be reviewed output by the language processing model.

[0057] In one embodiment, after the fourth review question is input into the language processing model and the third score sheet of the document to be reviewed output by the language processing model is obtained, the first target superordinate planning material for reviewing the document to be reviewed is obtained according to the third score sheet; when the first target superordinate planning material does not cover all the superordinate planning materials, the uncovered second target superordinate planning material is obtained; according to the second target superordinate planning material, the fifth review question of the document to be reviewed is determined; the fifth review question is input into the language processing model to obtain the fourth score sheet of the document to be reviewed output by the language processing model. For example, if the first target superordinate planning material only covers part of the review requirements or standards, it is necessary to find the second target superordinate planning material to ensure that the review is comprehensive and accurate. By obtaining the second target superordinate planning material, the key issues of the review can be re-examined and new review questions can be generated, wherein the new questions can explore the degree of compliance of the document to be reviewed in the uncovered areas in a more specific and in-depth manner.

[0058] The embodiment of the present invention ensures that the review process is systematic and structured through a clearly defined review process and indicator system, which helps to reduce confusion and subjectivity in the review process and improve the objectivity and consistency of the review. In addition, by introducing higher-level planning indicators and multi-stage review questions, it ensures that the review covers multiple levels and dimensions, which helps to capture the compliance of documents in different aspects, thereby more comprehensively evaluating their quality and compliance.

[0059] In order to further analyze and illustrate the review method proposed in the present invention, refer to the following examples.

[0060] The main purpose of the embodiment of the present invention is to provide an intelligent review method based on a large model, aiming to solve the following problems existing in existing planning schemes and project technical reviews: (1) Inconsistency: The traditional technical review process relies on manual evaluation. The experience, knowledge background and subjective judgment of different reviewers may lead to inconsistency in the review results. The present invention introduces an intelligent large model system to unify the review standards and processes, thereby ensuring the fairness and consistency of the review results.

[0061] (2) Low efficiency: Due to the complexity of planning projects and the huge volume of related materials, the traditional manual review process is time-consuming and inefficient. The intelligent review method proposed in this invention can automatically process and analyze a large amount of documents and data, quickly give evaluation results, and greatly improve the efficiency of the review process.

[0062] (3) Difficulty in updating knowledge: With the development of technology and changes in policies, traditional review methods are difficult to quickly adapt to new situations. The intelligent review system of the present invention has the ability to continuously learn and self-update, and can absorb the latest planning standards and technical information in real time to maintain the cutting-edge and adaptability of the review content.

[0063] (4) Improving the scientific nature of decision support: By utilizing artificial intelligence technology, the system can accurately analyze and understand complex planning content and standards, provide more scientific and reasonable decision support, and help decision makers quickly grasp core information and formulate effective strategies.

[0064] By achieving the above objectives, the present invention provides an efficient, scientific and fair technical review tool for the planning and implementation of planning schemes, which helps to promote the healthy development of urban intelligent construction.

[0065] In order to improve the efficiency and consistency of planning schemes and project technical reviews, an embodiment of the present invention provides an intelligent review method based on a large model. The method is committed to solving the problem of inconsistent review scales caused by human factors and the low review efficiency caused by the complexity of domain knowledge. The method mainly includes the following contents: Document upload and automatic parsing: First, users need to upload the special planning and top-level design (including project list) files to be technically reviewed. These files need to meet the specified format and size requirements before being stored in the system. The system calls the document parsing plug-in to automatically extract the key text information in the file and perform initial automatic document parsing.

[0066] Human-computer interaction and intelligent questioning: Through the human-computer interaction interface, users can put forward specific review requirements to the system through the preset question format (promptbuild), such as requesting to score the uploaded document. The system uses the prompt splicing plug-in to closely combine the user's questions with the content parsed from the document, the relevant scoring rules and indicators, and form a detailed question prompt.

[0067] Intelligent processing of planning scheme materials: If the review content involves planning schemes, the system first checks whether the required planning scheme materials have been stored in the warehouse. For materials that have not been stored in the warehouse, the system will prompt that they need to be manually supplemented. For materials that have been stored in the warehouse, the system will call the planning scheme prompt plug-in, combine the specific content of the planning scheme, and rebuild the prompt for detailed analysis.

[0068] Automatic scoring and result generation: The optimized and spliced ​​prompts are submitted to the big model for processing. The big model analyzes the questions and content in the prompts based on the embedded knowledge base and learning algorithm, and outputs accurate review results. Then, the system automatically generates a detailed technical review score sheet based on the established scoring rule template through the scoring result combination plug-in.

[0069] In one embodiment, reference Figure 2 ,The detailed steps of the intelligent review method based on the big model are as follows: Step 1: Upload the review documents; Step 2: Call the document parsing plug-in to parse the text data in the document; Step 3: prompt build to build the basic question template; Step 4: Call the prompt disassembly plug-in to disassemble the prompt item by item according to the three-level indicators of the scoring standard; Step 5: Ask LLM questions one by one prompt; Step 6: Determine whether all scoring indicators are covered according to the scoring table output by LLM. If yes, go to step 7; if not, go to step 5; Step 7: Does the indicator involve higher-level planning? If so, proceed to step 8. If not, the big model will give a scoring table, including the score and scoring explanation. Step 8: Call the upper planning document extraction plug-in; Step 9: Output the list of upper-level planning files in the document; Step 10: Check whether the upper-level planning materials have been put into storage; if so, proceed to step 11; if not, it will be prompted that they have not been put into storage, and the upper-level planning files will be put into storage; Step 11, call the upper planning prompt splicing plug-in to splice prompt; Step 12: Plan the materials one by one and splice prompts for LLM questions; Step 13: Check whether all higher-level planning materials are covered. If not, proceed to step 11. If yes, the big model will give a scoring table, including the score and scoring explanation.

[0070] The intelligent review method based on a large model provided by the embodiment of the present invention can significantly reduce human bias in technical review and improve the accuracy and efficiency of the review work. On the other hand, the automated process and intelligent analysis reduce the reliance on professional background knowledge, making the review process faster and more scientific. Compared with the prior art, the embodiment of the present invention has the following advantages: (1) Improve the consistency and accuracy of reviews: Through the intelligent review method of the big model, the impact of human factors in the review process is reduced, ensuring the consistency of the review scale and standards for special plans, top-level designs, and project lists. At the same time, the powerful data processing and analysis capabilities of the big model are used to improve the accuracy of the review. (2) Improve review efficiency: Automated document parsing and intelligent questioning mechanisms greatly reduce the time for document reading, information extraction, and question construction in the traditional review process, greatly improving the efficiency of the review work. In particular, when dealing with a large number of documents and complex information, this solution can significantly shorten the review cycle. (3) Reduce the burden on experts: By automating and intelligently processing most of the pre-processing work in the review, the workload of experts in data collection, preliminary analysis, and review preparation stages is reduced. Experts can focus more on in-depth analysis and decision-making of the review content, improving the quality of the review. (4) Intelligent processing of complex domain knowledge: For technical reviews involving complex domain knowledge and professional requirements, the big model can quickly and accurately identify key information and review indicators, provide intelligent evaluation based on deep learning, and reduce the difficulty and time cost of traditional methods in collecting and understanding domain expertise. (5) Strengthen decision support: The intelligent review method can not only output a score sheet, but also generate a list of questions, a technical review report, and key points and materials of meeting minutes according to needs, providing more comprehensive and accurate information support for review experts and decision makers, and strengthening the scientificity and rationality of the decision-making process. (6) Strengthen knowledge accumulation and sharing: The database of planning scheme materials, review indicators and other contents involved in the system has been established, providing a rich knowledge base for planning schemes and technical review work. This not only facilitates the sharing and accumulation of experience, but also continuously optimizes and updates the review standards and processes, and promotes the continuous improvement and development of planning scheme review work.

[0071] The review device provided by the present invention is described below. The review device described below and the review method described above can be referenced to each other.

[0072] refer to Figure 3 The review device provided by the present invention includes a receiving module 301, a splicing module 302 and a review module 303.

[0073] A receiving module 301 is used to receive a first review question of a document to be reviewed; A splicing module 302 is used to splice the preset review parameters, the text content of the document to be reviewed and the first review question to obtain a second review question; The review module 303 is used to input the second review question into a pre-trained language processing model to obtain a first scoring table of the document to be reviewed output by the language processing model; the language processing model is obtained by training the review question samples.

[0074] In one embodiment, the review module 303 is specifically used to: decompose the second review question according to preset scoring indicators to obtain multiple review points; input the multiple review points into the language processing model to obtain a first scoring table for the document to be reviewed output by the language processing model.

[0075] In one embodiment, the review module 303 is further used to: obtain a first target scoring indicator for reviewing the document to be reviewed based on the first scoring table; if the first target scoring indicator does not cover all the scoring indicators, obtain an uncovered second target scoring indicator; determine a third review question for the document to be reviewed based on the second target scoring indicator; input the third review question into the language processing model to obtain a second scoring table for the document to be reviewed output by the language processing model.

[0076] In one embodiment, the review module 303 is also used to: when the first target scoring indicator covers all the scoring indicators and the first target scoring indicator includes the upper-level planning indicator, obtain a list of upper-level planning files from the file to be reviewed; determine the fourth review question of the file to be reviewed based on the list of upper-level planning files; input the fourth review question into the language processing model to obtain a third scoring table of the file to be reviewed output by the language processing model.

[0077] In one embodiment, the review module 303 is further used to: obtain the upper-level planning material from the upper-level planning file list; and when the upper-level planning material is stored in the library, determine the fourth review question of the document to be reviewed according to the upper-level planning material.

[0078] In one embodiment, the review module 303 is also used to: obtain the first target superordinate planning materials for reviewing the document to be reviewed according to the third scoring table; if the first target superordinate planning materials do not cover all the superordinate planning materials, obtain the uncovered second target superordinate planning materials; determine the fifth review question for the document to be reviewed according to the second target superordinate planning materials; input the fifth review question into the language processing model to obtain the fourth scoring table for the document to be reviewed output by the language processing model.

[0079] In one embodiment, the acquisition module 301 is specifically used to: send review prompt information of the to-be-reviewed document to the user through a human-computer interaction interface; and receive a first review question of the to-be-reviewed document input by the user based on voice.

[0080] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the review method, which includes: receiving a first review question of a document to be reviewed; splicing the preset review parameters, the text content of the document to be reviewed and the first review question to obtain a second review question; inputting the second review question into a pre-trained language processing model to obtain a first score sheet of the document to be reviewed output by the language processing model; the language processing model is obtained by training the review question sample.

[0081] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0082] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the review method provided by the above-mentioned methods, which includes: receiving a first review question of a document to be reviewed; splicing preset review parameters, the text content of the document to be reviewed, and the first review question to obtain a second review question; inputting the second review question into a pre-trained language processing model to obtain a first score sheet for the document to be reviewed output by the language processing model; the language processing model is obtained by training review question samples.

[0083] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the review method provided by the above-mentioned methods, the method comprising: receiving a first review question of a document to be reviewed; concatenating preset review parameters, the text content of the document to be reviewed, and the first review question to obtain a second review question; inputting the second review question into a pre-trained language processing model to obtain a first score sheet of the document to be reviewed output by the language processing model; the language processing model is obtained by training review question samples.

[0084] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0085] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A review method, characterized in that: include: Receive the first review questions of the documents to be reviewed; The preset review parameters, the text content of the document to be reviewed and the first review question are combined to obtain a second review question; Inputting the second review question into a pre-trained language processing model to obtain a first scoring table for the document to be reviewed output by the language processing model; The language processing model is obtained by training on review question samples.

2. The review method according to claim 1, characterized in that: The step of inputting the second review question into a pre-trained language processing model to obtain a first scoring table of the to-be-reviewed document output by the language processing model comprises: According to the preset scoring indicators, the second review question is disassembled to obtain multiple review points; The plurality of review points are input into the language processing model to obtain a first scoring table of the document to be reviewed output by the language processing model.

3. The review method according to claim 2, characterized in that: After inputting the second review question into the pre-trained language processing model to obtain the first score sheet of the to-be-reviewed document output by the language processing model, the method further includes: According to the first scoring table, obtaining a first target scoring indicator for reviewing the document to be reviewed; In the case that the first target scoring indicator does not cover all the scoring indicators, obtaining the uncovered second target scoring indicator; Determining the third review question for the document to be reviewed according to the second target scoring indicator; The third review question is input into the language processing model to obtain a second scoring table for the document to be reviewed output by the language processing model.

4. The review method according to claim 3, characterized in that: After inputting the second review question into the pre-trained language processing model to obtain the first score sheet of the to-be-reviewed document output by the language processing model, the method further includes: When the first target scoring indicator covers all the scoring indicators and the first target scoring indicator includes a higher-level planning indicator, obtaining a list of higher-level planning documents from the document to be reviewed; Determine the fourth review question of the document to be reviewed according to the list of upper-level planning documents; The fourth review question is input into the language processing model to obtain a third scoring table for the document to be reviewed output by the language processing model.

5. The review method according to claim 4, characterized in that: Determining the fourth review question of the to-be-reviewed document according to the upper-level planning document list includes: Acquire the upper-level planning materials from the upper-level planning file list; When the superior planning material is stored in the warehouse, the fourth review question of the document to be reviewed is determined according to the superior planning material.

6. The review method according to claim 5, characterized in that: After inputting the fourth review question into the language processing model to obtain the third score sheet of the to-be-reviewed document output by the language processing model, the method further includes: According to the third scoring table, obtaining first target higher-level planning materials for reviewing the document to be reviewed; In the case that the first target higher-level planning material does not cover all the higher-level planning materials, acquiring the uncovered second target higher-level planning material; Determine the fifth review question of the document to be reviewed based on the second target higher-level planning materials; The fifth review question is input into the language processing model to obtain a fourth scoring table for the document to be reviewed output by the language processing model.

7. The review method according to claim 1, characterized in that: The first review question of receiving the document to be reviewed includes: Sending review prompt information of the to-be-reviewed document to the user through a human-computer interaction interface; A first review question about the to-be-reviewed document input by the user based on voice is received.

8. An evaluation device, characterized in that: include: A receiving module, used for receiving a first review question of a document to be reviewed; A splicing module, used for splicing the preset review parameters, the text content of the document to be reviewed and the first review question to obtain a second review question; A review module, configured to input the second review question into a pre-trained language processing model to obtain a first scoring table for the document to be reviewed output by the language processing model; The language processing model is obtained by training on review question samples.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the review method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the review method according to any one of claims 1 to 7 is implemented.

Images