Software achievement acceptance method based on large language model

Abstract

The invention discloses a software achievement acceptance method based on a large language model. The method comprises the steps that a to-be-accepted file is obtained, and the current acceptance type is judged; when the acceptance type is a document, analyzing the document file into a structured text; utilizing the trained large language model to classify and recognize the structured text to obtain attribution classification; the large language model obtains corresponding acceptance requirements from an acceptance knowledge base according to the attribution classification; the multi-modal large language model performs item-by-item acceptance according to acceptance requirements to generate a document acceptance result; when the acceptance type is a code, identifying a development language of the code by using the trained large language model, and obtaining an acceptance review rule corresponding to the development language from an acceptance knowledge base; the multi-modal large language model carries out acceptance according to the acceptance review rule and generates a software code acceptance result; and summarizing all acceptance results and forming an acceptance report. According to the method, the flexibility of the LLM and the accuracy of the acceptance knowledge base are combined, the manual acceptance cost is remarkably reduced, and the review efficiency and consistency are improved.

Description

Technical Field

[0001] This invention relates to the field of automated software acceptance review, and in particular to a method for software deliverable acceptance based on a large language model. Background Technology

[0002] Existing technologies primarily use NLP-based semantic analysis models for document acceptance. The specific process is as follows: Obtain the document to be processed; extract the target sentence from the document; input the target sentence into a pre-defined NLP-based semantic analysis model, and use the model to perform semantic recognition on the target sentence to obtain semantic result keywords; obtain the expected verification results of the verification items corresponding to the verification item keywords from a pre-defined set of verification items; input the verification result keywords and expected verification results into the test function corresponding to the verification item, and use the test function to check whether the verification result keywords match the expected verification results to obtain the verification results of the verification items corresponding to the verification item keywords; determine the acceptance result of the document to be processed based on the verification results of each verification item.

[0003] Existing technologies, when conducting document acceptance, limit the acceptance content to textual content and do not support the acceptance of charts, formulas, or other content within the document; similarly, they cannot accept software code. Summary of the Invention

[0004] The purpose of this invention is to provide a software deliverable acceptance method based on a large language model. By combining the flexibility of LLM with the accuracy of the acceptance knowledge base, it can significantly reduce the cost of manual acceptance and improve the efficiency and consistency of the review.

[0005] The technical solution adopted in this invention is: A software deliverable acceptance method based on a large language model includes the following steps: Step 1: Obtain the file to be accepted and determine the current acceptance type; if the acceptance type is document, proceed to Step 2; if the acceptance type is code, proceed to Step 6. Step 2, (using the document extractor) parses the document file into structured text; Step 3: Use the trained existing mature Large Language Model (LLM) to classify and identify the structured text to obtain the document's classification (such as project initiation meeting minutes, project plan, etc.).

[0006] Step 4: The Large Language Model (LLM) retrieves the corresponding acceptance requirements or standards from the acceptance knowledge base based on the document's classification. Step 5: The Multimodal Large Language Model (MLLM) performs item-by-item acceptance of all documents to be accepted according to the acceptance requirements, and generates document acceptance results before proceeding to step 9. Step 6: Use a trained, existing mature large language model (LLM) to identify the development language used in the code to be accepted; Step 7: The Large Language Model (LLM) retrieves the corresponding acceptance review rules for the development language from the acceptance knowledge base based on the development language. Step 8: The Multimodal Large Language Model (MLLM) performs acceptance review on the software code according to the acceptance review rules, and generates the software code acceptance results before proceeding to Step 9. Step 9: Summarize all acceptance results and generate an acceptance report.

[0007] Specifically, Large Language Model (LLM) is a deep learning model trained on massive amounts of text data and possesses powerful natural language processing capabilities; Multimodal Large Language Model (MLLM), on the other hand, supports the processing of various types of content input, such as images, speech, and files, based on the Large Language Model.

[0008] Furthermore, the document classification in step 3 includes project initiation meeting minutes and project plan.

[0009] Furthermore, in step 5, the document acceptance results will indicate the issues that failed the acceptance process.

[0010] Furthermore, the development language or programming language in step 6 includes Java and Python.

[0011] Furthermore, the acceptance review rules in step 7 include code standards and security requirements.

[0012] Furthermore, the code acceptance results in step 8 include compliance information and a list of defects.

[0013] Furthermore, the acceptance report in step 9 includes detailed conclusions of each acceptance, a summary of problems, and suggestions for improvement.

[0014] Furthermore, the final acceptance report in step 9 is available for users to view and archive.

[0015] Furthermore, by using the large model to preprocess the records of acceptance requirements and acceptance review rules in the acceptance knowledge base, the output structured content that facilitates knowledge base indexing and retrieval is obtained, and then imported into the acceptance knowledge base to improve the quality of the knowledge base and enhance the ability to accept results.

[0016] The steps for documenting the acceptance requirements and review rules for large language model preprocessing are as follows: S1. Set prompt words for the large language model, assign the role of knowledge base document preprocessor, and set the output acceptance rule document structure format; each acceptance rule is on one line, and each rule includes the acceptance category, acceptance object, and acceptance requirements; at the same time, provide suggestions for the knowledge base segment length and separator for the corresponding acceptance rule document; S2. Upload the usual acceptance rules document to the large language model for conversion processing to obtain the processed document; 33. Upload the processed acceptance rules document to the knowledge base and build a retrieval index.

[0017] This invention, employing the above technical solution, possesses the following technical advantages: 1. Enhanced document acceptance capability: By integrating a multimodal large language model (MLLM) with an acceptance knowledge base, it enables document acceptance processing for multiple elements, including text, charts, and formulas. 2. Integrated resource acceptance: It innovatively constructs a unified multi-type resource acceptance platform, fundamentally solving the management challenge of separately reviewing heterogeneous resources such as documents and code. Through modular architecture design, the system can be flexibly expanded to support intelligent acceptance of various resources, achieving standardized management across teams and domains. Attached Figure Description

[0018] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments; Figure 1 This is a flowchart illustrating a software deliverable acceptance method based on a large language model according to the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0020] like Figure 1 As shown, this invention discloses a software deliverable acceptance method based on a large language model, which includes the following steps: Step 1: Obtain the file to be accepted and determine the current acceptance type; if the acceptance type is document, proceed to Step 2; if the acceptance type is code, proceed to Step 6. Step 2, (using the document extractor) parses the document file into structured text; Step 3: Use the trained existing mature Large Language Model (LLM) to classify and identify the structured text to obtain the document's classification (such as project initiation meeting minutes, project plan, etc.).

[0021] Step 4: The Large Language Model (LLM) retrieves the corresponding acceptance requirements or standards from the acceptance knowledge base based on the document's classification. Step 5: The Multimodal Large Language Model (MLLM) performs item-by-item acceptance of all documents to be accepted according to the acceptance requirements, and generates document acceptance results before proceeding to step 9. Step 6: Use a trained, existing mature large language model (LLM) to identify the development language used in the code to be accepted; Step 7: The Large Language Model (LLM) retrieves the corresponding acceptance review rules for the development language from the acceptance knowledge base based on the development language. Step 8: The Multimodal Large Language Model (MLLM) performs acceptance review on the software code according to the acceptance review rules, and generates the software code acceptance results before proceeding to Step 9. Step 9: Summarize all acceptance results and generate an acceptance report.

[0022] Specifically, Large Language Model (LLM) is a deep learning model trained on massive amounts of text data and possesses powerful natural language processing capabilities; Multimodal Large Language Model (MLLM), on the other hand, supports the processing of various types of content input, such as images, speech, and files, based on the Large Language Model.

[0023] Furthermore, the document classification in step 3 includes project initiation meeting minutes and project plan.

[0024] Furthermore, in step 5, the document acceptance results will indicate the issues that failed the acceptance process.

[0025] Furthermore, the development language or programming language in step 6 includes Java and Python.

[0026] Furthermore, the acceptance review rules in step 7 include code standards and security requirements.

[0027] Furthermore, the code acceptance results in step 8 include compliance information and a list of defects.

[0028] Furthermore, the acceptance report in step 9 includes detailed conclusions of each acceptance, a summary of problems, and suggestions for improvement.

[0029] Furthermore, the final acceptance report in step 9 is available for users to view and archive.

[0030] Furthermore, by using the large model to preprocess the records of acceptance requirements and acceptance review rules in the acceptance knowledge base, the output structured content that facilitates knowledge base indexing and retrieval is obtained, and then imported into the acceptance knowledge base to improve the quality of the knowledge base and enhance the ability to accept results.

[0031] The steps for documenting the acceptance requirements and review rules for large language model preprocessing are as follows: S1. Set prompt words for the large language model, assign the role of knowledge base document preprocessor, and set the output acceptance rule document structure format; each acceptance rule is on one line, and each rule includes the acceptance category, acceptance object, and acceptance requirements; at the same time, provide suggestions for the knowledge base segment length and separator for the corresponding acceptance rule document; S2. Upload the usual acceptance rules document to the large language model for conversion processing to obtain the processed document; 33. Upload the processed acceptance rules document to the knowledge base and build a retrieval index.

[0032] This invention first uses LLM for simple text processing and classification to obtain acceptance rules. Then, when it comes to acceptance based on the rules, MLLM is used for acceptance processing and outputs the results. The combination of the two models saves resources and improves the efficiency of acceptance.

[0033] In addition, as another implementation method, for situations where the acceptance knowledge base requires manual optimization of segmentation parameters, segmentation identifiers, maximum segment length, segmentation overlap length, etc. by uploading documents, which requires repeated debugging and is time-consuming, this invention can preprocess the acceptance rule documents by a large model, output structural content that is conducive to better indexing and retrieval of the knowledge base, and then import it into the knowledge base to improve the quality of the knowledge base and enhance the acceptance capability of the results.

[0034] The steps for the large language model preprocessing acceptance rules document are as follows: 1. Set prompt words for the large language model, assign the role of knowledge base document preprocessor, and set the output acceptance rule document structure format. Each acceptance rule should be on one line, and each rule should have the acceptance category, acceptance object, and acceptance requirements. At the same time, provide suggestions for the knowledge base segment length and separator for this acceptance rule document. 2. Upload the standard acceptance rules document to the large language model for conversion and processing to obtain the processed document; 3. Upload the processed acceptance rules document to the knowledge base and build a retrieval index; This invention uses a multimodal large language model to realize the acceptance of software deliverables documents and code. The multimodal large language model supports parsing text, images, tables, and formulas to achieve multi-dimensional acceptance. In addition, it can also analyze and reason about the code to achieve acceptance at the software code level, thus solving the problem of single acceptance capabilities in the background technology.

[0035] This invention, employing the above technical solution, possesses the following technical advantages: 1. Enhanced document acceptance capability: By integrating a multimodal large language model (MLLM) with an acceptance knowledge base, it enables document acceptance processing for multiple elements, including text, charts, and formulas. 2. Integrated resource acceptance: It innovatively constructs a unified multi-type resource acceptance platform, fundamentally solving the management challenge of separately reviewing heterogeneous resources such as documents and code. Through modular architecture design, the system can be flexibly expanded to support intelligent acceptance of various resources, achieving standardized management across teams and domains.

[0036] Obviously, the described embodiments are only a part of the embodiments of this application, not all of them. Without conflict, the embodiments and features in the embodiments of this application can be combined with each other. The components of the embodiments of this application described and illustrated herein can generally be arranged and designed in various different configurations. Therefore, the detailed description of the embodiments of this application is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

Claims

1. A method for accepting software deliverables based on a large language model, characterized in that: It includes the following steps: Step 1: Obtain the file to be accepted and determine the current acceptance type; if the acceptance type is document, proceed to Step 2; if the acceptance type is code, proceed to Step 6. Step 2: Use the document extractor to parse the document file into structured text; Step 3: Use the trained existing mature large language model to classify and identify the structured text to obtain the document's classification. Step 4: The large language model retrieves the corresponding acceptance requirements or standards from the acceptance knowledge base based on the document's classification. Step 5: The multimodal large language model performs item-by-item acceptance of all documents to be accepted according to the acceptance requirements, and generates document acceptance results before proceeding to step 9. Step 6: Use the trained, existing mature large language model to identify the development language used in the code to be accepted; Step 7: The large language model retrieves the corresponding acceptance review rules for the development language from the acceptance knowledge base based on the development language. Step 8: The multimodal large language model performs acceptance review on the software code according to the acceptance review rules, and after generating the software code acceptance results, proceeds to step 9; Step 9: Summarize all acceptance results and generate an acceptance report.

2. The software deliverable acceptance method based on a large language model according to claim 1, characterized in that: In step 3, the document classification includes project initiation meeting minutes and project plan.

3. The software deliverable acceptance method based on a large language model according to claim 1, characterized in that: In step 5, the issues that failed the acceptance process are marked in the document acceptance results.

4. The software deliverable acceptance method based on a large language model according to claim 1, characterized in that: The development language or programming language in step 6 includes Java and Python.

5. The software deliverable acceptance method based on a large language model according to claim 1, characterized in that: The acceptance review rules in step 7 include code standards and security requirements.

6. The software deliverable acceptance method based on a large language model according to claim 1, characterized in that: The code acceptance results in step 8 include compliance information and a list of defects.

7. The software deliverable acceptance method based on a large language model according to claim 1, characterized in that: The acceptance report in step 9 includes detailed conclusions of each acceptance, a summary of problems, and suggestions for improvement.

8. The software deliverable acceptance method based on a large language model according to claim 1, characterized in that: The final acceptance report in step 9 is available for users to view and archive.

9. The software deliverable acceptance method based on a large language model according to claim 1, characterized in that: By using a large model to preprocess the records of acceptance requirements and acceptance review rules in the acceptance knowledge base, the output structured content that is conducive to knowledge base indexing and retrieval is obtained, and then imported into the acceptance knowledge base to improve the quality of the knowledge base and enhance the ability to accept results.

10. The software deliverable acceptance method based on a large language model according to claim 9, characterized in that: The steps for documenting the acceptance requirements and review rules for large language model preprocessing are as follows: S1. Set prompt words for the large language model, assign the role of knowledge base document preprocessor, and set the output acceptance rule document structure format; each acceptance rule is on one line, and each rule includes the acceptance category, acceptance object, and acceptance requirements; at the same time, provide suggestions for the knowledge base segment length and separator for the corresponding acceptance rule document; S2. Upload the usual acceptance rules document to the large language model for conversion processing to obtain the processed document; S3. Upload the processed acceptance rules document to the knowledge base and build a retrieval index.

Images