Solution auditing method and device based on large language model and storage medium
By using a hybrid expert model based on a large language model, the solution review method improves review efficiency and adaptability, enables self-learning and dynamic adjustment of professional knowledge, and is suitable for reviewing solution content.
Patent Information
- Application Number
- CN202510807008.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-17
- Publication Date
- 2025-11-07
AI Technical Summary
Existing text content review methods are inefficient and reliant on a single approach in solution management, making it difficult to integrate professional knowledge and adapt to business changes. Traditional methods are costly and difficult to implement.
A hybrid expert model based on a large language model is adopted. Through text preprocessing, generating standard question-answer pairs, and constructing dynamic prompts, a fine-grained hybrid approach combining the Transformer architecture and MoE technology is used to achieve dynamic review.
It improves the efficiency of automated workflow for solution review, adapts to business changes, solves the problems of insufficient manpower for multi-expert review and insufficient experience of single experts, and enables self-learning and dynamic adjustment of professional knowledge.
Smart Images
Figure CN120910261A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of computer, in particular to a solution auditing method and device based on a large language model and a storage medium. BACKGROUND
[0002] LLM (Large Language Model): refers to a deep learning model trained using a large amount of text data, which can generate natural language text or understand the meaning of language text. Large language models can handle a variety of natural language tasks such as text classification, question answering, and dialogue, and are an important way to artificial intelligence.
[0003] MoE (Mixture of Experts): an ensemble learning algorithm that breaks down a complex task into multiple subtasks and trains multiple expert models to solve these subtasks. Each expert model focuses on solving a specific subtask, so they can better learn and understand specific aspects of the data.
[0004] The popularity and development of the Internet have led to more and more system-level audits, and with the continuous development of artificial intelligence technology, automated means are increasingly used to replace manual content audits. Current mainstream text content auditing technology mainly relies on string matching based on word libraries, which is too single in form and has low accuracy and reproducibility. In addition, integrating deep learning neural network technology into text auditing can improve auditing efficiency to some extent, but there are still problems such as high cost and difficulty in rule sorting and model training.
[0005] In daily telecommunications work, business personnel need to conduct content audits on solutions provided by solution managers, i.e., solution checks, to control whether hardware, software, and cloud network services can be replaced by self-developed products and capabilities to optimize solutions, reduce costs, and improve profit margins. Such audits require a combination of extensive professional knowledge and experience, and as business changes, corresponding audit strategies and conditions will also change. Traditional text auditing methods are too limited for solution checks, and more efficient content auditing methods are needed to incorporate professional knowledge and requirements. SUMMARY
[0006] The present application provides a solution auditing method and device based on a large language model and a storage medium to address the deficiencies in the prior art, which is suitable for solution content checking and addresses the efficiency of content audits required by professional requirements in actual business.
[0007] To achieve the above purpose, the present application adopts the following technical solutions:
[0008] A solution auditing method based on a large language model, comprising the following steps:
[0009] Step S1, text preprocessing is performed on an input document; the input document comprises a solution document and a knowledge document; after preprocessing, a solution text vector and a knowledge vector are obtained;
[0010] Step S2, a standard question and answer pair is generated based on the knowledge vector;
[0011] Step S3, the standard question and answer pair is classified based on a category of professional knowledge, a plurality of LLMs are trained using the solution text vector and the standard question and answer pairs of different categories, and LLMs corresponding to different categories of professional knowledge are obtained as hybrid expert models;
[0012] Step S4, a dynamic prompt is constructed, and a solution to be audited is dynamically audited based on the dynamic prompt and the hybrid expert model.
[0013] To optimize the above technical solution, the following specific measures are taken:
[0014] Further, in step S1, the text preprocessing of the input document is specifically: converting the input document into text content, performing text cleaning and text parsing on the converted text content, and vectorizing the segmented converted text through embedding operation.
[0015] Further, in step S2, the source object of the standard question and answer pair is a human or a large model, if the source object is a human, the ready-made question and answer pair is format-converted to obtain the standard question and answer pair, and if the source object is a large model, a prompt instruction prompt is input into an LLM to generate a question and answer pair;
[0016] The form of the prompt instruction prompt is "You are a professional question and answer pair extraction expert, according to the following knowledge document Generate a set of question and answer pairs, and the question and answer pair is in the form of 'Q: {question}, A: {answer}'.
[0017] The question and answer pair reaches a target number C t , and the knowledge document and the standard question and answer pair set are obtained
[0018]
[0019] In the formula, D knowledge represents the standard question and answer pair.
[0020] Further, in step S3, the plurality of LLMs are trained using the solution text vector and the standard question and answer pairs of different categories specifically as follows:
[0021] Step S3.1, the LLM is based on a Transformer architecture, and fine-grained mixing is performed by executing MoE at each layer of the Transformer to construct a mixed expert model, which is expressed as follows:
[0022]
[0023] wherein, is the mixed expert model, W l is a linear transformation, g i (·) is a routing function with sparse output characteristics, used to activate part of the experts, is the feedforward sublayer of the ith expert in the lth layer, x is the text and knowledge features composed of the solution text vector and the standard question and answer pair; N is the number of experts;
[0024] The routing function achieves knowledge filtering by specifying the number of routes, which is expressed as follows:
[0025] g i (W l x)=SoftMax(top k (W l x))
[0026] In the formula, SoftMax is the SoftMax activation function, top k (W l x) represents selecting top k routes; the route represents the source of knowledge;
[0027] Step S3.2, the LLM is balanced trained using knowledge from routes other than the top k routes; the expression of the load balancing loss is as follows:
[0028]
[0029] wherein, L loss represents the load balancing loss, u i represents the comprehensive features, p i represents the probability, β is the current data batch processing parameter, α is the hyperparameter, g i (W l x) is the knowledge representation after routing filtering, and N is the number of experts.
[0030] Further, in step S4, the constructing a dynamic prompt is specifically:
[0031] Step S4.1, constructing an audit rule model S require :
[0032] Srequire = [S n , S desc , S exp ]
[0033] Wherein, S n is the rule name, S desc is the rule description, and S exp is the rule example.
[0034] Step S4.2, fill in the audit rule model S require to build specific rules, the content is: "You are a solution expert, content audit rule S n , there are problems in S exp : S desc ";
[0035] Step S4.3, all audit rules are collected to form a set of requirements for audit.
[0036] Further, in step S4, the dynamic audit of the solution to be audited based on dynamic prompt and mixed expert model is specifically:
[0037] Step S4.4, select data item d in state "to be audited" from the solution to be audited D target to form the target data set D target :
[0038] D target = {d ∈ D | d = "to be audited"}
[0039] Step S4.5, filter the target data set D target , determine the audit type according to the data source S, business type B and core data identifier C:
[0040] Audit type = f(S, B, C)
[0041] Wherein, f is a function that determines the audit type according to the input S, B and C;
[0042] Step S4.6, judge whether the parallel audit condition is met, if the data source exists dependency, the business type exists dependency and there is core data, the parallel audit condition is met; Start mixed expert model parallel audit, otherwise execute single expert audit process, the audit result is R;
[0043] Step S4.7, after classifying the audit results, traverse the audit rules in the set of requirements for audit Use the audit rules to build prompt templates according to the audit result categories, and input the prompt templates into LLM for detailed audit.
[0044] Further, step S4.7 is specifically:
[0045] Step S4.7.1, calculate the comprehensive index value of the audit content sample;
[0046] Step S4.7.2, split the audit results of the mixed expert model according to the comprehensive index value of the audit content sample,
[0047]
[0048] represents the i-th split expert audit result;
[0049] Step S4.7.3, construct a prompt template according to the type of audit result using the audit rule, the prompt template is as follows: "audit rule whether there is a problem indicated by the rule";
[0050] Step S4.7.4, input the prompt template into the LLM to obtain a refined audit result
[0051]
[0052] wherein, represents the i-th refined audit result.
[0053] Further, step S4.7.1 is specifically:
[0054] Step S4.7.1.1, based on the relationship between the audit index and the audit content, construct an index dimension matrix P m×n , wherein m represents the number of audit content samples, and n represents the number of indexes;
[0055] Step S4.7.1.2, perform index standardization operations, including positive audit indexes: negative audit indexes: wherein, i is the audit content sample number, j is the index number, x ij represents the original value of the j-th index of the i-th audit content sample, and the positive audit index and the negative audit index are taken as the standardized index min(x j ) represents the minimum value of the j-th index x j in all samples, and max(x j ) represents the maximum value of the j-th index x j in all samples.
[0056] Step S4.7.1.3, calculating the proportion p of the jth standardized index of the ith audit content sample ij :
[0057]
[0058] Step S4.7.1.4, calculating the entropy value e of the jth standardized index j , and its difference coefficient g j :
[0059]
[0060] g j = 1-e j
[0061] Step S4.7.1.5, calculating the weight w of the standardized index using the difference coefficient of the standardized index j :
[0062]
[0063] Step S4.7.1.6, calculating the comprehensive index value of the ith audit content sample using the weight
[0064]
[0065] In the formula, F i represents the comprehensive index value of the ith audit content sample.
[0066] The present application also proposes an electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the computer program to implement the solution audit method based on the large language model as described above.
[0067] The present application also proposes a computer readable storage medium storing a computer program, wherein the computer program enables a computer to execute the solution audit method based on the large language model as described above.
[0068] The present application has the following advantages: 1. The present application constructs a hybrid expert through a large model, which acts on self-learning and feedback audit of knowledge in different business fields, solving the problems of manpower in multi-expert audit and incomplete experience in single-expert audit. 2. The present application accumulates audit results and feedback through an online learning mechanism, dynamically adjusts audit strategies and audit conditions to adapt to changing business needs. 3. The present application realizes the conversion of artificial audit link to large model audit link, improving the efficiency of automatic flow of scheme review. BRIEF DESCRIPTION OF DRAWINGS
[0069] Figure 1A flowchart of a solution auditing method based on a large language model according to the present application.
[0070] Figure 2 A block diagram of a solution auditing system based on a large language model according to the present application.
[0071] Figure 3 A solution deconstruction template. DETAILED DESCRIPTION
[0072] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.
[0073] Embodiment one
[0074] The present application proposes a solution auditing method based on a large language model. The overall flow of the method is shown in Figure 1 , and includes the following steps:
[0075] Step S1, text preprocessing is performed on an input document; the input document includes a solution document and a knowledge document; after preprocessing, a solution text vector and a knowledge vector are obtained; the preprocessing specifically includes: converting the input document into text content, performing text cleaning and text parsing on the converted text content, and performing vectorization on the segmented converted text through embedding operation.
[0076] Step S2, a standard question-answer pair D knowledge =(d1,d2,...,d n ) is generated based on the knowledge vector.
[0077] The question-answer pair has the following form:
[0078]
[0079] The "from" object can be set to "human" or "gpt", indicating that the source object is a human or a gpt large model. When the "from" object is "human", the professional knowledge construction example is:
[0080]
[0081] When the "from" object is "gpt", the professional knowledge construction example is:
[0082]
[0083] If the source object is a human, the ready-made question and answer pair is format-converted to obtain a standard question and answer pair, and if the source object is a large model, a prompt instruction prompt is input into the LLM to generate a question and answer pair;
[0084] The form of the prompt instruction prompt is "You are a professional question and answer pair extraction expert, and according to the following knowledge document generate a set of question and answer pairs in the form of 'Q: {question}, A: {answer}'";
[0085] The prompt instruction prompt is defined as follows in the embodiment:
[0086] "You are a professional question and answer pair extraction expert, and according to the following content 'The hardware product device type field in the scheme disassembly template can only be one of the following fields, including server, gateway, router, switch, camera, storage, power supply device, wireless terminal, fusion device, security device, auxiliary material, and other', generate a set of question and answer pairs in the form of 'Q: {question}, A: {answer}'"
[0087] The scheme disassembly template is as shown in Figure 3 .
[0088] The question and answer pair reaches the target number C t , and the knowledge document and the standard question and answer pair set are obtained
[0089]
[0090] In the formula, D knowledge represents the standard question and answer pair.
[0091] Step S3, classify the standard question and answer pair based on the category of professional knowledge, train multiple LLMs using the solution text vector and standard question and answer pairs of different categories, obtain LLMs corresponding to different categories of professional knowledge, and use them as mixed expert models; that is, one LLM is one expert, and multiple LLMs are mixed experts. Step S3 includes the following sub-steps:
[0092] Step S3.1, the LLM is based on the Transformer architecture, and fine-grained mixing is performed by executing MoE at each layer of the Transformer to construct a mixed expert model, which is expressed by the following formula:
[0093]
[0094] wherein, is the mixed expert model, W l is a linear transformation, g i (·) is a routing function with sparse output characteristics, which is used to activate part of the experts, is the feedforward sub-layer of the ith expert in the lth layer, x is the text and knowledge features composed of solution text vector and standard question-answer pairs; N is the number of experts;
[0095] The routing function realizes the filtering of knowledge by specifying the number of routes, and the expression is as follows:
[0096] g i (W l x)=SoftMax(top k (W l x))
[0097] In the formula, SoftMax is a SoftMax activation function, top k (W l x) represents selecting top k routes; route represents the source of knowledge;
[0098] Step S3.2, balanced training of LLM using knowledge from routes other than top k routes; the expression of the load balancing loss is as follows:
[0099]
[0100] Where, L loss represents the load balancing loss, u i represents the comprehensive features, p i represents the probability, β is the current data batch parameter, α is the hyperparameter, g i (W l x) is the knowledge representation after routing filtering, and N is the number of experts.
[0101] Step S4, constructing a dynamic prompt, and dynamically auditing the to-be-audited solution based on the dynamic prompt and the mixed expert model. Including the following sub-steps:
[0102] Step S4.1, constructing an audit rule model S require :
[0103] S require =[S n ,S desc ,S exp ]
[0104] Where, S n is the rule name, S desc is the rule description, and S exp is the rule example.
[0105] Step S4.2, filling the audit rule model S requireTo build specific rules, fill in the content: "You are a solution expert, there are problems in the rule S n Content review, S exp There are problems: S desc ";
[0106] Review rules, for example:
[0107] "You are a solution expert, content review of 'Telecom self-capability application use', there are problems in'software, first-level function is network operation center, second-level function is empty, self-capability name is empty, and capability code is empty': does not meet the requirements of telecom self-capability application use, software first-level function 'network operation center' matches the approximate capability 'network security and operation platform'"
[0108] Step S4.3, collect all review rules Form a set of requirements for review.
[0109] Step S4.4, select data items d with status "to be reviewed" from the to-be-reviewed solution D to form the target data set D target :
[0110] D target = {d ∈ D | d = "to be reviewed"}
[0111] Example of data to be reviewed:
[0112] Data A: software, first-level function is "network operation center", second-level function is "resource library construction", self-capability name is empty, and capability code is empty.
[0113] Data B: software, first-level function is "customer relationship management", second-level function is "marketing", self-capability name is "customer data integration", and capability code is "CRM-001".
[0114] Step S4.5, filter the target data set D target According to the data source S, business type B and core data identifier C to determine the review type:
[0115] Review type = f(S, B, C)
[0116] Where f is a function that determines the review type based on the input S, B and C;
[0117] Parse data A to obtain its data configuration parameters:
[0118]
[0119] Step S4.6, determine whether the parallel audit condition is met, if there is a dependency between data sources, a dependency between business types, and there is core data, the parallel audit condition is met; start the mixed expert model parallel audit, otherwise execute the single expert audit process, and the audit result is R; take data A as an example, and the prompt template is: “software self-capability cannot be empty, does data A have similar problems?”
[0120] Return the audit result “data A has problems, software self-capability is empty”.
[0121] Step S4.7, after classifying the audit results, traverse the audit rules in the required audit set Using the audit rules, construct the prompt template according to the audit result category, and input the prompt template into the LLM for detailed audit. Step S4.7 includes the following sub-steps:
[0122] Step S4.7.1, calculate the comprehensive index value of the audit content sample; step S4.7.1 includes the following sub-steps:
[0123] Step S4.7.1.1, based on the relationship between the audit index and the audit content, construct the index dimension matrix P m×n , where m represents the number of audit content samples, and n represents the number of indexes;
[0124] Step S4.7.1.2, perform index standardization operation, including positive audit index: Negative audit index: Where i is the audit content sample number, j is the index number, x ij represents the original value of the jth index of the ith audit content sample, and the positive audit index and the negative audit index are used as the standardized index mig(x j ) represents the minimum value of the jth index x j in all samples, and max(x j ) represents the maximum value of the jth index x j in all samples.
[0125] Step S4.7.1.3, calculate the proportion p ij of the jth standardized index of the ith audit content sample:
[0126]
[0127] Step S4.7.1.4, calculate the entropy value e j of the jth standardized index and its difference coefficient g j :
[0128]
[0129] g j =1-e j
[0130] Step S4.7.1.5: Calculate the weight w of the standardized indicator using the difference coefficient of the standardized indicator. j :
[0131]
[0132] Step S4.7.1.6: Calculate the comprehensive index value of the i-th sample of reviewed content using weights:
[0133]
[0134] In the formula, F i This represents the comprehensive index value of the i-th sample of reviewed content.
[0135] Step S4.7.2: Based on the comprehensive index values of the audit content sample, split the audit results of the hybrid expert model.
[0136]
[0137] This represents the i-th expert review result after splitting.
[0138] Step S4.7.3: Using the audit rules, construct a prompt template according to the audit result type. The prompt template is as follows: "Audit Rules" Does the problem indicated by the rule exist?
[0139] Step S4.7.4: Input the prompt template into the LLM file to obtain detailed review results.
[0140]
[0141] In the formula, This represents the i-th refined review result.
[0142] Example 2
[0143] This invention proposes a solution review system based on a large language model, corresponding to the method in Embodiment 1, such as... Figure 2 As shown, it includes:
[0144] The preprocessing module receives user input or knowledge data, i.e., solution documents. After relevant data cleaning and text parsing operations, the documents are vectorized and stored to provide a data foundation for subsequent processes.
[0145] Hybrid expert module: this module is based on the type of knowledge base to build the corresponding expert module, including the construction of audit questions, the formulation of audit strategy, the generation of audit conditions.
[0146] Dynamic audit module: this module is based on the dynamic prompt of the hybrid expert module and the large model, realizes the dynamic generation of the audit strategy and the audit conditions, and outputs the professional audit content for the current environment.
[0147] The implementation mode of each module and the function of the module in the system is completely consistent with the method of the embodiment one, so it will not be repeated here.
[0148] Embodiment three
[0149] The application provides an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the solution audit method based on the large language model is realized.
[0150] Embodiment four
[0151] The application provides a computer readable storage medium, which stores a computer program, and the computer program enables a computer to execute the solution audit method based on the large language model.
[0152] In the embodiments disclosed in the present application, the computer storage medium can be a tangible medium, which can contain or store programs for use by or in conjunction with an instruction execution system, device or equipment. The computer storage medium can include but is not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the above. More specific examples of computer storage medium can include one or more wire-based electrical connections, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.
[0153] Those skilled in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in the present application can be realized by electronic hardware or a combination of electronic hardware and computer software. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.
[0154] The above merely describes the preferred embodiments of the present application, and the protection scope of the present application is not limited to the above-mentioned embodiments. Any technical scheme falling within the concept of the present application shall fall within the protection scope of the present application. It should be noted that, for ordinary skilled in the art, some improvements and refinements without departing from the principles of the present application shall be considered as falling within the protection scope of the present application.
Claims
1. A method for auditing a solution based on a large language model, characterized in that, The method comprises the following steps: Step S1, text preprocessing is performed on an input document; the input document comprises a solution document and a knowledge document; After preprocessing, a solution text vector and a knowledge vector are obtained; Step S2, a standard question and answer pair is generated based on the knowledge vector; Step S3, the standard question and answer pair is classified based on a category of professional knowledge, a plurality of LLMs are trained using the solution text vector and the standard question and answer pairs of different categories, and LLMs corresponding to different categories of professional knowledge are obtained as a hybrid expert model; Step S4, a dynamic prompt is constructed, and the solution to be audited is dynamically audited based on the dynamic prompt and the hybrid expert model. 2.The large language model-based solution audit method of claim 1, wherein, In step S1, the text preprocessing of the input document is specifically: converting the input document into text content, performing text cleaning and text analysis on the converted text content, and performing vectorization on the segmented converted text through embedding operation. 3.The large language model-based solution review method of claim 1, wherein, In step S2, the source object of the standard question and answer pair is a human or a large model, if the source object is a human, the existing question and answer pair is format-converted to obtain the standard question and answer pair, and if the source object is a large model, a prompt instruction prompt is input into an LLM to generate a question and answer pair; The form of the prompt instruction prompt is "You are a professional question and answer pair extraction expert, according to the following knowledge document Generate a set of question and answer pairs, and the form of the question and answer pairs is as follows: 'Q: {question}, A: {answer}'" Question and answer pairs reach a target number C t Subsequently, a knowledge document and a set of standard question and answer pairs are obtained In the formula, D knowledge represents a standard question-answer pair. 4.The large language model-based solution review method of claim 1, wherein, In step S3, the training of the plurality of LLMs using the solution text vector and the standard question and answer pairs of different categories is specifically: In step S3.1, the LLM is based on a Transformer architecture, and fine-grained mixing is performed by executing MoE at each layer of the Transformer to construct a hybrid expert model, which is expressed by the following formula: wherein, is a hybrid expert model, W l is a linear transformation, g i (·) is a routing function with sparse output characteristics to activate partial experts, is the feed-forward sub-layer of the ith expert in the lth layer, x is the text and knowledge features composed of solution text vectors and standard question-answer pairs; N is the number of experts; The routing function filters knowledge by specifying the number of routes, and the expression is as follows: g i (W l x) = SoftMax(top k (W l x)) where SoftMax is a SoftMax activation function, top k (W l x) represents selecting top k routes; a route represents a source of knowledge. Step S3.2, balance training of LLM by using knowledge of routes other than the top k routes. The expression of load balancing loss is as follows: where L loss represents the loss of load balancing, u i represents the comprehensive features, p i represents the probability, β is the current data batch processing parameter, α is the hyperparameter, g i (W l x) is the knowledge representation after routing filtering, and N is the number of experts. 5.The large language model-based solution review method of claim 1, wherein, In step S4, the construction of the dynamic prompt is specifically: Step S4.1, constructing an audit rule model S require : S require = [S n , S desc , S exp ] wherein S n is a rule name, S desc is a rule description, S exp is a rule example; Step S4.2, filling the audit rule model S require To build a specific rule, the content is filled in as: "You are a solution expert, and the rule S n is performed for content audit, S exp There are problems: S desc "; Step S4.3, Collect all audit rules A set of audit requirements is formed. 6.The large language model-based solution review method of claim 1, wherein, In step S4, the dynamic auditing of the solution to be audited based on the dynamic prompt and the hybrid expert model is specifically: Step S4.
4. Selecting from the solution D to be audited the data items d having the status "to be audited", constituting the target data set D target : D target = { d e D | d = "to be reviewed"} Step S4.5, screening the target dataset D target Filtering, determining the audit type according to the data source S, the business type B and the core data identifier C: Audit type = f(S, B, C) Where f is a function that determines the audit type according to the input S, B and C; Step S4.6, determine whether the parallel audit condition is met, if the data source exists, the business type exists, and there is core data, the parallel audit condition is met; start the hybrid expert model parallel audit, otherwise execute the single expert audit process, and the audit result is R; Step S4.7, after classifying the audit results, traversing the audit rules in the audit set Using the audit rules, constructing a prompt template according to the audit result category, and inputting the prompt template into the LLM for refined audit. 7.The large language model-based solution review method of claim 6, wherein, Step S4.7 is specifically: Step S4.7.1, calculate the comprehensive index value of the audit content sample; Step S4.7.2, split the audit result of the hybrid expert model according to the comprehensive index value of the audit content sample, represents the i-th split expert review result; Step S4.7.3, using the audit rule, constructing a prompt template according to the audit result type, the prompt template being as follows: "Audit rule Does the problem indicated by the rule exist?" Step S4.7.4, input the prompt template into the LLM to get refined review results In the formula, represents the i-th refined review result. 8.The large language model-based solution audit method of claim 7, wherein, Step S4.7.1 is specifically: Step S4.7.1.1, constructing the index dimension matrix P based on the relationship between the audit index and the audit content m×n , wherein m represents the number of audit content samples, and n represents the number of indexes; Step S4.7.1.2, performing index standardization operation, including positive audit index: Negative audit index: Wherein, i is the audit content sample number, j is the index number, x ij represents the original value of the jth index of the ith audit content sample, and the positive audit index and the negative audit index are standardized as the index min(x j ) represents the minimum value of the jth index x j in all samples, and max(x j ) represents the maximum value of the jth index x j in all samples; Step S4.7.1.3, calculating the proportion p of the jth standardized indicator of the ith audit content sample ij : Step S4.7.1.4, calculating the entropy value e of the jth normalized indicator j , and its coefficient of variation g j : Step S4.7.1.
5. Calculating the weight w of the normalized indicator using the coefficient of variation of the normalized indicator j : Step S4.7.1.6, calculate the comprehensive index value of the i-th audit content sample using the weight: In the formula, F i represents the comprehensive index value of the i-th audit content sample.
9. An electronic device, comprising: It comprises: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the solution auditing method based on the large language model according to any one of claims 1-8.
10. A computer readable storage medium storing a computer program, characterized in that, The computer program enables the computer to execute the solution auditing method based on the large language model according to any one of claims 1-8.