Automatic intelligent evaluation method and system based on large language model

Abstract

The invention provides an automatic intelligent evaluation method and system based on a large language model, and relates to the technical field. The method comprises the steps of obtaining a descriptive text and an evaluation task of a to-be-evaluated object; constructing an automatic intelligent evaluation system based on a large language model; inputting the descriptive text and the evaluation task into an index system construction module based on a large model to generate a plurality of evaluation indexes; inputting the evaluation indexes and the descriptive text into a large-model-based index score calculation module, and calculating a score corresponding to each evaluation index through a predefined algorithm; inputting the evaluation indexes into an index weight calculation module based on a large model, and calculating a weight value of each evaluation index by adopting an analytic hierarchy process; performing weighted summation on the weight value of each evaluation index and the score corresponding to each evaluation index to obtain a comprehensive total score; and inputting the comprehensive score into the trained large language model to obtain a final evaluation result. According to the invention, the task evaluation efficiency can be improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an automated intelligent evaluation method and system based on a large language model. Background Art

[0002] Evaluation is one of the commonly used decision support methods in management and production life. It can estimate the overall score of the target by integrating the performance of multiple different dimensions, and use this as the basis for ranking to support the subsequent decision-making process. The main process of evaluation can be summarized as five main steps: building an indicator system, collecting evaluation data, calculating indicator scores, calculating indicator weights, and calculating evaluation results. Although there are mature algorithms or methods to support the implementation of the above five evaluation steps, including hierarchical analysis method, fuzzy evaluation and TOPSIS evaluation, traditional evaluation methods have the following problems: (1) It requires human intervention, has high implementation costs, and cannot be implemented automatically; among them, building an indicator system, calculating indicator scores, and calculating indicator weights all require expert experience and designing corresponding mathematical calculation models. The whole process is inefficient and cannot be implemented automatically; (2) The cost of data collection is high, and the workload of data preprocessing is large; traditional evaluation methods require the data and scores of each indicator to be sorted and calculated separately, which is inefficient, requires a lot of manpower support, and is not flexible enough.

[0003] Therefore, the traditional method of assessment tasks is a process with high cost, low efficiency and heavy reliance on expert experience, and its degree of automation and intelligence is low. Summary of the Invention

[0004] To address the existing technical issues of high cost, low efficiency, heavy reliance on expert experience, and low automation and intelligence, the present invention provides an automated intelligent evaluation method and system based on a large language model. The technical solution is as follows:

[0005] In one aspect, a method for automated intelligent evaluation based on a large language model is provided. The method is implemented by an automated intelligent evaluation device based on a large language model, and the method includes:

[0006] S1. Obtain descriptive text data and evaluation tasks of the object to be evaluated;

[0007] S2. Constructing an automated intelligent evaluation system based on a large language model; the system includes: an indicator system construction module based on the large model, an indicator score calculation module based on the large model, and an indicator weight calculation module based on the large model;

[0008] S3. Inputting the descriptive text data and the evaluation task into an indicator system construction module based on a large model, analyzing the requirements of the evaluation task and the content of the descriptive text through the large language model, and automatically generating multiple evaluation indicators suitable for the task;

[0009] S4, inputting the multiple evaluation indicators and the descriptive text data into a large model-based indicator score calculation module, matching the descriptive text data with each evaluation indicator, and calculating the score corresponding to each evaluation indicator through a predefined algorithm;

[0010] S5. Input multiple evaluation indicators into the indicator weight calculation module based on the large model, and use the hierarchical analysis method to calculate the weight value of each evaluation indicator;

[0011] S6. Perform a weighted summation of the weight value of each evaluation indicator and the score corresponding to each evaluation indicator to obtain a comprehensive total score of the object to be evaluated;

[0012] S7. Input the comprehensive score of the object to be evaluated into the trained large language model to obtain the final evaluation result.

[0013] Optionally, the process of performing weighted summation of the weight value of each indicator and the score corresponding to each indicator is represented by the following formula (1):

[0014] (1)

[0015] in, Indicates the comprehensive total score of the object to be evaluated; represents the weight of the i-th evaluation indicator; represents the i-th evaluation indicator.

[0016] Optionally, the comprehensive total score of the evaluation object is used to compare the advantages and disadvantages of different evaluation objects, or to perform a qualitative evaluation on a single evaluation object.

[0017] Optionally, the large model-based indicator system construction module is used to generate an indicator system for the evaluation task based on the input descriptive text data of the object to be evaluated.

[0018] The large model-based indicator score calculation module is used to generate a score for each evaluation indicator for each indicator of the indicator system of the evaluation task;

[0019] The large-model-based indicator weight calculation module is used to determine the weight of each evaluation indicator in the comprehensive evaluation.

[0020] Optionally, the predefined algorithm is based on predefined rules, including: percentage calculation, threshold level, segment score and level score.

[0021] Optionally, the step S5 inputs multiple evaluation indicators into an indicator weight calculation module based on a large model, and uses a hierarchical analysis method to calculate the weight value of each evaluation indicator, including:

[0022] S51. Setting basic rules for weight calculation and assigning initial weights to the obtained indicators;

[0023] S52. Based on the obtained indicators, an importance matrix is ​​constructed, and the weight of each indicator is calculated using a matrix decomposition algorithm;

[0024] S53. Use the hierarchical analysis method to optimize the weight of each indicator and obtain the final weight of each indicator.

[0025] In another aspect, an automated intelligent evaluation system based on a large language model is provided. The system is applied to an automated intelligent evaluation method based on a large language model, and the system includes:

[0026] A first acquisition unit is used to acquire descriptive text data of the object to be evaluated and the evaluation task;

[0027] A construction unit for constructing an automated intelligent evaluation system based on a large language model; the system includes: an indicator system construction module based on the large model, an indicator score calculation module based on the large model, and an indicator weight calculation module based on the large model;

[0028] A generating unit, configured to input the descriptive text data and the evaluation task into an indicator system construction module based on a large model, analyze the requirements of the evaluation task and the content of the descriptive text through the large language model, and automatically generate a plurality of evaluation indicators suitable for the task;

[0029] A first calculation unit is configured to input the plurality of evaluation indicators and the descriptive text data into a large model-based indicator score calculation module, match the descriptive text data with each evaluation indicator, and calculate a score corresponding to each evaluation indicator using a predefined algorithm;

[0030] The second calculation unit is used to input multiple evaluation indicators into the indicator weight calculation module based on the large model, and calculate the weight value of each evaluation indicator using the hierarchical analysis method;

[0031] The second obtaining unit is used to perform weighted summation on the weight value of each evaluation indicator and the score corresponding to each evaluation indicator to obtain a comprehensive total score of the object to be evaluated;

[0032] The third acquisition unit is used to input the comprehensive score of the object to be evaluated into the trained large language model to obtain the final evaluation result.

[0033] Optionally, the process of performing weighted summation of the weight value of each indicator and the score corresponding to each indicator is represented by the following formula (1):

[0034] (1)

[0035] in, Indicates the comprehensive total score of the object to be evaluated; represents the weight of the i-th evaluation indicator; represents the i-th evaluation indicator.

[0036] Optionally, the comprehensive total score of the evaluation object is used to compare the advantages and disadvantages of different evaluation objects, or to perform a qualitative evaluation on a single evaluation object.

[0037] Optionally, the large model-based indicator system construction module is used to generate an indicator system for the evaluation task based on the input descriptive text data of the object to be evaluated.

[0038] The large model-based indicator score calculation module is used to generate a score for each evaluation indicator for each indicator of the indicator system of the evaluation task;

[0039] The large-model-based indicator weight calculation module is used to determine the weight of each evaluation indicator in the comprehensive evaluation.

[0040] Optionally, the predefined algorithm is based on predefined rules, including: percentage calculation, threshold level, segment score and level score.

[0041] Optionally, the second computing unit is configured to:

[0042] Set the basic rules for weight calculation and assign initial weights to the obtained indicators;

[0043] Based on the obtained indicators, an importance matrix is ​​constructed and the weight of each indicator is calculated using the matrix decomposition algorithm;

[0044] The analytic hierarchy process is used to optimize the weight of each indicator and obtain the final weight of each indicator.

[0045] On the other hand, an automated intelligent evaluation device based on a large language model is provided, comprising: a processor; and a memory, wherein computer-readable instructions are stored on the memory, and when the computer-readable instructions are executed by the processor, any one of the above-mentioned automated intelligent evaluation methods based on a large language model is implemented.

[0046] On the other hand, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, and the at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned automated intelligent evaluation methods based on a large language model.

[0047] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0048] The embodiment of the present invention first obtains descriptive text data and evaluation tasks of an object to be evaluated; secondly, constructs an automated intelligent evaluation system based on a large language model; the system includes: an indicator system construction module based on the large model, an indicator score calculation module based on the large model, and an indicator weight calculation module based on the large model; the descriptive text data and the evaluation task are input into the indicator system construction module based on the large model, and multiple evaluation indicators of the evaluation task are generated through the large language model; the multiple evaluation indicators and the descriptive text data are input into the indicator score calculation module based on the large model, the descriptive text data is matched with each evaluation indicator, and the score corresponding to each evaluation indicator is calculated through a predefined algorithm; finally, the multiple evaluation indicators are input into the indicator weight calculation module based on the large model, and the hierarchical analysis method is used to calculate the weight value of each evaluation indicator; the weight value of each evaluation indicator and the score corresponding to each evaluation indicator are weighted and summed to obtain a comprehensive total score of the object to be evaluated; the comprehensive score of the object to be evaluated is input into the trained large language model to obtain a final evaluation result.

[0049] The embodiments of the present invention do not require a large amount of expert experience, and at the same time have low requirements on the structuring degree of the data or information of the evaluation object. The overall implementation cost is low and the efficiency is high, which can effectively solve the problems existing in traditional evaluation methods and systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0051] Figure 1 This is a flow chart of an automated intelligent evaluation method based on a large language model provided by an embodiment of the present invention;

[0052] Figure 2 This is a structural diagram of an implementation of an automated intelligent evaluation method based on a large language model provided by an embodiment of the present invention;

[0053] Figure 3 This is a schematic diagram of the structure of an intelligent assessment method based on an education scenario provided by an embodiment of the present invention;

[0054] Figure 4 This is a schematic diagram of the structure of an intelligent evaluation method based on an enterprise management scenario provided by an embodiment of the present invention;

[0055] Figure 5 This is a schematic diagram of the structure of an intelligent evaluation method based on a medical field scenario provided by an embodiment of the present invention;

[0056] Figure 6 This is a schematic diagram of the structure of an intelligent evaluation method based on a legal field scenario provided by an embodiment of the present invention;

[0057] Figure 7 This is a block diagram of an automated intelligent evaluation system based on a large language model provided by an embodiment of the present invention;

[0058] Figure 8 This is a structural diagram of an automated intelligent evaluation device based on a large language model provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0059] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0060] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as an "exemplary" in the present invention should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of the word "exemplary" is intended to present concepts in a concrete manner. Furthermore, in the embodiments of the present invention, "and / or" can mean both or either of the two.

[0061] In the embodiments of the present invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, when the distinction is not emphasized, the meanings they convey are the same. The terms "of," "corresponding," and "corresponding" may sometimes be used interchangeably. It should be noted that, when the distinction is not emphasized, the meanings they convey are the same.

[0062] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0063] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0064] The embodiment of the present invention provides an automated intelligent evaluation method based on a large language model. The method can be implemented by an automated intelligent evaluation device based on a large language model. The automated intelligent evaluation device based on a large language model can be a terminal or a server. Figure 1 The flowchart of the automated intelligent evaluation method based on a large language model is shown. The processing flow of the method may include the following steps:

[0065] S1. Obtain descriptive text data and evaluation tasks of the object to be evaluated.

[0066] The descriptive text data may be data read from a file system, data recorded in a database, or data uploaded via a network interface.

[0067] Among them, descriptive text data needs to be converted and preliminarily cleaned before entering the system. The preliminary cleaning process includes removing noise data, standardization, text deduplication and processing of missing values, etc., which can ensure the consistency and availability of data.

[0068] S2. Construct an automated intelligent evaluation system based on a large language model; the system includes: an indicator system construction module based on a large model, an indicator score calculation module based on a large model, and an indicator weight calculation module based on a large model.

[0069] Optionally, an indicator system construction module based on a large model is used to generate an indicator system for the evaluation task based on the input descriptive text data of the object to be evaluated.

[0070] Among them, the descriptive text data related to the task to be evaluated is used as input into the indicator system construction module based on the big model. By understanding the themes, relationships, trends and important concepts in the text data, the various indicators related to the evaluation task are automatically identified.

[0071] In a feasible implementation, the specific process of calculating the indicator system building module based on the large model includes:

[0072] (1) Collect input data; input data refers to all data content related to the task to be evaluated. According to the different organizational forms of data, input data can be divided into two categories: structured data and unstructured data. Structured data refers to tabular data stored in a fixed format, such as database records and Excel data; unstructured data mainly includes descriptive information in the form of natural language text, which can include reports, documents and user comments.

[0073] (2) Set the indicator quantity constraint; in one feasible implementation, set a minimum indicator quantity threshold so that the number of generated indicators is lower than the set threshold, which can ensure that the generated indicators are rich and comprehensive. When generating indicators, the minimum indicator quantity threshold can be set in the prompt. The code for setting the minimum indicator quantity threshold is shown in the table below;

[0074] Table 1

[0075]

[0076] (3) Pre-set first-level indicators. The large language model can be used to supplement the secondary indicators under each first-level indicator in the prompt. That is, by inputting a preset first-level indicator, such as "equipment reliability," the large language model generates secondary indicators, such as "mean time between failures," "abnormal response speed," and "spare parts replacement cycle." The generated secondary indicators are shown in Table 2.

[0077] Table 2

[0078]

[0079] (4) The number of secondary indicators under the primary indicator is set to no less than a specific value, and a dual constraint strategy is adopted to set the secondary indicators, including quantity control and quality verification.

[0080] Quantity control requires that each primary indicator be accompanied by ≥3 secondary indicators, such as "equipment reliability" → "mean time between failures", "abnormal response speed", "spare parts replacement cycle", and "failure rate".

[0081] Among them, the quality verification is carried out through semantic similarity analysis. For example, setting the cosine similarity < 0.2 ensures the orthogonality of the secondary indicators. and , its cosine similarity is expressed by the following formula (1):

[0082] (1)

[0083] in, Represents the first vector and the second vector The inner product of Represents the first vector The model; Represents the second vector The model; Cosine similarity is a value between 0 and 1, and the closer it is to 1, the higher the similarity.

[0084] (3) Reduce the correlation between indicators through optimization and verification. In order to reduce the correlation between the generated secondary indicators as much as possible, you can set a requirement in the prompt to emphasize low correlation; Table 3 shows the requirements set in the prompt.

[0085] Table 3

[0086]

[0087] Among them, the large model indicator score calculation module is used to generate the score of each evaluation indicator for each indicator of the indicator system of the evaluation task;

[0088] Among them, the indicator weight calculation module based on the large model is used to determine the weight value of each evaluation indicator in the comprehensive evaluation.

[0089] S3. Input the descriptive text data and evaluation tasks into the indicator system construction module based on the large model. The large language model is used to analyze the requirements of the evaluation task and the content of the descriptive text, and multiple evaluation indicators suitable for the task are automatically generated.

[0090] S4. Input multiple evaluation indicators and descriptive text data into the large model-based indicator score calculation module, match the descriptive text data with each evaluation indicator, and calculate the score corresponding to each evaluation indicator through a predefined algorithm.

[0091] In one feasible implementation, direct matching can be performed on structured data. For structured data, each data item can be directly matched to the corresponding indicator. For example, if the indicator is "equipment failure frequency", it can be directly matched based on fields in the data table such as "number of failures / operating time".

[0092] In a feasible implementation, for example, for the "equipment failure frequency" indicator, the system extracts the number of failures and operating time from the data and calculates the equipment failure frequency using the following formula (2):

[0093] (2)

[0094] One feasible implementation utilizes semantic matching for unstructured data. For unstructured text, a large language model is used to parse the text, extracting key information from the text to determine the correspondence between input data and indicators. For example, a large language model might be instructed to "find information related to equipment failures from the following text, including key information such as the number of failures, operating hours, and maintenance times, and summarize it in a dictionary format. The following is an output example: Equipment failure: {Number of failures: 2, Operating hours: 24 hours,..., Maintenance times: 0}."

[0095] For example, if the unstructured text input is: "This equipment experienced two failures in the past week, operated for a total of 24 hours, and was not repaired," the model would extract the following key information: Number of failures: 2; Operating time: 24 hours; Number of repairs: 0. Based on this extracted key information, the system calculates the "Equipment Failure Frequency" and further assigns a score.

[0096] Optionally, the predefined algorithm is based on predefined rules, including: percentage calculation, threshold level, segment score and level score.

[0097] The calculation of the score corresponding to each evaluation indicator can refer to industry standards or be optimized through model training based on historical data. The score corresponding to each evaluation indicator can also be smoothed and weighted to ensure that the calculation results meet industry standards or evaluation requirements.

[0098] S5. Input multiple evaluation indicators into the indicator weight calculation module based on the large model, and use the hierarchical analysis method to calculate the weight value of each evaluation indicator.

[0099] Optionally, the specific implementation process of S5 includes S51-S53:

[0100] S51. Setting basic rules for weight calculation and assigning initial weights to the obtained indicators;

[0101] S52. Based on the obtained indicators, an importance matrix is ​​constructed, and the weight of each indicator is calculated using a matrix decomposition algorithm;

[0102] S53. Use the hierarchical analysis method to optimize the weight of each indicator and obtain the final weight of each indicator.

[0103] In a feasible implementation, the process of calculating the weight value of each evaluation indicator includes:

[0104] (1) Set the basic rules for weight calculation. When calculating weights, it is necessary to assign initial weights to the indicator system based on the indicator's discrimination and industry standard requirements. For example, the largest weight should be assigned to indicators with high discrimination; the largest weight should be assigned to indicators that are relatively important in the task; and the smallest weight can be assigned to indicators with low importance.

[0105] In a feasible implementation, a prompt with a weight allocation rule is input into a large language model so that it automatically performs basic weight allocation. For example, the task objective is a product performance evaluation task, and different products are scored according to the following five indicators. The score range of each indicator is 0 to 100; wherein, a higher score indicates that the product performs better in the dimension corresponding to the evaluation indicator, such as technological innovation and cost-effectiveness. ; A weight is assigned to each indicator through the following weight allocation rule. The weight allocation rule includes: (1) Indicators with high discrimination. If an indicator can significantly distinguish the performance of different products, a large weight is assigned to the indicator, such as 30% or more. For example, evaluation indicators such as "technological innovation" are crucial for distinguishing the pros and cons of different products, so a large weight should be assigned; (2) Relative importance in the task; wherein, in the task objective, some indicators are more important than other indicators. For example, if the product's "price / performance ratio" is more critical than "appearance design" in decision-making, "price / performance ratio" should be assigned a higher weight; (3) Avoid excessive concentration of weights. The total weight of all indicators should be 100%, and the weight of any single indicator should not exceed 40%; (4) The weights of other indicators; For the remaining indicators, appropriate adjustments can be made based on their discrimination and importance. According to the above rules, combined with the task objectives and the provided data, the weight of each indicator is assigned.

[0106] (2) Constructing an importance matrix; constructing an importance matrix can provide a more systematic method for weight calculation.

[0107] When constructing the importance matrix, consider the indicator's importance, discrimination, and practical significance in the evaluation task. For example, the elements in the matrix represent the relative importance of two indicators, and can range from 1 to 9, with 1 being very unimportant and 9 being very important. Assume that after constructing the large-scale indicator system, there are five evaluation indicators: technological innovation, cost-effectiveness, appearance design, user experience, and brand reputation. Based on these five evaluation indicators, a relative importance matrix is ​​constructed, as shown in Table 4 below.

[0108] Table 4

[0109]

[0110] In a feasible implementation, the weight of each indicator is calculated by using a matrix decomposition method including eigenvalue decomposition, singular value decomposition and principal component analysis according to constructing the importance matrix;

[0111] In a feasible implementation, according to the weight of each indicator, one of the following methods, hierarchical analysis method, entropy method or fuzzy comprehensive evaluation method, can be used to optimize the weight to obtain the final weight of each indicator.

[0112] In a multi-industry evaluation system, for different application scenarios such as production, healthcare, and finance, the present invention automatically adjusts the weights of various indicators based on industry characteristics and evaluation objectives. This dynamic adjustment allows evaluation results to better meet actual needs, ensuring the accuracy and effectiveness of the evaluation system.

[0113] Among them, different industries or application scenarios have different requirements for the importance of each indicator. Dynamic adjustment of weights can make the evaluation more in line with the actual needs of the industry.

[0114] In one feasible implementation, in a production scenario, production efficiency and equipment failure frequency are usually key indicators in production evaluation. In a production environment, production efficiency is crucial to the output value and profits of a company, while the frequency of equipment failure directly affects the continuity of the production process and the availability of the equipment. In medical scenarios, special attention is usually paid to safety and patient health indicators. For example, the incidence of medical accidents and the success rate of treatment may have a higher weight because the patient's life and health are the core of the evaluation. Cost control and resource utilization may have a lower weight in medical scenarios, especially in public health systems, where safety and health outcomes are the priority factors.

[0115] S6. Perform a weighted summation of the weight value of each evaluation indicator and the score corresponding to each evaluation indicator to obtain a comprehensive total score of the object to be evaluated.

[0116] Optionally, the comprehensive total score of the evaluation object is used to compare the advantages and disadvantages of different evaluation objects, or to perform a qualitative evaluation of a single evaluation object.

[0117] Among them, the qualitative evaluation of a single assessment object can be judged as excellent, good and qualified according to the score range.

[0118] Optionally, the process of weighting and summing the weight value of each indicator and the score corresponding to each indicator is expressed by the following formula (1):

[0119] (1)

[0120] in, Indicates the comprehensive total score of the object to be evaluated; represents the weight of the i-th evaluation indicator; represents the i-th evaluation indicator.

[0121] S7. Input the comprehensive score of the object to be evaluated into the trained large language model to obtain the final evaluation result.

[0122] in, Figure 2This is a structural diagram of the implementation of an automated intelligent evaluation method based on a large language model provided by an embodiment of the present invention; in a feasible implementation method, descriptive text data and a task to be evaluated are obtained. The descriptive text data and the task to be evaluated are input into an indicator system construction module based on a large model, and multiple evaluation indicators of the evaluation task are generated through the large language model; the multiple evaluation indicators and descriptive text data are input into an indicator score calculation module based on a large model, the descriptive text data is matched with each evaluation indicator, and the score corresponding to each evaluation indicator is calculated through a predefined algorithm; the multiple evaluation indicators are input into an indicator weight calculation module based on a large model, and the weight value of each evaluation indicator is calculated using the hierarchical analysis method; the weight value of each evaluation indicator and the score corresponding to each evaluation indicator are weighted and summed to obtain a comprehensive total score of the object to be evaluated; the comprehensive score of the object to be evaluated is input into the trained large language model to obtain the final evaluation result.

[0123] Among them, the embodiments of the present invention can be applied to multiple scenarios such as education field scenarios, enterprise management scenarios, medical field scenarios and legal field scenarios.

[0124] in, Figure 3 This is a structural diagram of an intelligent evaluation method based on an education scenario provided by an embodiment of the present invention. In a feasible implementation method, in an education scenario, multiple dimensions such as article structure, language expression, and content depth are processed separately to generate weights and scores corresponding to the indicators. The scores and weights are combined into a prompt, which is input into a trained large language model to obtain the evaluation results. The automated intelligent evaluation method proposed in the embodiment of the present invention helps to improve evaluation efficiency, quickly process large amounts of text data, reduce subjective errors in manual evaluation, and ensure more accurate and fair results.

[0125] in, Figure 4 It is a structural diagram of an intelligent evaluation method based on the enterprise management field scenario provided by an embodiment of the present invention; in a feasible implementation method, in the enterprise management field scenario, data such as work task reports, project summaries and employee learning records are processed respectively to generate scores and weight values ​​corresponding to the indicators; the scores and weight values ​​corresponding to the indicators are combined into Prompt, and input into the trained large language model to obtain the evaluation results. Among them, the evaluation result can be a specific numerical value, such as employee performance scores and project risk levels, or it can be a detailed evaluation report, including analysis, comprehensive evaluation and suggestions for each indicator. The automated intelligent evaluation method proposed in the embodiment of the present invention helps to objectively and impartially evaluate employees and projects, improves evaluation efficiency, thereby motivating employee development, and contributes to the long-term development of the enterprise.

[0126] in, Figure 5This is a schematic diagram of the structure of an intelligent assessment method based on a medical context, provided by an embodiment of the present invention. In one feasible implementation, in a medical context, data from electronic medical records, medical literature databases, clinical testing equipment, and other data are processed to generate scores and weights corresponding to indicators. These scores and weights are combined into a prompt and input into a trained large language model to obtain an assessment result. The automated intelligent assessment method proposed in this invention facilitates further interpretation of disease assessment results to generate diagnostic recommendations, thereby improving diagnostic accuracy and efficiency.

[0127] in, Figure 6 This is a schematic diagram of the structure of an intelligent evaluation method based on a legal field scenario provided by an embodiment of the present invention. In a feasible implementation, in a legal field scenario, data such as legal knowledge, legal documents, legal works, legal journals, judicial practices and cases are processed to generate scores and weights corresponding to the indicators. The scores and weights corresponding to the indicators are combined into a prompt and input into a trained large language model to obtain the final evaluation results. The automated intelligent evaluation method proposed in this embodiment of the present invention helps to accelerate the case processing process in the legal field, improve the speed of legal retrieval and knowledge integration, thereby improving the quality of legal services and promoting the development of legal research and education.

[0128] The embodiment of the present invention first obtains descriptive text data and evaluation tasks of an object to be evaluated; secondly, constructs an automated intelligent evaluation system based on a large language model; the system includes: an indicator system construction module based on the large model, an indicator score calculation module based on the large model, and an indicator weight calculation module based on the large model; the descriptive text data and the evaluation task are input into the indicator system construction module based on the large model, and multiple evaluation indicators of the evaluation task are generated through the large language model; the multiple evaluation indicators and the descriptive text data are input into the indicator score calculation module based on the large model, the descriptive text data is matched with each evaluation indicator, and the score corresponding to each evaluation indicator is calculated through a predefined algorithm; finally, the multiple evaluation indicators are input into the indicator weight calculation module based on the large model, and the hierarchical analysis method is used to calculate the weight value of each evaluation indicator; the weight value of each evaluation indicator and the score corresponding to each evaluation indicator are weighted and summed to obtain a comprehensive total score of the object to be evaluated; the comprehensive score of the object to be evaluated is input into the trained large language model to obtain a final evaluation result.

[0129] The embodiments of the present invention do not require a large amount of expert experience, and at the same time have low requirements on the structuring degree of the data or information of the evaluation object. The overall implementation cost is low and the efficiency is high, which can effectively solve the problems existing in traditional evaluation methods and systems.

[0130] Figure 7This is a block diagram of an automated intelligent evaluation system based on a large language model according to an exemplary embodiment. The system is used for an automated intelligent evaluation method based on a large language model. Figure 7 The system includes a first acquisition unit 710, a construction unit 720, a generation unit 730, a first calculation unit 740, a second calculation unit 750, a second acquisition unit 760, and a third acquisition unit 770.

[0131] A first acquisition unit 710 is used to acquire descriptive text data of the object to be evaluated and the evaluation task;

[0132] A construction unit 720 is used to construct an automated intelligent evaluation system based on a large language model; the system includes: an indicator system construction module based on the large model, an indicator score calculation module based on the large model, and an indicator weight calculation module based on the large model;

[0133] A generating unit 730 is configured to input the descriptive text data and the evaluation task into an indicator system construction module based on a large model, analyze the requirements of the evaluation task and the content of the descriptive text through the large language model, and automatically generate a plurality of evaluation indicators suitable for the task;

[0134] A first calculation unit 740 is configured to input the plurality of evaluation indicators and the descriptive text data into a large model-based indicator score calculation module, match the descriptive text data with each evaluation indicator, and calculate a score corresponding to each evaluation indicator using a predefined algorithm;

[0135] The second calculation unit 750 is used to input multiple evaluation indicators into the indicator weight calculation module based on the large model and calculate the weight value of each evaluation indicator using the hierarchical analysis method;

[0136] The second obtaining unit 760 is configured to perform weighted summation on the weight value of each evaluation indicator and the score corresponding to each evaluation indicator to obtain a comprehensive total score of the object to be evaluated;

[0137] The third acquisition unit 770 is used to input the comprehensive score of the object to be evaluated into the trained large language model to obtain the final evaluation result.

[0138] Optionally, the process of performing weighted summation of the weight value of each indicator and the score corresponding to each indicator is represented by the following formula (1):

[0139] (1)

[0140] in, Indicates the comprehensive total score of the object to be evaluated; represents the weight of the i-th evaluation indicator; represents the i-th evaluation indicator.

[0141] Optionally, the comprehensive total score of the evaluation object is used to compare the advantages and disadvantages of different evaluation objects, or to perform a qualitative evaluation on a single evaluation object.

[0142] Optionally, the large model-based indicator system construction module is used to generate an indicator system for the evaluation task based on the input descriptive text data of the object to be evaluated.

[0143] The large model-based indicator score calculation module is used to generate a score for each evaluation indicator for each indicator of the indicator system of the evaluation task;

[0144] The large-model-based indicator weight calculation module is used to determine the weight of each evaluation indicator in the comprehensive evaluation.

[0145] Optionally, the predefined algorithm is based on predefined rules, including: percentage calculation, threshold level, segment score and level score.

[0146] Optionally, the second calculating unit 750 is configured to:

[0147] Set the basic rules for weight calculation and assign initial weights to the obtained indicators;

[0148] Based on the obtained indicators, an importance matrix is ​​constructed and the weight of each indicator is calculated using the matrix decomposition algorithm;

[0149] The analytic hierarchy process is used to optimize the weight of each indicator and obtain the final weight of each indicator.

[0150] The embodiment of the present invention first obtains descriptive text data and evaluation tasks of an object to be evaluated; secondly, constructs an automated intelligent evaluation system based on a large language model; the system includes: an indicator system construction module based on the large model, an indicator score calculation module based on the large model, and an indicator weight calculation module based on the large model; the descriptive text data and the evaluation task are input into the indicator system construction module based on the large model, and multiple evaluation indicators of the evaluation task are generated through the large language model; the multiple evaluation indicators and the descriptive text data are input into the indicator score calculation module based on the large model, the descriptive text data is matched with each evaluation indicator, and the score corresponding to each evaluation indicator is calculated through a predefined algorithm; finally, the multiple evaluation indicators are input into the indicator weight calculation module based on the large model, and the hierarchical analysis method is used to calculate the weight value of each evaluation indicator; the weight value of each evaluation indicator and the score corresponding to each evaluation indicator are weighted and summed to obtain a comprehensive total score of the object to be evaluated; the comprehensive score of the object to be evaluated is input into the trained large language model to obtain a final evaluation result.

[0151] The embodiments of the present invention do not require a large amount of expert experience, and at the same time have low requirements on the structuring degree of the data or information of the evaluation object. The overall implementation cost is low and the efficiency is high, which can effectively solve the problems existing in traditional evaluation methods and systems.

[0152] Figure 8 This is a schematic diagram of the structure of an automated intelligent evaluation device based on a large language model provided by an embodiment of the present invention. Figure 8 As shown, the automated intelligent evaluation device based on the large language model may include the above Figure 7 The automated intelligent evaluation system based on a large language model is shown. Optionally, the automated intelligent evaluation device 810 based on a large language model may include a first processor 2001 .

[0153] Optionally, the large language model-based automated intelligent assessment device 810 may further include a memory 2002 and a transceiver 2003 .

[0154] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

[0155] The following combination Figure 8 The components of the large language model-based automated intelligent assessment device 810 are described in detail:

[0156] The first processor 2001 is the control center of the large language model-based automated intelligent assessment device 810 and can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), or application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs) or one or more field programmable gate arrays (FPGAs).

[0157] Optionally, the first processor 2001 may execute various functions of the large language model-based automated intelligent assessment device 810 by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002 .

[0158] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 8CPU0 and CPU1 are shown in FIG.

[0159] In a specific implementation, as an embodiment, the automatic intelligent evaluation device 810 based on the large language model may also include multiple processors, such as Figure 8 1 and 2. The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). A processor herein can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0160] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled by the first processor 2001 for execution. The specific implementation method can refer to the above method embodiment and will not be repeated here.

[0161] Alternatively, the memory 2002 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, an optical disc storage (including a compact disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001 or exist independently and accessed through the interface circuit ( Figure 8 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0162] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0163] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 8 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0164] Optionally, the transceiver 2003 may be integrated with the first processor 2001 or may exist independently and be connected to the first processor 2001 through the interface circuit ( Figure 8 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0165] It should be noted that Figure 8 The structure of the large language model-based automated intelligent evaluation device 810 shown in the figure does not constitute a limitation on the router. The actual knowledge structure recognition device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0166] In addition, the technical effects of the large language model-based automated intelligent evaluation device 810 can refer to the technical effects of the large language model-based automated intelligent evaluation method described in the above method embodiment, and will not be repeated here.

[0167] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, etc.

[0168] It should also be understood that the memory in the embodiments of the present invention may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0169] The above embodiments can be implemented in whole or in part via software, hardware (e.g., circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product comprises one or more computer instructions or computer programs. When loaded or executed on a computer, the processes or functions described in accordance with the embodiments of the present invention are fully or partially performed. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable system. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired means (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium accessible by a computer or a data storage device such as a server or data center that contains a collection of one or more available media. The available medium can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media. The semiconductor media can be a solid-state drive.

[0170] It should be understood that the term "and / or" as used herein simply describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A alone, A and B together, or B alone. A and B can be singular or plural. Furthermore, the character " / " as used herein generally indicates an "or" relationship between the associated objects, but it may also indicate an "and / or" relationship. For specific understanding, please refer to the context.

[0171] In this disclosure, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, "at least one of a, b, or c" can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or plural.

[0172] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0173] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0174] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0175] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, systems, and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of the system or unit, which can be electrical, mechanical or other forms.

[0176] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0177] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0178] If the functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or the portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include various media that can store program code, such as USB flash drives, mobile hard drives, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical disks.

[0179] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. An automated intelligent evaluation method based on a large language model, characterized in that: The method comprises: S1. Obtain descriptive text data and evaluation tasks of the object to be evaluated; S2. Constructing an automated intelligent evaluation system based on a large language model; the system includes: an indicator system construction module based on the large model, an indicator score calculation module based on the large model, and an indicator weight calculation module based on the large model; S3. Inputting the descriptive text data and the evaluation task into an indicator system construction module based on a large model, analyzing the requirements of the evaluation task and the content of the descriptive text through the large language model, and automatically generating multiple evaluation indicators suitable for the task; S4, inputting the multiple evaluation indicators and the descriptive text data into a large model-based indicator score calculation module, matching the descriptive text data with each evaluation indicator, and calculating the score corresponding to each evaluation indicator through a predefined algorithm; S5. Input multiple evaluation indicators into the indicator weight calculation module based on the large model, and use the hierarchical analysis method to calculate the weight value of each evaluation indicator; S6. Perform a weighted summation of the weight value of each evaluation indicator and the score corresponding to each evaluation indicator to obtain a comprehensive total score of the object to be evaluated; S7. Input the comprehensive score of the object to be evaluated into the trained large language model to obtain the final evaluation result.

2. The automated intelligent evaluation method based on a large language model according to claim 1, characterized in that: The process of weighting and summing the weight value of each indicator and the score corresponding to each indicator is expressed by the following formula (1): （1） in, Indicates the comprehensive total score of the object to be evaluated; represents the weight of the i-th evaluation indicator; represents the i-th evaluation indicator.

3. The automated intelligent evaluation method based on a large language model according to claim 1, characterized in that: The comprehensive total score of the evaluation object is used to compare the advantages and disadvantages of different evaluation objects, or to perform qualitative evaluation on a single evaluation object.

4. The automated intelligent evaluation method based on a large language model according to claim 1, characterized in that: The indicator system construction module based on the large model is used to generate an indicator system for the evaluation task based on the input descriptive text data of the object to be evaluated; The large model-based indicator score calculation module is used to generate a score for each evaluation indicator for each indicator of the indicator system of the evaluation task; The large-model-based indicator weight calculation module is used to determine the weight of each evaluation indicator in the comprehensive evaluation.

5. The automated intelligent evaluation method based on a large language model according to claim 1, characterized in that: The predefined algorithm is based on predefined rules, including percentage calculation, threshold level, segmentation score and level score.

6. The automated intelligent evaluation method based on a large language model according to claim 1, characterized in that: The step S5 inputs multiple evaluation indicators into an indicator weight calculation module based on a large model, and uses the hierarchical analysis method to calculate the weight value of each evaluation indicator, including: S51. Setting basic rules for weight calculation and assigning initial weights to the obtained indicators; S52. Based on the obtained indicators, an importance matrix is ​​constructed, and the weight of each indicator is calculated using a matrix decomposition algorithm; S53. Use the hierarchical analysis method to optimize the weight of each indicator and obtain the final weight of each indicator.

7. An automated intelligent evaluation system based on a large language model, wherein the automated intelligent evaluation system based on a large language model is used to implement the automated intelligent evaluation method based on a large language model according to any one of claims 1 to 6, characterized in that: The system comprises: A first acquisition unit is used to acquire descriptive text data of the object to be evaluated and the evaluation task; A construction unit for constructing an automated intelligent evaluation system based on a large language model; the system includes: an indicator system construction module based on the large model, an indicator score calculation module based on the large model, and an indicator weight calculation module based on the large model; A generating unit, configured to input the descriptive text data and the evaluation task into an indicator system construction module based on a large model, analyze the requirements of the evaluation task and the content of the descriptive text through the large language model, and automatically generate a plurality of evaluation indicators suitable for the task; A first calculation unit is configured to input the plurality of evaluation indicators and the descriptive text data into a large model-based indicator score calculation module, match the descriptive text data with each evaluation indicator, and calculate a score corresponding to each evaluation indicator using a predefined algorithm; The second calculation unit is used to input multiple evaluation indicators into the indicator weight calculation module based on the large model, and calculate the weight value of each evaluation indicator using the hierarchical analysis method; The second obtaining unit is used to perform weighted summation on the weight value of each evaluation indicator and the score corresponding to each evaluation indicator to obtain a comprehensive total score of the object to be evaluated; The third acquisition unit is used to input the comprehensive score of the object to be evaluated into the trained large language model to obtain the final evaluation result.

8. The automated intelligent evaluation system based on a large language model according to claim 7, characterized in that: The process of weighting and summing the weight value of each indicator and the score corresponding to each indicator is expressed by the following formula (1): （1） in, Indicates the comprehensive total score of the object to be evaluated; represents the weight of the i-th evaluation indicator; represents the i-th evaluation indicator.

9. An automated intelligent evaluation device based on a large language model, characterized in that: The automated intelligent evaluation device based on the large language model includes: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 6 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program code, which can be called by a processor to execute the method according to any one of claims 1 to 6.

Images